Learning device, image generation device, learning method, image generation method, learning program, and image generation program

Abstract

To provide a learning device, an image generation device, a learning method, an image generation method, a learning program and an image generation program which can generate a pseudo two-dimensional image that accurately reproduces a shape of a lesion of a breast.SOLUTION: A learning device detects a first region of interest including calcification and a second region of interest including another lesion on the basis of any of a composite two-dimensional image, a tomographic image and a normal two-dimensional image obtained from a plurality of projection images or a plurality of tomographic images with tomosynthesis imaging, updates a weight of a network of an image generation model on the basis of a pseudo two-dimensional image and the normal two-dimensional image or the composite two-dimensional image or both of them output by the image generation model in which a weight of the first region of interest is set to be the heaviest and a weight of the second region of interest is set to be heavier than a weight of a region other than the first region of interest and the second region of interest, and reduces a loss to learn the image generation model.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to a learning device, an image generating device, a learning method, an image generating method, a learning program, and an image generating program. [Background technology]

[0002] There is known a technique for generating a radiological image equivalent to a normal two-dimensional image obtained by normal imaging by synthesizing a series of multiple projection images obtained by tomosynthesis imaging performed by irradiating the breast with radiation or multiple tomographic images generated from a series of multiple projection images. For example, Patent Document 1 describes a technique for generating a pseudo two-dimensional image from projection images acquired by tomosynthesis imaging using a trained model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-141867 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 sometimes does not have sufficient accuracy in reproducing the fine shapes of lesions such as calcifications in ordinary 2D images. Therefore, there is a demand for a technology for generating pseudo 2D images that accurately reproduce the fine shapes of lesions such as calcifications.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide a learning device, an image generation device, a learning method, an image generation method, a learning program, and an image generation program that are capable of generating pseudo-two-dimensional images that accurately reproduce the shape of breast lesions. [Means for solving the problem]

[0006] In order to achieve the above object, a learning device according to a first aspect of the present disclosure is a learning device for an image generation model that generates a pseudo two-dimensional image from a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images, and includes at least one processor, which acquires normal two-dimensional images captured by irradiating the breast with radiation, and generates a synthetic two-dimensional image obtained by synthesizing at least a portion of the series of multiple projection images or multiple tomographic images. ,or Regular 2D image from The first region of interest containing the breast calcification was and from the tomographic image. A second region of interest including lesions other than calcification is detected, and the image generation model is trained by updating the weights of the network of the image generation model to reduce the loss based on the loss between the pseudo 2D image output by the image generation model in which the weight of the first region of interest is set to be the heaviest and the weight of the second region of interest is set to be equal to or greater than the weights of other regions other than the first region of interest and the second region of interest, and the normal 2D image or the synthetic 2D image or both.

[0008] The present disclosure 2 The learning device of this embodiment is a first embodiment. Dear In the learning device, the processor updates the network weights differently depending on the type of other lesion.

[0009] The present disclosure 3 The learning device of the embodiment is the first embodiment or the second aspect In the learning device, the processor typically derives the glandular volume of the breast from a two-dimensional image, and adjusts at least one value of the weight of the first region of interest, the weight of the second region of interest, and the weight of the third region of interest according to the derived glandular volume.

[0010] The present disclosure 4 The learning device of this aspect is any one of the first to third aspects. 3 In the learning device of any one of the above aspects, the processor corrects the positional deviation between a series of multiple projection images or multiple tomographic images and a normal two-dimensional image, and inputs the corrected series of multiple projection images or multiple tomographic images into an image generation model to perform learning.

[0011] In order to achieve the above object, an image generating device according to a fifth aspect of the present disclosure is an image generating device for generating a pseudo two-dimensional image using an image generation model trained by the learning device of the present disclosure, and includes at least one processor, wherein the processor acquires a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images as images for generating the pseudo two-dimensional image, synthesizes at least some of the images for generation to generate a synthetic two-dimensional image, and Therefore, it is at least one of calcification and other lesions. Detecting a region of interest containing a lesion; This is a part of the image to be generated. An image of the region of interest is input to an image generation model, a pseudo-2D image of the region of interest is output from the image generation model, and the synthetic 2D image and the pseudo-2D image of the region of interest are combined to generate a pseudo-2D image.

[0012] In order to achieve the above object, the first aspect of the present disclosure 6 The learning method of the embodiment is a learning method for an image generation model that generates a pseudo two-dimensional image from a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from a series of multiple projection images, and obtains normal two-dimensional images photographed by irradiating a breast with radiation, and synthesizes at least a part of the series of multiple projection images or multiple tomographic images to generate a synthetic two-dimensional image. ,or Regular 2D image from The first region of interest containing the breast calcification was and from the tomographic image. This is a learning method in which a computer executes a process to detect a second region of interest that includes a lesion other than calcification, and based on the loss between a pseudo 2D image output by an image generation model in which the weight of the first region of interest is set to be the heaviest and the weight of the second region of interest is set to be equal to or greater than the weights of other regions other than the first region of interest and the second region of interest, and a normal 2D image or a synthetic 2D image, or both, thereby updating the weights of the network of the image generation model to reduce the loss, thereby learning the image generation model.

[0013] In order to achieve the above object, an image generation method according to a seventh aspect of the present disclosure is an image generation method for generating a pseudo two-dimensional image using an image generation model trained by a learning device according to the present disclosure, the method comprising: acquiring a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images as images for generating the pseudo two-dimensional image; synthesizing at least a portion of the images for generation to generate a synthetic two-dimensional image; Therefore, it is at least one of calcification and other lesions. Detecting a region of interest containing a lesion; This is a part of the image to be generated. This is an image generation method in which a computer inputs an image of a region of interest into an image generation model, obtains a pseudo-2D image of the region of interest output from the image generation model, and combines the composite 2D image with the pseudo-2D image of the region of interest to generate a pseudo-2D image.

[0014] In order to achieve the above object, the first aspect of the present disclosure 8 The learning program of the present embodiment is a learning program for causing a computer to execute a process of learning an image generation model that generates a pseudo two-dimensional image from a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images, and the learning program acquires normal two-dimensional images photographed by irradiating a breast with radiation, and generates a synthetic two-dimensional image obtained by synthesizing at least a part of the series of multiple projection images or multiple tomographic images. ,or Regular 2D image from The first region of interest containing the breast calcification was and from the tomographic image. The computer is caused to execute a process of detecting a second region of interest that includes a lesion other than calcification, and based on the loss between a pseudo 2D image output by an image generation model in which the weight of the first region of interest is set to be the heaviest and the weight of the second region of interest is set to be equal to or greater than the weights of other regions other than the first region of interest and the second region of interest, updating the weights of the network of the image generation model to reduce the loss, thereby learning the image generation model.

[0015] In order to achieve the above object, an image generation program according to a ninth aspect of the present disclosure is an image generation program for causing a computer to execute a process of generating a pseudo two-dimensional image using an image generation model learned by a learning device according to the present disclosure, the image generation program acquiring a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images as images for generating the pseudo two-dimensional image, synthesizing at least a portion of the images for generation to generate a synthetic two-dimensional image, and Therefore, it is at least one of calcification and other lesions. Detecting a region of interest containing a lesion; This is a part of the image to be generated. The method involves inputting an image of the region of interest into an image generation model, obtaining a pseudo-2D image of the region of interest output from the image generation model, and causing a computer to execute a process of combining the composite 2D image with the pseudo-2D image of the region of interest to generate a pseudo-2D image. [Effects of the Invention]

[0016] According to the present disclosure, it is possible to generate a pseudo two-dimensional image that accurately reproduces the shape of a breast lesion. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a configuration diagram schematically illustrating an example of the overall configuration of a radiation image capturing system according to an embodiment; [Figure 2] FIG. 1 is a diagram illustrating an example of tomosynthesis imaging. [Figure 3] FIG. 1 is a diagram illustrating an example of an image generation model. [Figure 4] FIG. 4 is a diagram illustrating an example of an intermediate layer of the image generation model illustrated in FIG. 3. [Figure 5] FIG. 1 is a block diagram illustrating an example of a configuration of an image processing apparatus according to an embodiment. [Figure 6] FIG. 1 is a schematic diagram illustrating an outline of a learning flow of an image generation model in an image processing apparatus according to an embodiment. [Figure 7] FIG. 2 is a functional block diagram of an example of a configuration related to a function for generating an image generation model in the image processing apparatus according to the embodiment. [Figure 8] 10 is a flowchart illustrating an example of the flow of a learning process performed by the image processing apparatus according to the embodiment. [Figure 9] 1 is a schematic diagram for explaining an outline of a flow of generating a pseudo two-dimensional image using an image generation model in an image processing apparatus according to an embodiment. [Figure 10] 2 is a functional block diagram of an example of a configuration related to a function for generating a pseudo two-dimensional image in the image processing apparatus according to the embodiment. FIG. [Figure 11] 10 is a flowchart illustrating an example of the flow of an image generation process performed by the image processing apparatus according to the embodiment. [Figure 12] FIG. 10 is a schematic diagram for explaining an outline of the flow of learning an image generation model in the image processing device of the first modified example. [Figure 13] FIG. 10 is a schematic diagram for explaining an outline of the flow of learning an image generation model in the image processing device of Modification 2. [Figure 14] FIG. 11 is a schematic diagram for explaining an outline of the flow of learning an image generation model in the image processing device of Modification 3. [Figure 15] 13 is a flowchart illustrating an example of the flow of a learning process by an image processing device according to a fourth modification. [Figure 16] 13 is a flowchart illustrating an example of the flow of a learning process by an image processing device according to the fifth modification. [Figure 17] FIG. 20 is a schematic diagram for explaining an outline of the flow of learning an image generation model in an image processing device according to a sixth modification. [Figure 18] FIG. 20 is a schematic diagram for explaining an outline of the flow of learning an image generation model in an image processing device according to a seventh modification. [Figure 19] FIG. 20 is a schematic diagram for explaining an outline of the flow of generating a pseudo two-dimensional image using an image generation model in an image processing device of Modification 8. [Figure 20] FIG. 20 is a functional block diagram of an example of a configuration related to a function for generating a pseudo two-dimensional image in an image processing device according to Modification 8. [Figure 21] 13 is a flowchart showing an example of the flow of an image generation process by the image processing device of Modification 8. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the embodiment.

[0019] First, an example of the overall configuration of the radiographic imaging system of this embodiment will be described. Fig. 1 shows a configuration diagram illustrating an example of the overall configuration of the radiographic imaging system 1 of this embodiment. As shown in Fig. 1, the radiographic imaging system 1 of this embodiment includes a mammography apparatus 10, a console 12, a PACS (Picture Archiving and Communication Systems) 14, and an image processing device 16. The console 12, the PACS 14, and the image processing device 16 are connected via a network 17 by wired communication or wireless communication.

[0020] First, the mammography device 10 of this embodiment will be described. Figure 1 shows a side view of an example of the appearance of the mammography device 10 of this embodiment. Note that Figure 1 shows an example of the appearance of the mammography device 10 when viewed from the left side of the subject.

[0021] The mammography device 10 of this embodiment operates under the control of the console 12 and is an apparatus that takes a radiographic image of a subject's breast by irradiating the breast with radiation R (e.g., X-rays) from a radiation source 29. The mammography device 10 of this embodiment also has the function of performing normal imaging, in which imaging is performed with the radiation source 29 positioned at an irradiation position normal to the detection surface 20A of the radiation detector 20, and so-called tomosynthesis imaging (described in detail below), in which imaging is performed by moving the radiation source 29 to each of multiple irradiation positions.

[0022] As shown in FIG. 1, the mammography apparatus 10 includes an imaging table 24, a base 26, an arm 28, and a compression unit 32.

[0023] A radiation detector 20 is disposed inside the imaging table 24. As shown in Fig. 2, in the mammography apparatus 10 of this embodiment, when imaging is performed, the breast U of the subject is positioned on the imaging surface 24A of the imaging table 24 by the user.

[0024] The radiation detector 20 detects radiation R that has passed through the subject's breast U. In detail, the radiation detector 20 detects radiation R that has entered the subject's breast U and the imaging table 24 and reached the detection surface 20A of the radiation detector 20, generates a radiographic image based on the detected radiation R, and outputs image data representing the generated radiographic image. Hereinafter, the series of operations of irradiating radiation R from the radiation source 29 and generating a radiographic image by the radiation detector 20 may be referred to as "imaging." The type of radiation detector 20 in this embodiment is not particularly limited, and may be, for example, an indirect conversion type radiation detector that converts radiation R into light and then converts the converted light into electric charges, or a direct conversion type radiation detector that directly converts radiation R into electric charges.

[0025] The compression plate 30 used to compress the breast when imaging is attached to a compression unit 32 provided on the imaging table 24, and is moved toward or away from the imaging table 24 (hereinafter referred to as the "up and down direction") by a compression plate drive unit (not shown) provided on the compression unit 32. By moving in the up and down direction, the compression plate 30 compresses the breast of the subject between itself and the imaging table 24.

[0026] The arm 28 can rotate relative to the base 26 via the shaft 27. The shaft 27 is fixed to the base 26, and the shaft 27 and arm 28 rotate together. The shaft 27 and the compression unit 32 of the imaging table 24 are each provided with a gear, and by switching between an engaged state and a disengaged state of these gears, the compression unit 32 of the imaging table 24 and the shaft 27 can be switched between a state in which they are connected and rotate together, and a state in which the shaft 27 is separated from the imaging table 24 and rotates freely. Note that the switching between transmitting and disengaging power to the shaft 27 is not limited to the gear, and various mechanical elements can be used. The arm 28 and the imaging table 24 can rotate independently relative to the base 26, with the shaft 27 as the rotation axis.

[0027] When performing tomosynthesis imaging in the mammography device 10, the radiation source 29 is moved sequentially to each of a plurality of irradiation positions with different irradiation angles by the rotation of the arm unit 28. The radiation source 29 has a radiation tube (not shown) that generates radiation R, and the radiation tube is moved to each of the plurality of irradiation positions in accordance with the movement of the radiation source 29. FIG. 2 shows a diagram for explaining an example of tomosynthesis imaging. Note that the compression paddle 30 is not shown in FIG. 2. In this embodiment, as shown in FIG. 2, the radiation source 29 is moved sequentially to each of the irradiation positions 19 with irradiation angles that differ by a predetermined angle β. t (t=1, 2, . . . , the maximum value is 7 in FIG. 2), in other words, the radiation detector 20 is moved to a position where the radiation R irradiates the detection surface 20A of the radiation detector 20 at different irradiation angles. t In the radiation imaging system 1, radiation R is irradiated from the radiation source 29 toward the breast U in response to an instruction from the console 12, and a radiation image is captured by the radiation detector 20. t and move to each of the irradiation positions 19 tIn the example of Fig. 2, when tomosynthesis imaging is performed to capture radiation images at each irradiation position 19, seven radiation images are obtained. In the following, in tomosynthesis imaging, a radiation image captured at each irradiation position 19 is referred to as a "projection image" when distinguishing it from other radiation images, and a plurality of projection images captured in one tomosynthesis imaging is referred to as a "series of a plurality of projection images." In addition to projection images, radiation images, regardless of the type such as tomographic images and normal two-dimensional images, which will be described later, are collectively referred to simply as "radiation images." In the following, each irradiation position 19 is referred to as a "projection image." t Projection images taken at irradiation position 19 t For the images corresponding to the above, the symbols representing each image are followed by the irradiation position 19. t The symbol "t" is added to indicate this.

[0028] 2, the irradiation angle of radiation R refers to the angle α formed between a normal CL to the detection surface 20A of the radiation detector 20 and a radiation axis RC. The radiation axis RC is an axis connecting the focal point of the radiation source 29 at each irradiation position 19 and a predetermined position such as the center of the detection surface 20A. Here, the detection surface 20A of the radiation detector 20 is assumed to be a surface that is approximately parallel to the imaging surface 24A.

[0029] On the other hand, when performing normal imaging in the mammography device 10, the radiation source 29 is positioned at the irradiation position 19 where the irradiation angle α is 0 degrees. t (Irradiation position along the normal direction 19 t 2 ), the radiation source 29 irradiates the patient with radiation R, and the radiation detector 20 captures a radiographic image. In this embodiment, a radiographic image captured in normal radiography is referred to as a "normal two-dimensional image" to distinguish it from other radiographic images.

[0030] The mammography device 10 and the console 12 are connected by wired or wireless communication. A radiographic image captured by the radiation detector 20 in the mammography device 10 is output to the console 12 by wired or wireless communication via a communication I / F (Interface) unit (not shown).

[0031] As shown in FIG. 1, the console 12 of this embodiment includes a control unit 40, a storage unit 42, a user I / F unit 44, and a communication I / F unit 46.

[0032] As described above, the control unit 40 of the console 12 has the function of controlling the capture of radiographic images of the breast by the mammography device 10. The control unit 40 may be, for example, a computer system equipped with a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory).

[0033] The storage unit 42 has a function of storing information related to radiographic image capture, radiographic images acquired from the mammography apparatus 10, etc. The storage unit 42 is a non-volatile storage unit, such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0034] The user I / F unit 44 includes input devices such as various buttons and switches operated by users such as technicians in relation to radiographic imaging, and display devices such as lamps and displays that display information about imaging, radiographic images, etc.

[0035] The communication I / F unit 46 communicates various data, such as information related to radiographic image capture and radiographic images obtained by capture, with the mammography apparatus 10 via wired or wireless communication. The communication I / F unit 46 also communicates various data, such as radiographic images, with the PACS 14 and image processing device 16 via the network 17 via wired or wireless communication.

[0036] 1, the PACS 14 of this embodiment includes a storage unit 50 and a communication I / F unit (not shown) that store a radiographic image group 52. The radiographic image group 52 includes radiographic images captured by the mammography device 10 and acquired from the console 12 via the communication I / F unit (not shown).

[0037] The image processing device 16 is used when a doctor or the like (hereinafter simply referred to as "doctor") interprets a radiological image. The image processing device 16 of this embodiment has a function of generating a pseudo two-dimensional image equivalent to a normal two-dimensional image from a plurality of tomographic images using an image generation model. The image processing device 16 of this embodiment is an example of a learning device of the present disclosure.

[0038] First, an example of an image generation model used to generate a pseudo two-dimensional image in the image processing device 16 of this embodiment will be described. FIG. 3 shows an example of an image generation model 66 of this embodiment. The image generation model 66 of this embodiment uses a convolutional neural network (CNN) that has undergone machine learning through deep learning. The image generation model 66 generates a plurality of tomographic images 100 (k images in FIG. 3: tomographic images 1001 to 1002) obtained from a series of projection images. k ) is input and a pseudo two-dimensional image 102 is output.

[0039] 3 includes an input layer 200, an intermediate layer 202, and an output layer 204, each of which is provided for each of the k tomographic images 100. In this embodiment, the input layers 200 provided for each of the tomographic images 100 have the same configuration.

[0040] A tomographic image 100 is input to the input layer 200. The input layer 200 includes a plurality of nodes 300, each corresponding to a pixel of the tomographic image 100. The input layer 200 performs convolution processing when propagating information for each pixel (each pixel) of the input tomographic image 100 to the intermediate layer 202. For example, if the size of the image to be processed is 28 pixels x 28 pixels and is grayscale data, the size of the data propagated from the input layer 200 to the intermediate layer 202 is 28 x 28 x 1 = 784.

[0041] The present invention is not limited to this embodiment, and information may be input to the input layer 200 in units of voxels extracted from a plurality of tomographic images 100 .

[0042] As shown in Fig. 4, the intermediate layer 202 includes an encoder 203e and a decoder 203d. The encoder 203e includes a plurality of convolutional layers conv that perform convolutional processing and pooling layers pool that perform pooling processing, the number of which corresponds to the number of layers of the encoder 203e (the number of layers illustrated in the encoder 203e in Fig. 4 is "2"). Note that the plurality of nodes 3021 illustrated in Fig. 3 correspond to the plurality of nodes included in the first layer convolutional layer conv included in the encoder 203e.

[0043] In the convolution process, a three-dimensional convolution operation is performed on each pixel to output a pixel value Icp(x, y, z) corresponding to each pixel of interest Ip. In this way, three-dimensional output data Dc is output, which includes multiple pieces of output data DIc having pixel values ​​Icp(x, y, z) arranged two-dimensionally. One piece of output data Dc is output for each 3x3x3 filter F. When multiple filters F of different types are used, output data Dc is output for each filter F. The filter F represents a neuron (node) in the convolution layer, and since the features that can be extracted are determined for each filter F, the number of features that can be extracted from one piece of input data D in the convolution layer is the number of filters F.

[0044] In addition, the pooling layer pool performs a pooling process that reduces the original image while preserving its features. In other words, the pooling layer pool performs a pooling process that selects a local representative value, reduces the resolution of the input image, and reduces the image size. For example, if the pooling process, which selects a representative value from a 2x2 pixel block, is performed with a stride of "1," that is, shifted by one pixel, a reduced image that is half the size of the input image is output.

[0045] On the other hand, the decoder 203d includes a plurality of convolution layers conv that perform convolution processing and upsampling layers upsmp that perform upsampling processing, in accordance with the number of layers of the decoder 203d (the number of layers illustrated in the decoder 203d in FIG. 4 is "2"). u correspond to a plurality of nodes included in the first layer convolutional layer conv included in the decoder 203d.

[0046] The convolution layer conv included in the decoder 203d performs the same processing as the convolution layer conv included in the encoder 203e. On the other hand, the upsampling layer upsmp receives the output of the encoder 203e as input and performs processing so that the image quality of the pseudo-synthesized 2D image becomes equal to that of the encoder 203e.

[0047] On the other hand, the output layer 204 is composed of all nodes 302 included in the convolution layer conv arranged at the end of the intermediate layer 202. u The image size in the output layer 204 is the same as the size of the pseudo 2D image 102 output from the image generation model 66, and multiple nodes 304 included in the output layer 204 correspond to each pixel of the pseudo 2D image 102.

[0048] In this way, the image generation model 66 of this embodiment outputs a pseudo two-dimensional image 102 when a plurality of tomographic images 100 are input.

[0049] Fig. 5 shows a block diagram illustrating an example of the configuration of the image processing device 16 of this embodiment. As shown in Fig. 5, the image processing device 16 of this embodiment includes a control unit 60, a storage unit 62, a display unit 70, an operation unit 72, and a communication I / F unit 74. The control unit 60, the storage unit 62, the display unit 70, the operation unit 72, and the communication I / F unit 74 are connected via a bus 79 such as a system bus or a control bus so that various information can be exchanged between them.

[0050] The control unit 60 controls the overall operation of the image processing device 16. The control unit 60 includes a CPU 60A, a ROM 60B, and a RAM 60C. The ROM 60B stores various programs and the like for control by the CPU 60A in advance. The RAM 60C temporarily stores various data.

[0051] The storage unit 62 is a non-volatile storage unit, and specific examples include an HDD, an SSD, etc. The storage unit 62 stores various information such as a learning program 63A, an image generation program 63B, learning data 64 for learning an image generation model 66, the image generation model 66 described above, and a region of interest detector 68 (described in detail later).

[0052] The display unit 70 displays radiographic images and various types of information. The display unit 70 is not particularly limited and may be any of various displays. The operation unit 72 is used by the user to input instructions and various types of information for a doctor to diagnose breast lesions using radiographic images. The operation unit 72 is not particularly limited and may be, for example, any of various switches, a touch panel, a touch pen, a mouse, etc. The display unit 70 and the operation unit 72 may be integrated into a touch panel display.

[0053] The communication I / F unit 74 communicates various types of information with the console 12 and the PACS 14 via the network 17 by wireless or wired communication.

[0054] The functions of the image processing device 16 of this embodiment will be described below. There is a learning phase in which the image generation model 66 is trained, and an operation phase in which the image generation model 66 is used to generate a pseudo two-dimensional image from a plurality of tomographic images.

[0055] (Learning phase) First, an example of the learning phase of the image processing device 16 of this embodiment will be described. Fig. 6 shows a schematic diagram for explaining an outline of the learning flow of the image generation model 66 in the image processing device 16 of this embodiment.

[0056] As shown in Figure 6, the learning data 64 consists of a set of a normal two-dimensional image 111 obtained by normal breast imaging and multiple tomographic images 101 obtained from a series of multiple projection images obtained by tomosynthesis imaging of the same breast.

[0057] In the learning phase, a plurality of tomographic images of the learning data 64 are input to the image generation model 66. The image generation model 66 outputs the pseudo two-dimensional image 103 as described above.

[0058] Loss function calculation unit 85 calculates a loss function, which is a function that represents the degree of difference between pseudo 2D image 103 output from image generation model 66 and normal 2D image 111. The closer the value of the loss function is to 0, the more similar pseudo 2D image 103 is to normal 2D image 111, and the more accurately the shape of the lesion is reproduced in pseudo 2D image 103.

[0059] The loss function calculation unit 85 of this embodiment calculates, as a loss function, the sum of values ​​obtained by multiplying the differences between the pseudo 2D image 103 and each of the following regions in the normal 2D image 111: a first region of interest 1111 that includes breast calcification P1, a second region of interest 1112 that includes other lesions P2 other than calcification P1, and other regions 1113, by weights appropriate for each region. Specifically, the loss function calculation unit 85 calculates the loss function based on the following equation (1): Difference between pseudo two-dimensional image 103 and first region of interest 1111 × weight W1111 + difference between pseudo two-dimensional image 103 and second region of interest 1112 × weight W1112 + difference between pseudo two-dimensional image 103 and other region 1113 × weight W1112 (1)

[0060] In the above equation (1), the weight W1111 is the weight of the first region of interest 1111 and is a weight corresponding to the calcification P1. The weight W1112 is the weight of the second region of interest 1112 and is a weight corresponding to the other lesion P2. Furthermore, the weight W1113 is the weight of the other region 1113 and is a weight corresponding to other, for example, normal tissue.

[0061] In the image generation model 66 of this embodiment, weights for each region of interest are preset such that the weight W1111 of the first region of interest 1111 is the heaviest and is equal to or greater than the weight W1112 of the second region of interest 1112 and the weight W1113 of the other region 1113. In other words, the weights W1111 to W1113 in the image generation model 66 satisfy the relationship of the following equation (2). Weight W1111 > Weight W1112 ≥ Weight W1113 (2)

[0062] Among lesions, calcifications in particular tend to be smaller and more detailed than other lesions. Furthermore, the shape of calcifications is an important finding when a doctor diagnoses a lesion. Therefore, as shown in the relationship of the above formula (2), the image processing device 16 assigns the heaviest weight W1111 to the first region of interest 1111 including the calcification P1, thereby focusing on the calcification P1 among the lesions in particular, rather than other lesions, normal tissue, etc., when training the image generation model 66.

[0063] Furthermore, as shown in the relationship of equation (2) above, the image processing device 16 can prevent a decrease in reproduction accuracy for other lesions by training the image generation model 66 by setting the weight W1112 of the second region of interest 1112, which includes other lesions P2 other than calcification P1, to be equal to or greater than the weight W1113 of the other region 1113.

[0064] Therefore, a first region of interest 1111, a second region of interest 1112, and other regions 1113 are detected from the normal two-dimensional image 111 by the region of interest detector 68.

[0065] As an example, in this embodiment, a detector using a known CAD (Computer-Aided Diagnosis) algorithm is used as region of interest detector 68. In the CAD algorithm, a probability (likelihood) that a pixel in normal two-dimensional image 111 represents calcification P1 is derived, and pixels for which this probability is equal to or greater than a predetermined threshold are detected as calcification P1. A region including calcification P1 is detected as first region of interest 1111. Similarly, a probability (likelihood) that a pixel in normal two-dimensional image 111 represents a lesion other than calcification P2, such as a tumor (hereinafter referred to as "other lesion"), is derived, and pixels for which this probability is equal to or greater than a predetermined threshold are detected as other lesion P2. A region including other lesion P2 is detected as second region of interest 1112. Note that region of interest detector 68 for detecting calcification P1 and region of interest detector 68 for detecting other lesion P2 may be a single detector or separate detectors. In other words, the CAD in the region of interest detector 68 may be a CAD for detecting calcification P1 and other lesions P2, or may be a combination of a CAD for detecting calcification P1 and a CAD for detecting other lesions P2.

[0066] In the example shown in FIG. 6 , both the first region of interest 1111 and the second region of interest 1112 are rectangular regions surrounding the calcification P1 and the other lesion P2, respectively, and strictly speaking, also include breast tissue other than the calcification P1 and the other lesion P2. Note that, in this embodiment, each of the first region of interest 1111 and the second region of interest 1112 may be a region including only the calcification P1 and the other lesion P2. Furthermore, the region of the normal two-dimensional image 111 excluding the first region of interest 1111 and the second region of interest 1112 is identified as the other region 1113. Hereinafter, the first region of interest 1111 and the second region of interest 1112 will be collectively referred to simply as the "regions of interest."

[0067] For example, the relationship of the above formula (2) may be satisfied for the weights of all pixels included in each of the first region of interest 1111, the second region of interest 1112, and the other region 1113. Furthermore, for example, the relationship of the above formula (2) may be satisfied for the average value of the weights for the pixels included in the first region of interest 1111, the average value of the weights for the pixels included in the second region of interest 1112, and the average value of the weights for the pixels included in the other region 1113.

[0068] The weight update unit 86 updates the weights of the network in the image generation model 66 in accordance with the loss function calculated by the loss function calculation unit 85. Specifically, in accordance with the loss function calculated by the loss function calculation unit 85, the weights indicating the strength of connections between nodes in the previous and next layers, which are coefficients of each filter F in the image generation model 66 described above, and the weight w of the difference connecting a node 304 in the output layer 204 to a node 310 in the connection layer 210 in each tomographic image 101, etc. are changed by backpropagation, stochastic gradient descent, etc.

[0069] In the learning phase, a series of processes, including inputting multiple tomographic images 101 of learning data 64 to an image generation model 66, outputting a pseudo two-dimensional image 103 from the image generation model 66, calculating a loss function, detecting a region of interest, and updating weights by applying the above equation (2), are repeated to reduce the loss function.

[0070] 7 shows a functional block diagram of an example of a configuration related to a function for generating an image generation model 66 in the image processing device 16 of this embodiment. As shown in Fig. 7, the image processing device 16 includes a learning data acquisition unit 80, a region of interest detection unit 82, and an image generation model generation unit 84. As an example, in the image processing device 16 of this embodiment, the CPU 60A of the control unit 60 executes a learning program 63A stored in the storage unit 62, so that the CPU 60A functions as the learning data acquisition unit 80, the region of interest detection unit 82, and the image generation model generation unit 84.

[0071] The training data acquisition unit 80 has a function of acquiring training data 64 from the storage unit 62. Although one piece of training data 64 is shown in FIG. 6, the storage unit 62 actually stores a sufficient amount of training data 64 for training the image generation model 66. The training data acquisition unit 80 outputs the acquired training data 64 to the region of interest detection unit 82 and the image generation model generation unit 84.

[0072] As described above, region of interest detection unit 82 uses region of interest detector 68 to detect first region of interest 1111, second region of interest 1112, and other region 1113. Note that if calcification P1 is not included in normal two-dimensional image 111, first region of interest 1111 will not be detected. Also, if other lesion P2 is not included in normal two-dimensional image 111, second region of interest 1112 will not be detected. Region of interest detection unit 82 outputs the detected first region of interest 1111, second region of interest 1112, and other region 1113 to image generation model generation unit 84.

[0073] The image generation model generation unit 84 includes a loss function calculation unit 85 and a weight update unit 86. The image generation model generation unit 84 has a function of generating an image generation model 66 that receives a plurality of tomographic images as input and outputs a pseudo two-dimensional image by performing machine learning on a machine learning model using the learning data 64 as described above.

[0074] As described above, the loss function calculation unit 85 calculates a loss function that represents the degree of difference between the pseudo 2D image 103 output from the image generation model 66 and the normal 2D image 111 of the training data 64 using the above equation (1).

[0075] As described above, the weight update unit 86 updates the weights of the network in the image generation model 66 in accordance with the loss function calculated by the loss function calculation unit 85.

[0076] The image generation model generation unit 84 stores the generated image generation model 66 in the storage unit 62.

[0077] Next, the operation of the image processing device 16 of this embodiment in the learning phase will be described with reference to Fig. 8. The CPU 60A executes the learning program 63A stored in the storage unit 62, thereby executing the learning process shown in Fig. 8.

[0078] In step S100 of FIG. 8, the learning data acquisition unit 80 acquires the learning data 64 from the storage unit 62 as described above.

[0079] In the next step S102, the region of interest detection unit 82 detects a first region of interest 1111, a second region of interest 1112, and other regions 1113 from the normal two-dimensional image 111 contained in the learning data 64 acquired in step S100, as described above.

[0080] In the next step S104, the image generation model generating unit 84 inputs the plurality of tomographic images 101 included in the learning data 64 acquired in step S100 to the image generation model 66, as described above.

[0081] In the next step S106, the loss function calculation unit 85 acquires the pseudo 2D image 103 output from the image generation model 66, as described above, and calculates a loss function that represents the degree of difference between the pseudo 2D image 103 output from the image generation model 66 and the normal 2D image 111 of the training data 64.

[0082] In the next step S108, the region of interest detection unit 82 determines whether or not to end learning. In this embodiment, the image generation model 66 that has undergone learning a predetermined number of times and has the smallest output of the loss function is ultimately adopted. Therefore, the region of interest detection unit 82 determines whether or not learning has been performed a predetermined number of times. Specifically, the region of interest detection unit 82 determines whether or not the processes of steps S104 and S106 have been performed a predetermined number of times. If learning has not been performed a predetermined number of times, in other words, if the number of times learning has been performed is less than the predetermined number of times, learning has not yet ended, so the determination in step S108 is negative, and the process proceeds to step S110.

[0083] In step S110, the weight update unit 86 updates the network weights in the image generation model 66 in accordance with the loss function calculated in step S106, as described above.

[0084] After the network weights of the image generation model 66 are updated in step S110, the process returns to step S104, and learning is performed again by repeating the processes of steps S104 to S108. This results in multiple image generation models 66 corresponding to the number of times learning has been performed.

[0085] On the other hand, if the number of times learning has been performed is equal to or greater than the predetermined number of times, learning is terminated, so the determination in step S108 becomes positive and the process proceeds to step S112.

[0086] In step S112, the detection unit 84 selects, from among the multiple image generation models 66 corresponding to the number of times of learning, the one with the smallest loss function calculated in step S106 above as the image generation model 66 finally obtained by learning. When the processing of step S112 ends, the learning process shown in FIG. 8 ends.

[0087] It should be noted that this learning process is not limited to this. For example, a threshold may be set depending on whether the pseudo two-dimensional image 103 output from the image generation model 66 is sufficiently close to the normal two-dimensional image 111, or whether the accuracy of reproducing the shape of the calcification P1 in the pseudo two-dimensional image 103 can be said to be high. When the output of the loss function falls below the threshold, the learning may be terminated, and the model at that time may be used as the image generation model 66.

[0088] (Operational phase) Next, the operation phase in which a pseudo two-dimensional image is generated using the image generation model 66 trained as described above will be described.

[0089] 9 is a schematic diagram for explaining an outline of the flow of generating a pseudo two-dimensional image 102 using the image generation model 66 in the image processing device 16 of this embodiment. As shown in FIG. 9, the image processing device 16 generates the pseudo two-dimensional image 102 by inputting a plurality of tomographic images 100 into the image generation model 66 and outputting the pseudo two-dimensional image 102. Note that when operating the image generation model 66, the pseudo two-dimensional image 102 may be generated by inputting a plurality of tomographic images 100 in voxel units, creating patches of the pseudo two-dimensional image 102, and combining the patches.

[0090] 10 shows a functional block diagram of an example of a configuration related to a function of generating a pseudo two-dimensional image 102 in the image processing device 16. As shown in FIG. 10, the image processing device 16 includes a tomographic image generating unit 90, a pseudo two-dimensional image generating unit 92, and a display control unit 94. As an example, in the image processing device 16 of this embodiment, the CPU 60A of the control unit 60 executes an image generating program 63B stored in the storage unit 62, so that the CPU 60A functions as the tomographic image generating unit 90, the pseudo two-dimensional image generating unit 92, and the display control unit 94.

[0091] The tomographic image generating unit 90 has the function of generating multiple tomographic images from a series of multiple projection images. Based on an instruction to generate a pseudo 2D image, the tomographic image generating unit 90 acquires a desired series of multiple projection images from the console 12 of the mammography apparatus 10 or the PACS 14. The tomographic image generating unit 90 then generates multiple tomographic images 100, each at a different height from the imaging plane 24A, from the acquired series of multiple projection images. The method by which the tomographic image generating unit 90 generates the multiple tomographic images 100 is not particularly limited. For example, the tomographic image generating unit 90 can generate the multiple tomographic images 100 by reconstructing the series of multiple projection images using a back projection method such as the FBP (Filter Back Projection) method or an iterative reconstruction method. The tomographic image generating unit 90 outputs the generated multiple tomographic images 100 to the pseudo 2D image generating unit 92.

[0092] 9, the pseudo 2D image generation unit 92 has a function of generating a pseudo 2D image 102 using the image generation model 66. The pseudo 2D image generation unit 92 inputs a plurality of tomographic images 100 to the image generation model 66. As described above, the pseudo 2D image 102 is output from the image generation model 66. The pseudo 2D image generation unit 92 acquires the pseudo 2D image 102 output from the image generation model 66 and outputs it to the display control unit 94.

[0093] The display control unit 94 has a function of controlling the display of the pseudo two-dimensional image 102 generated by the pseudo two-dimensional image generating unit 92 on the display unit 70 .

[0094] Next, the operation of generating the pseudo two-dimensional image 102 in the image processing device 16 of this embodiment will be described with reference to Fig. 11. The image generation process shown in Fig. 11 is performed by the CPU 60A executing the image generation program 63B stored in the storage unit 62.

[0095] In step S200 of FIG. 11, the tomographic image generating unit 90 acquires a series of multiple projection images from the console 12 of the mammography apparatus 10 or the PACS 14, as described above.

[0096] In the next step S202, the tomographic image generating unit 90 generates a plurality of tomographic images 100 from the series of a plurality of projection images acquired in the above step S200, as described above.

[0097] In the next step S204, the pseudo two-dimensional image generation unit 92 generates the pseudo two-dimensional image 102 using the image generation model 66, as described above. Specifically, the pseudo two-dimensional image 102 output from the image generation model 66 is acquired by inputting the multiple tomographic images 100 generated in the above step S202 to the image generation model 66.

[0098] In the next step S206, the display control unit 94 controls the display unit 70 to display the pseudo two-dimensional image 102 obtained in step S204. The display format for displaying the pseudo two-dimensional image 102 on the display unit 70 is not particularly limited. For example, only the pseudo two-dimensional image 102 may be displayed on the display unit 70, or a plurality of tomographic images 100 and the pseudo two-dimensional image 102 may be displayed on the display unit 70. When the processing of step S206 is completed, the image generation processing shown in FIG. 11 is completed.

[0099] The learning phase and the operation phase described above are merely examples, and various modifications are possible. For example, the learning phase and the operation phase may be modified as follows.

[0100] (Variation 1: Variation of the learning phase) FIG. 12 is a schematic diagram for explaining the outline of the learning flow of the image generation model 66 in the image processing device 16 of this modified example.

[0101] In the above embodiments, the image to be detected for the first region of interest and the second region of interest is the normal two-dimensional image 111. However, the image to be detected for the first region of interest and the second region of interest is not limited to the normal two-dimensional image 111. The example shown in Fig. 12 shows an embodiment in which the image to be detected for the first region of interest and the second region of interest is the tomographic image 101.

[0102] As shown in FIG. 12, in this modification, a first region of interest 1011 including a calcification P1, a second region of interest 1012 including another lesion P2, and another region 1013 are detected from a tomographic image 101.

[0103] 8 , region of interest detection unit 82 detects a first region of interest 1011, a second region of interest 1012, and other regions 1013 from multiple tomographic images 101 included in learning data 64 acquired in step S100. Region of interest detection unit 82 of this modification detects calcification P1 and other lesions P2 from each of the multiple tomographic images 101, and when the detected calcifications P1 and other lesions P2 are superimposed in the height direction of the tomographic images 101, detects the outermost contours as the regions of calcification P1 and other lesions P2. Then, based on the detected regions of calcification P1 and other lesions P2, region of interest detection unit 82 detects first region of interest 1011, second region of interest 1012, and other regions 1013.

[0104] In this way, according to this modification, it is possible to make it easier to detect calcifications P1 and other lesions P2 that are difficult to see due to overlapping mammary glands and the like.

[0105] Note that the images to be used to detect the first region of interest and the second region of interest are not limited to the multiple tomographic images 101, but may also be a series of multiple projection images used to obtain the multiple tomographic images 101.

[0106] (Variation 2: Variation of the learning phase) FIG. 13 is a schematic diagram for explaining the outline of the learning flow of the image generation model 66 in the image processing device 16 of this modified example.

[0107] Since calcification P1 has a weak signal, it is buried in noise and difficult to see on a tomographic image with a low radiation dose per image. Therefore, it is preferable to detect calcification P1 from a two-dimensional image such as a normal two-dimensional image with a higher radiation dose than a tomographic image per image. On the other hand, masses are preferably detected from the tomographic image 101 because they may be hidden by overlapping mammary glands and become difficult to see. Furthermore, when mammary glands overlap, they may appear as a mass or spicules, so it is preferable to detect them from the tomographic image 101. Therefore, in this modified example, as shown in FIG. 13 , for calcification P1, calcification P1 (first region of interest 1111) is detected from the normal two-dimensional image 111 as described in the above embodiment, and for other lesions P2, other lesions P2 (second region of interest 111) are detected from each of the multiple tomographic images 101. 2) 13, the region of interest detector 681 for detecting calcification P1 and the region of interest detector 682 for detecting other lesions P2 are shown separately, but as in the above embodiment (see FIG. 6), the detection of calcification P1 and other lesions P2 may be performed by one region of interest detector 68.

[0108] 8 , region of interest detection unit 82 detects a first region of interest 1111 from normal two-dimensional image 111 included in learning data 64 acquired in step S100 above, and detects a second region of interest 1112 from each of a plurality of tomographic images 101 included in learning data 64. Furthermore, a region excluding first region of interest 1111 and second region of interest 1112 from normal two-dimensional image 111 is detected as other region 1113.

[0109] In this way, in this modification, calcification P1 (first region of interest 1111) is detected from the normal two-dimensional image 111, and other lesions P2 (second region of interest 1111) are detected from the normal two-dimensional image 111. 2) By detecting the calcification P1 and other lesions P2 from the tomographic image 101, the accuracy of detecting the calcification P1 and other lesions P2 can be improved.

[0110] As described above, it is preferable to detect the calcification P1 (first region of interest 1111) from a two-dimensional image. Therefore, instead of the normal two-dimensional image 111, the calcification P1 (first region of interest 1111) may be detected from a composite two-dimensional image obtained by combining at least a portion of a plurality of tomographic images 101. The method for generating the composite two-dimensional image is not particularly limited. For example, the composite two-dimensional image may be generated by combining a plurality of tomographic images 101 using an addition method, an averaging method, a maximum intensity projection method, a minimum intensity projection method, or the like.

[0111] (Variation 3: Variation of the learning phase) FIG. 14 is a schematic diagram for explaining the outline of the learning flow of the image generation model 66 in the image processing device 16 of this modified example.

[0112] In the above embodiments, the weighting relationship in the image generation model 66 is not defined for other lesions P2, and for example, the same weighting is used. However, depending on the type of other lesion P2, there may be cases where it is desired that the reproduction accuracy be higher than that of other types. Therefore, in this modification, the weighting to be updated for other lesions P2 is also different depending on the type of lesion.

[0113] In the example shown in FIG. 14, the other lesion P2 is a mass, and the region of interest detector 68 detects the calcification P1 (first region of interest 1111) and the other lesion P2, which is a mass (second region of interest 1111). 21 ) plus spicules P3 (second region of interest 111 23) In addition, the other region 1113 is detected by extracting the first region of interest 1111 and the second region of interest 1111 from the normal two-dimensional image 111. 21 , and a second region of interest 111 22 The area is the area excluding

[0114] The loss function calculation unit 85 calculates the loss function based on the following equation (3) instead of the above equation (1). Difference between pseudo two-dimensional image 103 and first region of interest 1111 × weight W1111 + difference between pseudo two-dimensional image 103 and second region of interest 11121 Difference from × weight W111 21 + Pseudo 2D image 103 and second region of interest 111 22 Difference from × weight W111 22 + difference between pseudo 2D image 103 and other region 1113 × weight W1112 (3)

[0115] For example, when the reproducibility of tumors is considered more important than the reproducibility of spicules, a weight satisfying the relationship of the following equation (4) is preset as a weight for each region of interest instead of the above equation (2).

[0116] Weight W1111>Weight W111 21 >Weight W111 22 ≧Weight W1113 (4)

[0117] Furthermore, the weight update unit 86 updates the weights of the network in the image generation model 66 in accordance with the loss function calculated by the loss function calculation unit 85 based on the above equation (3).

[0118] In this modified example, in step S102 of the learning process shown in FIG. 8, the region of interest detection unit 82 extracts a first region of interest 1111 and a second region of interest 1112 from the normal two-dimensional image 111 included in the learning data 64 acquired in step S100. 21 , and a second region of interest 111 22 Detect.

[0119] In this way, according to this modified example, the image generation model 66 can be trained to generate a pseudo two-dimensional image 102 with improved reproduction accuracy of the desired lesion, even among other lesions P2.

[0120] (Variation 4: Variation of the learning phase) The amount of glandular tissue in the breast can affect the appearance of calcifications P1 and masses in radiographic images. Masses are particularly susceptible to the influence of glandular tissue.

[0121] Therefore, in this modification, the other lesion P2 is treated as a mass, and the weight W1112 set for the second region of interest 1112 is adjusted according to the amount of mammary gland in the breast. For ease of explanation, in this modification, the other lesion P2 is referred to as a "mass P2."

[0122] For example, in the case of a so-called dense breast (high-density breast) with a large amount of mammary glands, the mass P2 may be hidden by the mammary glands and may be difficult to see. Therefore, in this modification, the weight W1112 set for the second region of interest 1112 for the mass P2 is adjusted according to the amount of mammary glands, thereby preventing a decrease in the reproduction accuracy of the mass P2.

[0123] The loss function calculation unit 85 of this embodiment derives the mammary gland amount of the breast from the normal two-dimensional image 111. Specifically, the loss function calculation unit 85 derives the mammary gland amount, which represents the mammary gland content rate in the thickness direction of the breast, which is the irradiation direction of the radiation R, as the mammary gland amount for each pixel of the normal two-dimensional image 111. When there are no mammary glands and the breast consists only of fat, the mammary gland content rate is "0," and the higher the mammary gland density value, the higher the mammary gland content rate. Note that the method by which the loss function calculation unit 85 derives the mammary gland content rate is not particularly limited, and known methods can be applied. For example, the loss function calculation unit 85 can derive the mammary gland content rate based on the pixel values ​​of regions in each tomographic image where the breast M is not captured (so-called blank regions), the pixel values ​​of pixels corresponding to fat, the pixel values ​​of pixels from which the mammary gland content rate is to be derived, and the ratio of the average attenuation coefficients of the mammary glands to the fat (average attenuation coefficient of the mammary glands / average attenuation coefficient of the fat).

[0124] Then, the loss function calculation unit 85 adjusts the weight W1112 of the second region of interest 1112 based on the glandular volume while maintaining the relationship of the above equation (2), and calculates the loss function using the adjusted weight W1112. As an example, when the glandular volume is small, the loss function calculation unit 85 significantly reduces the weight W1112 of the second region of interest 1112 corresponding to the mass P2 compared to the weight W1111 of the first region of interest 1111 corresponding to the calcification P1 (weight W1111 >> weight W1112). When the glandular volume is large, the loss function calculation unit 85 reduces the weight W1112 of the second region of interest 1112 corresponding to the mass P2 compared to the weight W1111 of the first region of interest 1111 corresponding to the calcification P1, to a degree less than when the glandular volume is small (weight W1111 > weight W1112). That is, the weight of the second region of interest 1112 corresponding to the mass P2 becomes smaller than the weight of the first region of interest 1111 corresponding to the calcification P1 as the amount of mammary gland decreases.

[0125] Fig. 15 shows a flowchart illustrating an example of the flow of the learning process by the image processing device 16 of this modified example. The learning process shown in Fig. 15 differs from the learning process of each of the above-described forms (see Fig. 8) in that step S103 is provided between step S102 and step S104, and step S105 is provided between step S104 and S106.

[0126] After step S102, in step S103, the weight update unit 86 derives the mammary gland volume as described above, and then proceeds to step S104. In this modification, as described above, the mammary gland content rate is derived for each pixel in the first region of interest 1111 and the second region of interest 1112.

[0127] Also, in step S105, the loss function calculation unit 85 adjusts the weight W1112 set for the second region of interest 1112 so that it is smaller than the weight W1111 of the first region of interest 1111 as the derived mammary gland content rate decreases, while maintaining the relationship of equation (2) above, as described above.

[0128] As a result, in step S106, the loss function calculation unit 85 calculates the loss function using the adjusted weight W1112.

[0129] In this way, according to this modified example, the weight W1112 set for the second region of interest 1112 is adjusted according to the amount of mammary gland in the breast, thereby suppressing the influence of the amount of mammary gland on the reproduction accuracy of the mass P2.

[0130] Although this modification has been described as adjusting the value of the weight W1112 set for the second region of interest 1112 according to the mammary gland amount (mammary gland content), the modification of the weight according to the mammary gland amount is not limited to this modification. For example, at least one of the weight W1111 set for the first region of interest 1111 and the weight W1112 set for the second region of interest 1112 may be adjusted according to the mammary gland amount. For example, the weight W1111 of the first region of interest 1111 corresponding to calcification P1 may be adjusted so that the greater the mammary gland amount, the greater the weight W1111, and the smaller the mammary gland amount, the smaller the weight W1111. Furthermore, for example, the value of the weight according to the mammary gland amount for the other region 1113 may also be adjusted. In this case, the other region 1113 is an example of a third region of interest in the present disclosure.

[0131] (Variation 5: Variation of the learning phase) Positional deviations of the breast M, calcification P1, other lesions P2, etc. may occur between the multiple tomographic images 101 included in the image generation model 66 and the normal two-dimensional image 111. If the timing at which the series of projection images for obtaining the tomographic image 101 is captured differs from the timing at which the normal two-dimensional image 111 is captured, positional deviations of the breast M, calcification P1, other lesions P2, etc. may occur between the multiple tomographic images 101 and the normal two-dimensional image 111. Even when tomosynthesis imaging and normal imaging are performed while the breast remains compressed by the compression paddle 30, in other words, without releasing the compression of the breast by the compression paddle 30 between tomosynthesis imaging and normal imaging, this positional deviation may occur due to body movement of the subject or deformation of the breast.

[0132] Therefore, the image generation model generating unit 84 of this modified example corrects the positional deviation of each of the multiple tomographic images 101 relative to the normal two-dimensional image 111, and then inputs the corrected positional deviation to the learning data 64. Note that the method by which the image generation model generating unit 84 corrects the positional deviation of each of the multiple tomographic images 100 relative to the normal two-dimensional image 111 is not particularly limited. For example, the image generation model generating unit 84 may derive the amount and direction of positional deviation of the skin line of the breast M, and correct the positional deviation so that each skin line of the multiple tomographic images 101 coincides with the skin line of the normal two-dimensional image 111. In addition, examples of the method for detecting and specifying the skin line include a method of sequentially searching for boundary points between the area of ​​the breast M and the blank area in each of the multiple tomographic images 101 and the normal two-dimensional image 111, and detecting the skin line by connecting the searched pixels, etc.

[0133] Fig. 16 shows a flowchart illustrating an example of the flow of the learning process by the image processing device 16 of this modified example. The learning process shown in Fig. 16 differs from the learning process of each of the above-described forms (see Fig. 8) in that step S101 is provided between step S100 and step S102.

[0134] After step S100, in step S101, the image generation model generating unit 84 corrects the positional deviation of each of the multiple tomographic images 100 relative to the normal two-dimensional image 111 so as to align the skin lines of the breasts M, as described above.

[0135] As a result, in step S104, the plurality of tomographic images 101 whose positional deviations with respect to the normal two-dimensional image 111 have been corrected are input to the image generation model 66.

[0136] In this way, according to this modified example, it is possible to prevent any positional deviation between the normal two-dimensional image 111, which is the learning data 64, and each of the multiple tomographic images 101, thereby further improving the reproduction accuracy of the pseudo two-dimensional image 102 by the image generation model 66.

[0137] (Variation 6: Variation of the learning phase) FIG. 17 is a schematic diagram for explaining the outline of the learning flow of the image generation model 66 in the image processing device 16 of this modified example.

[0138] As shown in FIG. 17, in this modification, the learning data 64 includes, instead of the normal two-dimensional image 111, a composite two-dimensional image 121 obtained by combining at least a portion of a plurality of tomographic images 101.

[0139] The image processing device 16 of this modification includes a composite 2D image generation unit 88. As an example, in this modification, the composite 2D image generation unit 88 has a function of generating a composite 2D image 121 that serves as learning data 64 by synthesizing at least a portion of a plurality of tomographic images 101. For example, the composite 2D image generation unit 88 can be the same as the composite 2D image generation unit 96 (see FIG. 19 ) in the above-described modification 6.

[0140] The loss function calculation unit 85 of this modification calculates a loss function between the composite 2D image 121 and the pseudo 2D image 103. A specific calculation method is to replace the normal image 111 in the loss calculation of the loss function calculation unit 85 in each of the above embodiments with the composite 2D image 121.

[0141] Furthermore, region of interest detection unit 82 of this modification detects a first region of interest 1211, a second region of interest 1212, and other regions 1213 from composite two-dimensional image 121 using region of interest detector 68. Region of interest detection unit 82 outputs the detected first region of interest 1211, second region of interest 1212, and other regions 1213 to image generation model generation unit 84.

[0142] Thus, according to this modification, the network weights of image generation model 66 are updated according to a loss function based on the difference between synthetic 2D image 121 and pseudo 2D image 103. In some cases, a tumor is difficult to see in normal 2D image 111. In such cases, by using synthetic 2D image 121 instead of normal 2D image 111, the accuracy of tumor detection can be improved.

[0143] (Variation 7: Variation of the learning phase) FIG. 18 is a schematic diagram for explaining the outline of the learning flow of the image generation model 66 in the image processing device 16 of this modified example.

[0144] 18, in this modification, the learning data 64 includes a normal two-dimensional image 111 and a composite two-dimensional image 121 obtained by combining at least a portion of a plurality of tomographic images 101. The composite two-dimensional image 121 included in the learning data is the same as the composite two-dimensional image 121 included in the learning data 64 in the above-described modification 6 (see FIG. 17).

[0145] The normal two-dimensional image 111 is a radiographic image in which calcification P1 is relatively easy to see. On the other hand, other lesions P2 such as tumors are difficult to see in the normal two-dimensional image 111, and may be easier to see in the composite two-dimensional image 121. Therefore, the training data 64 of this modified example includes both the normal two-dimensional image 111 and the composite two-dimensional image 121.

[0146] In this modification, loss function calculation unit 85 calculates a loss function using the difference between normal 2D image 111 and pseudo 2D image 103 for first region of interest 1111 including calcification P1, and the difference between synthetic 2D image 121 and pseudo 2D image 103 for second region of interest 1112 including other region of interest P2. For example, loss function calculation unit 85 calculates the loss function based on the following equation (5) instead of the above equation (1): Difference between pseudo 2D image 103 and first region of interest 1111 in normal 2D image 111 × weight W1111 + Difference between pseudo 2D image 103 and second region of interest 1112 in synthetic 2D image 121 × weight W1112 (5)

[0147] In the above equation (5), for the first region of interest 1111, the loss function is calculated using the difference between the pseudo 2D image 103 and the normal 2D image 111, and for the second region of interest 1112, the loss function is calculated using the difference between the pseudo 2D image 103 and the synthetic 2D image 121.

[0148] Although the other region 1113 is not used in the calculation of the loss function in the above equation (5), the other region 1113 may also be used in the calculation of the loss function. That is, the loss calculation unit 85 may calculate the loss function for the other region 11113 based on the value obtained by multiplying the difference between the pseudo 2D image 103 and the normal 2D image 111, or the difference between the pseudo 2D image 103 and the synthesized 2D image 121, by a weight W1113 and adding the result to the above equation (5).

[0149] 18 shows a configuration in which region of interest detection unit 82 detects each region of interest from normal two-dimensional image 111 using region of interest detector 68, but the radiation image from which each region of interest is detected is not limited to normal two-dimensional image 111. For example, region of interest detection unit 82 may detect each region of interest from composite two-dimensional image 121 using region of interest detector 68. Also, for example, region of interest detection unit 82 may detect a first region of interest 1111 from normal two-dimensional image 111 using region of interest detector 68, and detect a second region of interest 1112 from composite two-dimensional image 121 using region of interest detector 68.

[0150] As described above, according to this modification, the network weights of the image generation model 66 are updated in accordance with a loss function based on the difference between the normal 2D image 11 and the pseudo 2D image 103 and the difference between the synthetic 2D image 121 and the pseudo 2D image 103. This allows the loss function to be calculated using an easily visible radiographic image for each of the calcifications P1 and other lesions P2 such as tumors, and the network weights of the image generation model 66 to be updated, thereby improving the accuracy of the image generation model 66.

[0151] (Variation 8: Variation of Operation Phase) Image generation in the image generation model 66 requires a large amount of processing, which tends to result in high calculation costs. Therefore, the image processing device 16 may generate only the regions where calcification P1 and other lesions P2 are detected using the image generation model 66. The image processing device 16 of this modified example is an example of an image generation device of the present disclosure.

[0152] FIG. 19 is a schematic diagram illustrating the flow of generating a pseudo two-dimensional image 104 using an image generation model 66 in an image processing device 16 of this modified example. As shown in FIG. 19, the image processing device 16 uses a region of interest detector 68 to detect a first region of interest 1001 and a second region of interest 1002 from multiple tomographic images 100. The first region of interest 1001 is a region in which calcification P1 is detected, extracted as voxels, and the second region of interest 1002 is a region in which other lesion P2 is detected, extracted as voxels. The multiple first regions of interest 1001 and multiple second regions of interest 1002 are input to the image generation model 66. The image generation model 66 outputs a pseudo two-dimensional image 1021 including calcification P1, generated in accordance with the multiple first regions of interest 1001 input. Furthermore, the image generation model 66 outputs a pseudo two-dimensional image 1022 including the other lesion P2 generated in accordance with the input second regions of interest 1002.

[0153] 19, the composite two-dimensional image generating unit 96 generates a composite two-dimensional image 120 by combining a plurality of tomographic images 100. As described above in Modification 2, the method for generating the composite two-dimensional image 120 is not particularly limited, and for example, the composite two-dimensional image may be generated by combining a plurality of tomographic images 100 by an addition method, an averaging method, a maximum intensity projection method, a minimum intensity projection method, or the like.

[0154] 19 , in the image processing device 16, the 2D image combining unit 97 combines the composite 2D image 120 with the pseudo 2D image 1021 and the pseudo 2D image 1022 to generate the pseudo 2D image 104. Note that the method by which the 2D image combining unit 97 combines the pseudo 2D image 1021 and the pseudo 2D image 1022 is not particularly limited. For example, the pseudo 2D image 1021 and the pseudo 2D image 1022 may be combined by superimposing the pseudo 2D image 1021 and the pseudo 2D image 1022 at positions in the composite 2D image 120 corresponding to the pseudo 2D image 1021 and the pseudo 2D image 1022, respectively.

[0155] Fig. 20 shows a functional block diagram of an example of the configuration related to the function of generating a pseudo 2D image 102 in image processing device 16 of this modified example. As shown in Fig. 20, image processing device 16 of this modified example differs from image processing device 16 of each of the above forms (see Fig. 10) in the configuration of pseudo 2D image generation unit 92. Pseudo 2D image generation unit 92 of this modified example includes a region of interest detection unit 95, a composite 2D image generation unit 96, and a 2D image combination unit 97.

[0156] The image generation program 63B of this modified example is an example of an image generation program of the present disclosure. In the image processing device 16, the CPU 60A of the control unit 60 executes the image generation program 63B stored in the storage unit 62, whereby the CPU 60A functions as a tomographic image generation unit 90, a pseudo two-dimensional image generation unit 92, and a display control unit 94.

[0157] The tomographic image generating unit 90 and the display control unit 94 are similar to the tomographic image generating unit 90 and the display control unit 94 (see FIG. 10 ) of the image processing device 16 of each of the above embodiments, and therefore a description thereof will be omitted. On the other hand, the region of interest detecting unit 95 of the pseudo two-dimensional image generating unit 92 has a function of detecting a first region of interest 1001 and a second region of interest 1002 from each of the plurality of tomographic images 100 using the region of interest detector 68, as described above. The pseudo two-dimensional image generating unit 92 inputs the detected first region of interest 1001 and second region of interest 1002 to the image generation model 66.

[0158] As described above, the composite two-dimensional image generating unit 96 has a function of generating a composite two-dimensional image 120 by combining at least a portion of the multiple tomographic images 100.

[0159] As described above, the two-dimensional image combination unit 97 has the function of combining the pseudo two-dimensional image 1021 and the pseudo two-dimensional image 1022 output from the image generation model 66 with the composite two-dimensional image 120 to generate the pseudo two-dimensional image 104. The two-dimensional image combination unit 97 outputs the generated pseudo two-dimensional image 104 to the display control unit 94.

[0160] Fig. 21 shows a flowchart illustrating an example of the flow of image generation processing by the image processing device 16 of this modified example. The image generation processing shown in Fig. 21 differs from the image generation processing of each of the above-described embodiments (see Fig. 11) in that it includes steps S205A to S205D instead of step S204.

[0161] As shown in FIG. 21, in step S205A, pseudo two-dimensional image generating unit 92 detects first and second regions of interest 1001 and 1002 from each of the plurality of tomographic images 100, as described above.

[0162] In the next step S205B, the pseudo 2D image generation unit 92 uses the image generation model 66, as described above, to generate pseudo 2D images 1021 and 1022 of the first region of interest 1001 and the second region of interest 1002 detected in step S205A above.

[0163] In the next step S205C, the composite two-dimensional image generating unit 96 generates a composite two-dimensional image 120 by combining at least a portion of the multiple tomographic images 100 as described above.

[0164] In the next step S205D, the 2D image combination unit 97 generates the pseudo 2D image 104 by combining the pseudo 2D image 1021 and the pseudo 2D image 1022 generated in step S205B with the composite 2D image 120 generated in step S205C, as described above.

[0165] In this way, in this modification, a pseudo two-dimensional image is generated only for the region of interest using the image generation model 66. Therefore, compared to when pseudo two-dimensional images are generated for other regions using the image generation model 66, the amount of processing in the image generation model 66 can be reduced, and the so-called calculation cost can be prevented from increasing.

[0166] In the image processing device 16, whether to generate a pseudo two-dimensional image of only the region of interest using the image generation model 66, as in this modification, or to generate a pseudo two-dimensional image of the entire image including other regions may be selected automatically or in accordance with instructions from a doctor, etc. For example, when the number of regions of interest is relatively small or when the total area of ​​all regions of interest is relatively small compared to the entire image, the image processing device 16 may be configured to select to generate a pseudo two-dimensional image of only the region of interest.

[0167] As described above, the image processing device 16 of each of the above embodiments is a learning device for an image generation model 66 that generates a pseudo two-dimensional image 102 from a series of multiple projection images obtained by tomosynthesis imaging of the breast or multiple tomographic images 100 obtained from the series of multiple projection images. The image processing device 16 includes a CPU 60A, which acquires a normal two-dimensional image 111 captured by irradiating the breast with radiation, and detects a first region of interest 1111 including breast calcification P1 and a second region of interest 1112 including other lesions P2 that are lesions other than calcification P1, based on the normal two-dimensional image 111, the tomographic image 100, or the normal two-dimensional image 111 obtained by combining at least a portion of the series of multiple projection images or multiple tomographic images 101. The image generation model 66 is trained by updating the weights of the network of the image generation model 66 to reduce the loss function based on the loss function between the pseudo 2D image 103 and the normal 2D image 111 generated by the image generation model 66 in which the weight W1111 of the first region of interest 1111 is set to be the heaviest and the weight W1112 of the second region of interest 1112 is set to be equal to or greater than the weight W1113 of the other region 1113 other than the first region of interest W1111 and the second region of interest 1112.

[0168] As described above, in each of the above embodiments, the image generation model 66 is trained while updating the network weights by assigning a larger weight to calcification P1 than to other regions, thereby enabling training that focuses more on calcification P1 than other regions. This allows for higher accuracy in reproducing calcification P1. Calcifications tend to be smaller and have finer shapes than other lesions. Therefore, the image processing device 16 of each of the above embodiments can generate pseudo-2D images that accurately reproduce the shapes of breast lesions.

[0169] In the above embodiments, the region of interest detector 68 detects calcification P1 and other lesions P2 by applying a CAD algorithm based on the probability that the calcifications P1 and other lesions P2 are present, but the manner in which the region of interest detector 68 detects calcifications P1 and other lesions P2 is not limited to these embodiments. For example, the region of interest detector 68 may detect calcifications P1 and other lesions P2 by using a filtering process using a filter to detect each of the calcifications P1 and other lesions P2, or a detection model that has been machine-learned by deep learning or the like to detect each of the calcifications P1 and other lesions P2.

[0170] As described above, the image generation model 66 may be, for example, a U-Net, which is an encoder-decoder model using a convolutional neural network (CNN). In this case, it is assumed that only the encoder performs three-dimensional convolutional estimation. Alternatively, a multilayer perceptron (MLP) or the like may be applied.

[0171] Furthermore, in the above embodiment, the image processing device 16 learns the image generation model 66 and generates the pseudo two-dimensional image 102 using the image generation model 66. However, the image generation model 66 may be learned by a learning device other than the image processing device 16. In other words, the device that learns the image generation model 66 and the device that generates the pseudo two-dimensional image 102 using the image generation model 66 may be different devices.

[0172] In the above embodiment, the following various processors may be used as the hardware structure of processing units that perform various processes, such as the learning data acquisition unit 80, the region of interest detection unit 82, the image generation model generation unit 84, the tomographic image generation unit 90, the pseudo two-dimensional image generation unit 92, and the display control unit 94. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD), a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0173] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0174] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0175] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0176] In addition, in the above embodiments, the learning program 63A and the image generation program 63B are pre-stored (installed) in the storage unit 62, but this is not limiting. Each of the learning program 63A and the image generation program 63B may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Also, each of the learning program 63A and the image generation program 63B may be downloaded from an external device via a network. [Explanation of symbols]

[0177] 1. Radiography system 10 Mammography equipment 12 Console 14 PACS 16 Image processing device 17 Network 191-197, 19 t Irradiation position 20 Radiation detector, 20A detection surface 24 imaging table, 24A imaging surface 26 Foundation 27 Shaft 28 Arm section 29 Radiation source 30 Compression Plate 32 Compression Unit 40, 60 Control unit 42, 50, 62 storage section 44 User I / F section 46, 74 Communication I / F section 52 Radiography images 60A CPU, 60B ROM, 60C RAM 63A Learning Program, 63B Image Generation Program 64 training data 66 Image Generation Model 68, 681, 682 Region of Interest Detector 70 Display section 72 Control section 79 Bus 80 Learning data acquisition unit 82 Region of interest detection unit 84 Image generation model generation unit 85 Loss function calculation unit 86 Weight update unit 88, 96 Composite 2D image generation unit 90 Tomographic image generation unit 92 Pseudo 2D image generation unit 94 Display control unit 95 Region of interest detection unit 97 2D Image Combination Department 100, 1001, 100 k , 101 Tomographic images 102, 1021, 1022, 103, 104 Pseudo 2D images 110, 111 Normal two-dimensional image, 1001, 1111 First region of interest, 1002, 1112, 111 21 , 111 22 Second Area of ​​Interest, 1113 Other Areas 121 composite two-dimensional image, 1211 first region of interest, 1212 second region of interest, 1213 other region 200 input layers 202 Middle Class 203d decoder 203e Encoder 204 Output Layer 210 Connecting layer 300, 3021, 302 u , 304, 304 11 , 304 12 , 304 k1 , 304 k2 , 310, 3101, 3102 nodes F Filter Ip1~Ip4 target pixels D, DI1 to DI4 input data, Dc, DIc1, DIc2 output data M, U Breast P1 Calcification, P2 Other lesions (tumors), P3 Other lesions (spicules) R radiation, RC radiation axis conv convolutional layer pool pooling layer upsmp upsampling layer α, β angles

Claims

1. 1. A learning device for an image generation model that generates a pseudo two-dimensional image from a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images, comprising: at least one processor; The processor: irradiating the breast with radiation and acquiring a normal two-dimensional image; detecting a first region of interest including a calcification of the breast from the normal two-dimensional image or a composite two-dimensional image obtained by combining at least a portion of the series of projection images or the series of tomographic images, and detecting a second region of interest including a lesion other than the calcification from the tomographic image; and training the image generation model by updating weights of a network of the image generation model to reduce the loss based on the loss between the pseudo two-dimensional image output by the image generation model, in which the weight of the first region of interest is set to be the heaviest and the weight of the second region of interest is set to be equal to or greater than the weights of other regions other than the first region of interest and the second region of interest. Learning device.

2. The processor: The weights of the network to be updated are varied depending on the type of the other lesions. The learning device according to claim 1 .

3. The processor: Deriving the mammary gland volume of the breast from the normal two-dimensional image; adjusting at least one value of the weight of the first region of interest, the weight of the second region of interest, and the weight of the third region of interest according to the derived mammary gland volume; The learning device according to claim 1 or 2.

4. The processor: correcting a positional deviation between the series of multiple projection images or the multiple tomographic images and the normal two-dimensional image; The series of the plurality of projection images or the plurality of tomographic images after correction are input to the image generation model to perform the learning. The learning device according to any one of claims 1 to 3.

5. An image generation device for a pseudo two-dimensional image using an image generation model trained by the learning device according to any one of claims 1 to 4, at least one processor; The processor: a series of projection images obtained by tomosynthesis imaging of the breast or a series of tomographic images obtained from the series of projection images are acquired as images for generating the pseudo two-dimensional image; generating a composite two-dimensional image by synthesizing at least a portion of the images for generation; detecting a region of interest from the generation image that includes a lesion, the lesion being at least one of a calcification and another lesion; An image of the region of interest, which is a partial region of the image for generation, is input to the image generation model, and a pseudo two-dimensional image of the region of interest is output from the image generation model; generating a pseudo two-dimensional image by combining the synthetic two-dimensional image with a pseudo two-dimensional image of the region of interest; Image generating device.

6. 1. A method for learning an image generation model that generates a pseudo two-dimensional image from a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images, comprising: irradiating the breast with radiation and acquiring a normal two-dimensional image; detecting a first region of interest including a calcification of the breast from the normal two-dimensional image or a composite two-dimensional image obtained by combining at least a portion of the series of projection images or the series of tomographic images, and detecting a second region of interest including a lesion other than the calcification from the tomographic image; and training the image generation model by updating weights of a network of the image generation model to reduce the loss based on the loss between the pseudo two-dimensional image output by the image generation model, in which the weight of the first region of interest is set to be the heaviest and the weight of the second region of interest is set to be equal to or greater than the weights of other regions other than the first region of interest and the second region of interest. A learning method in which a computer performs processing.

7. 5. An image generation method for generating a pseudo two-dimensional image using an image generation model trained by the learning device according to claim 1, comprising: a series of projection images obtained by tomosynthesis imaging of the breast or a series of tomographic images obtained from the series of projection images are acquired as images for generating the pseudo two-dimensional image; generating a composite two-dimensional image by synthesizing at least a portion of the images for generation; detecting a region of interest from the generation image that includes a lesion, the lesion being at least one of a calcification and another lesion; An image of the region of interest, which is a partial region of the image for generation, is input to the image generation model, and a pseudo two-dimensional image of the region of interest is output from the image generation model; generating a pseudo two-dimensional image by combining the synthetic two-dimensional image with a pseudo two-dimensional image of the region of interest; An image generation method in which processing is performed by a computer.

8. A learning program for causing a computer to execute a process of learning an image generation model that generates a pseudo two-dimensional image from a series of multiple projection images obtained by tomosynthesis imaging of a breast or multiple tomographic images obtained from the series of multiple projection images, comprising: irradiating the breast with radiation and acquiring a normal two-dimensional image; detecting a first region of interest including a calcification of the breast from the normal two-dimensional image or a composite two-dimensional image obtained by combining at least a portion of the series of projection images or the series of tomographic images, and detecting a second region of interest including a lesion other than the calcification from the tomographic image; and training the image generation model by updating weights of a network of the image generation model to reduce the loss based on the loss between the pseudo two-dimensional image output by the image generation model, in which the weight of the first region of interest is set to be the heaviest and the weight of the second region of interest is set to be equal to or greater than the weights of other regions other than the first region of interest and the second region of interest. Learning program.

9. An image generation model trained by the training device according to any one of claims 1 to 4. An image generation program for causing a computer to execute a process for generating a pseudo two-dimensional image using a model, a series of projection images obtained by tomosynthesis imaging of the breast or a series of tomographic images obtained from the series of projection images are acquired as images for generating the pseudo two-dimensional image; generating a composite two-dimensional image by synthesizing at least a portion of the images for generation; detecting a region of interest from the generation image that includes a lesion, the lesion being at least one of a calcification and another lesion; An image of the region of interest, which is a partial region of the image for generation, is input to the image generation model, and a pseudo two-dimensional image of the region of interest is output from the image generation model; generating a pseudo two-dimensional image by combining the synthetic two-dimensional image with a pseudo two-dimensional image of the region of interest; Image generation program.

Images