Image processing apparatus, radiation image photographing system and program

Abstract

To provide an image processing apparatus capable of determining superiority of a structure of interest of the same kind by considering an interrelationship among structures of interest, a radiation image photographing system and program.SOLUTION: An image processing apparatus 4 includes: an image acquisition part 30 acquiring a projection image obtained by photographing a subject from a plurality of angles with radiation; a tomographic image reconstruction part 31 reconstructing a plurality of tomographic images from the acquired projection image; a structure of interest detection part 32 detecting a plurality of structures of interest from a plurality of reconstructed tomographic images; and a priority determination part 33 for determining priority by using an interrelationship among other structures of interest with respect to the respective structures of interest of the same kind in a plurality of detected structures of interest.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, a radiation image capturing system, and a program. [Background technology]

[0002] In recent years, imaging diagnosis using a radiation imaging device (called mammography) that captures the breast has been attracting attention in order to promote early detection of breast cancer. In addition, tomosynthesis imaging has been proposed in mammography, in which a radiation source is moved to irradiate the breast from multiple radiation source positions to capture images, and multiple projection images thus obtained are reconstructed to generate a tomographic image in which a desired tomographic plane is emphasized. In tomosynthesis imaging, depending on the characteristics of the imaging device and the required tomographic image, the radiation source is moved parallel to the radiation detector or moved in a circular or elliptical arc to capture multiple projection images by capturing images of the breast at multiple radiation source positions, and these projection images are reconstructed using a backprojection method such as a simple backprojection method or a filtered backprojection method, or an iterative reconstruction method, to generate a tomographic image.

[0003] By generating such tomographic images on multiple slices of the breast, it is possible to separate structures that overlap in the depth direction of the slices within the breast. This makes it possible to discover abnormal areas such as lesions that are difficult to detect in 2D images obtained by conventional plain radiography, in which radiation is irradiated onto the subject from a predetermined direction (hereinafter referred to as plain 2D images).

[0004] As a technology utilizing such tomographic images, Patent Document 1 discloses a technology in which a plurality of tomographic images, which are obtained by tomosynthesis imaging and have different distances (height direction positions) from the detection surface of a radiation detector toward the radiation source, are synthesized by an addition method, an averaging method, a maximum value projection method, a minimum value projection method, or the like, to generate a pseudo two-dimensional image (hereinafter referred to as a synthesized two-dimensional image) equivalent to a simple two-dimensional image.

[0005] When creating a composite 2D image, it is desirable to emphasize structures of interest as much as possible, but when structures of interest overlap, it is difficult to determine which structure of interest should be prioritized. Depending on the method of prioritization, the structure you want to refer to may become faint or disappear.

[0006] Therefore, Patent Document 2 discloses a technology for detecting objects using an independent recognition module for each type of object (structure of interest). In this technology, when there is a possibility that a first object and a second object overlap, the weight of the object is calculated from the recognition result of the type of the object, and a composite two-dimensional image is created according to the weight. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2014-128716 A [Patent Document 2] U.S. Pat. No. 1,144,5993 Summary of the Invention [Problem to be solved by the invention]

[0008] However, when multiple structures of interest exist in a composite 2D image, the structure of interest to be emphasized often differs depending on the interrelationships between the multiple structures of interest. For example, when multiple calcifications exist, dense calcifications often deserve more attention than calcifications that exist alone. Also, calcifications that exist along the mammary gland often deserve more attention than other calcifications.

[0009] In contrast, the technology disclosed in Patent Document 2 makes it possible to set a structure of interest to be emphasized depending on the type of structure of interest, but does not take into account the interrelationships between the structures of interest, and therefore has the problem that it is not possible to determine the superiority or inferiority of structures of interest of the same type by taking into account the interrelationships.

[0010] The present disclosure has been made in consideration of the above circumstances, and aims to provide an image processing device, a radiological image capturing system, and a program that can determine the superiority or inferiority of structures of interest of the same type by taking into account the interrelationships between the structures of interest. [Means for solving the problem]

[0011] In order to achieve the above object, an image processing device of a first aspect of the present disclosure includes at least one processor, which acquires projection images obtained by photographing a subject from multiple angles using radiation, reconstructs multiple tomographic images from the acquired projection images, detects multiple structures of interest from the reconstructed multiple tomographic images, and determines a priority for each of the detected structures of interest of the same type using its interrelationship with other structures of interest.

[0012] An image processing device according to a second aspect of the present disclosure is the image processing device according to the first aspect, in which the processor creates a composite two-dimensional image of the subject using the determined priority.

[0013] An image processing device of a third aspect of the present disclosure is an image processing device of the first aspect, in which the interrelationship is at least one of a three-dimensional perspective relationship with other structures of interest and a size relationship with other structures of interest.

[0014] An image processing device according to a fourth aspect of the present disclosure is the image processing device according to the third aspect, wherein the mutual relationship is both a perspective relationship and a size relationship.

[0015] An image processing device according to a fifth aspect of the present disclosure is the image processing device according to the first aspect, in which the structure of interest is at least one of calcification, a mass, a spicule, and a mammary gland.

[0016] An image processing device of a sixth aspect of the present disclosure is an image processing device of the first aspect, in which a processor detects a structure of interest using a machine learning model that has been pre-trained by machine learning, with information indicating a tomographic image as input information and information indicating a plurality of structures of interest as output information.

[0017] An image processing device of a seventh aspect of the present disclosure is the image processing device of the first aspect, in which the processor detects the structure of interest by pattern matching using a template image of the target structure of interest.

[0018] In order to achieve the above object, a radiation image capturing system according to an eighth aspect of the present disclosure includes the image processing device according to the first aspect, and a mammography device that acquires a projection image used by the image processing device.

[0019] Furthermore, in order to achieve the above-mentioned object, the program of a ninth aspect of the present disclosure causes a computer to execute processes of acquiring projection images obtained by photographing a subject from multiple angles using radiation, reconstructing multiple tomographic images from the acquired projection images, detecting multiple structures of interest from the reconstructed multiple tomographic images, and determining a priority for each of the detected structures of interest of the same type using its interrelationship with other structures of interest. Effect of the Invention

[0020] According to the present disclosure, the relative merits of structures of interest of the same type can be determined by taking into account the interrelationships between the structures of interest. [Brief description of the drawings]

[0021] [Figure 1] 1 is a schematic configuration diagram of a radiation image capturing system to which an image processing device according to an embodiment is applied; [Diagram 2] 2 is a diagram showing the mammography apparatus according to the embodiment as viewed in the direction of arrow A in FIG. [Diagram 3] 1 is a block diagram showing an example of an electrical configuration of an image processing apparatus according to an embodiment; [Figure 4] FIG. 2 is a functional block diagram showing a functional configuration of the image processing apparatus according to the embodiment. [Diagram 5] 4A and 4B are diagrams for explaining acquisition of a projection image according to the embodiment. [Figure 6] 4A to 4C are diagrams for explaining generation of a tomographic image according to the embodiment. [Figure 7] FIG. 11 is a diagram for explaining a method for determining a priority according to an embodiment. [Figure 8] FIG. 11 is a diagram for explaining a method for determining a priority according to an embodiment. [Figure 9] FIG. 11 is a diagram for explaining a method for determining a priority according to an embodiment. [Figure 10] FIG. 11 is a diagram for explaining a method for determining a priority according to an embodiment. [Figure 11] FIG. 11 is a diagram for explaining a method for determining a priority according to an embodiment. [Figure 12] 5 is a flowchart illustrating an example of image processing according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a schematic diagram of a radiographic imaging system 90 to which an image processing device 4 according to an embodiment of the present disclosure is applied, and Fig. 2 is a diagram of a mammography device 1 in the radiographic imaging system 90 as viewed from the direction of arrow A in Fig. 1.

[0023] 1, a radiation image capturing system 90 according to this embodiment is for capturing images of a subject, a breast M, from multiple radiation source positions to obtain multiple radiation images, i.e., multiple projection images, in order to perform tomosynthesis imaging of the breast and generate a tomographic image. The radiation image capturing system 90 according to this embodiment includes a mammography device 1, a console 2, an image storage system 3, and an image processing device 4.

[0024] The mammography device 1 has an arm unit 12 connected to a base (not shown) by a rotation shaft 11. An imaging table 13 is attached to one end of the arm unit 12, and a radiation irradiation unit 14 is attached to the other end so as to face the imaging table 13. The arm unit 12 is configured so that only the end to which the radiation irradiation unit 14 is attached can rotate, which makes it possible to rotate only the radiation irradiation unit 14 while fixing the imaging table 13.

[0025] A radiation detector 15 such as a flat panel detector is provided inside the imaging stand 13. The radiation detector 15 has a radiation detection surface 15A. Also provided inside the imaging stand 13 are a circuit board and the like on which are provided a charge amplifier that converts a charge signal read out from the radiation detector 15 into a voltage signal, a correlated double sampling circuit that samples the voltage signal output from the charge amplifier, and an AD (Analog-Digital) conversion unit that converts the voltage signal into a digital signal.

[0026] A radiation source 16 is housed inside the radiation irradiation unit 14. The radiation source 16 emits X-rays as radiation, and the timing at which radiation is emitted from the radiation source 16 and the radiation generation conditions in the radiation source 16, i.e., the selection of the target and filter materials, the tube voltage, the irradiation time, and the like, are controlled by the console 2.

[0027] The arm 12 is also provided with a compression plate 17 that is disposed above the imaging table 13 and presses down on the breast M, a support section 18 that supports the compression plate 17, and a movement mechanism 19 that moves the support section 18 in the vertical direction in Figures 1 and 2. The distance between the compression plate 17 and the imaging table 13, i.e., the compression thickness, is input to the console 2.

[0028] The console 2 has a function of controlling the mammography apparatus 1 using imaging orders and various information acquired from a RIS (Radiology Information System) (not shown) or the like via a network such as a wireless communication LAN (Local Area Network) and using instructions given directly by a technician or the like. Specifically, the console 2 causes the mammography apparatus 1 to perform tomosynthesis imaging of the breast M, thereby acquiring a plurality of projection images, as will be described later. As an example, in this embodiment, a server computer is used as the console 2.

[0029] The image storage system 3 is a system that stores image data such as a projection image captured by the mammography device 1. The image storage system 3 extracts from the stored images an image corresponding to a request from the console 2, the image processing device 4, etc., and transmits it to the device that issued the request. A specific example of the image storage system 3 is a PACS (Picture Archiving and Communication System).

[0030] Next, the image processing device 4 according to the present embodiment will be described. First, the hardware configuration of the image processing device 4 according to the present embodiment will be described with reference to FIG. 3. As shown in FIG. 3, the image processing device 4 is a computer such as a workstation, a server computer, or a personal computer, and includes a CPU (Central Processing Unit) 21, a non-volatile storage 23, and a memory 26 as a temporary storage area. The image processing device 4 also includes a display 24 such as a liquid crystal display, an input device 25 such as a keyboard and a mouse, and a network I / F (InterFace) 27 connected to a network (not shown). The CPU 21, the storage 23, the display 24, the input device 25, the memory 26, and the network I / F 27 are connected to a bus 28. The CPU 21 is an example of a processor in the present disclosure.

[0031] The storage 23 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 23 as a storage medium stores the image processing program 22 installed in the image processing device 4. The CPU 21 reads the image processing program 22 from the storage 23, expands it in the memory 26, and executes the expanded image processing program 22.

[0032] The image processing program 22 is stored in a storage device of a server computer connected to the network or in a network storage in an externally accessible state, and is downloaded and installed in a computer constituting the image processing device 4 upon request. Alternatively, the image processing program 22 is recorded on a recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) and distributed, and is installed in a computer constituting the image processing device 4 from the recording medium.

[0033] Next, a functional configuration of the image processing device 4 according to this embodiment will be described. Fig. 4 is a diagram showing the functional configuration of the image processing device 4 according to this embodiment.

[0034] 4, the image processing device 4 according to this embodiment includes an image acquiring unit 30, a tomographic image reconstructing unit 31, a structure of interest detecting unit 32, a priority determining unit 33, an image creating unit 34, and a display control unit 35. The CPU 21 executes the image processing program 22 to function as the image acquiring unit 30, the tomographic image reconstructing unit 31, the structure of interest detecting unit 32, the priority determining unit 33, the image creating unit 34, and the display control unit 35.

[0035] The image acquisition unit 30 acquires a plurality of projection images acquired by the console 2 causing the mammography apparatus 1 to perform tomosynthesis imaging of the breast M. The image acquisition unit 30 acquires the projection images from the console 2 or the image storage system 3 via the network I / F 27. In addition, the tomographic image reconstruction unit 31 reconstructs a plurality of tomographic images from the projection images acquired by the image acquisition unit 30.

[0036] Here, tomosynthesis imaging and generation of tomographic images will be described. When performing tomosynthesis imaging to generate tomographic images, the console 2 moves the radiation source 16 by rotating the arm unit 12 around the rotation axis 11, irradiates the breast M, which is the subject, with radiation at a plurality of radiation source positions by the movement of the radiation source 16 under predetermined imaging conditions for tomosynthesis imaging, detects the radiation transmitted through the breast M with the radiation detector 15, and obtains a plurality of projection images Gi (i=1 to n, n is the number of radiation source positions, for example, n=15) at the plurality of radiation source positions.

[0037] Fig. 5 is a diagram for explaining acquisition of projection images Gi. As shown in Fig. 5, radiation source 16 is moved to each radiation source position S1, S2, ..., Sn, and radiation source 16 is driven at each radiation source position to irradiate breast M with radiation, and X-rays that have passed through breast M are detected by radiation detector 15, thereby acquiring projection images G1, G2, ..., Gn corresponding to each radiation source position S1 to Sn. Note that the same dose of radiation is irradiated to breast M at each radiation source position S1 to Sn.

[0038] 5, radiation source position Sc is a radiation source position where the optical axis X0 of radiation emitted from the radiation source 16 is perpendicular to the detection surface 15A of the radiation detector 15. The radiation source position Sc is referred to as the reference radiation source position Sc.

[0039] Then, the tomographic image reconstruction unit 31 reconstructs the multiple projection images Gi to generate a tomographic image in which a desired cross-sectional plane of the breast M is emphasized. Specifically, the image processing device 4 reconstructs the multiple projection images Gi using a well-known back projection method such as a simple back projection method or a filtered back projection method, and generates multiple cross-sectional images Dj (j=1 to m) in each of the multiple cross-sectional planes of the breast M, as shown in Fig. 6. At this time, a three-dimensional coordinate position in a three-dimensional space including the breast M is set, and pixel values ​​of pixels corresponding to the set three-dimensional coordinate position in the multiple projection images Gi are reconstructed, and the pixel value of the coordinate position is calculated.

[0040] The structure of interest detection unit 32 detects structures of interest from the multiple tomographic images Dj. In this embodiment, calcifications, masses, spicules, and mammary glands contained in the breast M are detected as structures of interest.

[0041] The structure of interest detection unit 32 according to this embodiment detects the structure of interest using a machine learning model 32A consisting of a neural network that has been machine-learned by deep learning or the like using teacher data so as to extract the structure of interest from each of the tomographic images Dj. That is, the machine learning model 32A according to this embodiment is pre-machine-learned using information indicating the tomographic image as input information and information indicating a plurality of types of structures of interest as output information.

[0042] As the machine learning model 32A, for example, a well-known neural network such as a Convolutional Neural Network (CNN) or a Support Vector Machine (SVM) can be used.

[0043] In this embodiment, a single machine learning model capable of detecting multiple types of structures of interest is applied as the machine learning model 32A, but the present invention is not limited to this. For example, a machine learning model 32A may be prepared and applied for each type of target structure of interest.

[0044] In this manner, in this embodiment, the structure of interest is detected by the machine learning model 32A, but the present invention is not limited to this. For example, a known computer-aided image diagnosis (i.e., CAD) algorithm may be used to detect the structure of interest from the tomographic image Dj. In the CAD algorithm, a probability (likelihood) that indicates that a pixel in the tomographic image Dj is a structure of interest is derived, and a pixel with the probability equal to or greater than a predetermined threshold is detected as the structure of interest. Note that a CAD algorithm is prepared for each type of structure of interest. When a CAD algorithm is applied in this embodiment, a CAD algorithm for mass detection, a CAD algorithm for spicule detection, a CAD algorithm for calcification detection, and a CAD algorithm for mammary gland detection are prepared.

[0045] Furthermore, the detection of the structure of interest is not limited to using the machine learning model 32A or using CAD. For example, the structure of interest may be detected from the tomographic image Dj by pattern matching using a template image of the target structure of interest, or by filtering processing using a filter for detecting the structure of interest. Here, when pattern matching is applied, an already acquired image of the type of structure of interest to be extracted is applied as the template image.

[0046] In addition, in this embodiment, the structure of interest is detected using only the machine learning model 32A, but this is not limited to this. For example, the structure of interest may be detected using both the machine learning model 32A and pattern matching. Furthermore, in this embodiment, the method of detecting the structure of interest may be switched depending on the type of the structure of interest to be detected.

[0047] As described above, in this embodiment, all of calcification, mass, spicules, and mammary glands are applied as the target structures of interest, but the present invention is not limited to this. For example, any one of calcification, mass, spicules, and mammary glands, or a combination of multiple structures excluding all of them may be applied as the target structures of interest.

[0048] The priority determination unit 33 determines a priority for each of the structures of interest of the same type among the multiple structures of interest detected by the structure of interest detection unit 32, using the interrelationships with other structures of interest.

[0049] In this embodiment, both the perspective relationship of the three-dimensional distance with the other structure of interest and the size relationship with the other structure of interest are applied as the above-mentioned mutual relationship, but this is not limited thereto. For example, only one of the perspective relationship and the size relationship may be applied as the above-mentioned mutual relationship. The method of determining the priority by the priority determination unit 33 will be described in detail later.

[0050] The image creation unit 34 uses the priorities determined by the priority determination unit 33 to create a composite two-dimensional image of the subject (in this embodiment, the breast M).

[0051] In the image creation unit 34 according to this embodiment, a composite two-dimensional image is created by taking a weighted average for each pixel, with a weight value (e.g., 0.9) of a tomographic image including a structure of interest with a high priority determined by the priority determination unit 33 being set to a value greater than a weight value (e.g., 0.1) of a tomographic image other than the tomographic image. However, the method of creating the composite two-dimensional image is not limited to this averaging method, and an addition method, a maximum value projection method, a minimum value projection method, or the like may be applied as a method of creating the composite two-dimensional image.

[0052] For example, when applying minimum intensity projection as a method for creating a composite 2D image, an example of a form in which only the tomographic images that include the structure of interest with the above-mentioned high priority are targeted among the tomographic images to be projected can be mentioned.

[0053] In this manner, in the present embodiment, a composite two-dimensional image is applied as the image created by the image creation unit 34, but the present invention is not limited to this. For example, a slab image, which is an image having thickness information, may be applied as the image created by the image creation unit 34. In this case, an example of the form may be a form in which a slab image is created using only tomographic images including the structure of interest having a high priority.

[0054] The display control unit 35 displays the composite two-dimensional image generated by the image generation unit 34 on the display 24. This composite two-dimensional image is generated by increasing the weighting value for the tomographic image including the structure of interest with high priority compared to the other tomographic images. Therefore, the composite two-dimensional image displayed here is an extremely high quality image in which the structure of interest with high priority is emphasized.

[0055] In addition, the images created by the image creation unit 34 are not limited to the above-mentioned composite two-dimensional image and slab image, but may also be images for displaying a composite two-dimensional image created from multiple sectional images Dj in conjunction with the corresponding sectional images Dj.

[0056] That is, in general, the number of tomographic images may reach 100 or more depending on the application. Therefore, there is a problem that a relatively long time is required to interpret all the tomographic images.

[0057] Therefore, in this embodiment, the CPU 21 first creates and displays a composite two-dimensional image as described above. In response to this, the operator refers to the displayed composite two-dimensional image and specifies a structure of interest when the structure of interest displayed in the composite two-dimensional image is suspected to be a lesion. In response to this specification, the CPU 21 displays a tomographic image including the specified structure of interest. This makes it possible to significantly reduce the time required to interpret the tomographic image.

[0058] Next, a method of determining priorities by the priority determination unit 33 according to this embodiment will be specifically described with reference to Fig. 7 to Fig. 11. Fig. 7 to Fig. 11 are diagrams for explaining the method of determining priorities according to this embodiment, and here, a case will be described in which three types of structures of interest, namely calcification, mass, and spicules, are included.

[0059] In the example shown in Fig. 7, as shown in the upper diagram of Fig. 7, in slices 70 corresponding to each of the tomographic images Dj, the structures of interest, masses 60A, 60B, spicules 62, and calcifications 64A, 64B, are present in the depth direction in which the slices are aligned, respectively. When it is assumed that a composite two-dimensional image created by combining the tomographic images Dj in this state in the depth direction is viewed from the front, the state shown in the lower diagram of Fig. 7, that is, all the structures of interest are superimposed on a one-dimensional axis, is obtained.

[0060] In this case, when the composite two-dimensional image is created by combining all the tomographic images Dj at the same ratio (weight value), the densities of all the structures of interest are averaged and lightened in the composite two-dimensional image, as shown in Fig. 8 as an example. Therefore, when there are a plurality of structures of interest of the same type, for example, when one wishes to perform image interpretation focusing on one of the structures of interest, it is difficult to distinguish the structure of interest from the other structures of interest, which causes a problem in that the image interpretation is difficult.

[0061] Therefore, in the priority determination unit 33 according to this embodiment, as described above, the priority of each of the structures of interest of the same type is determined using the above-mentioned predetermined perspective relationship and the above-mentioned size relationship for each of the structures of interest of the same type.

[0062] The example shown in FIG. 9 is an example in which there are two masses 60 (60A, 60B) and two calcifications 64 (64A, 64B), and only one spicula 62. In this case, when it is desired to focus on calcification 64 during image interpretation, and when the perspective relationship is applied such that calcification 64 overlaps with other structures of interest in a three-dimensional manner, calcification 64B overlaps with mass 60A in three-dimensional space. For this reason, the priority of calcification 64B is set higher than that of other structures of interest. As a result, a composite two-dimensional image created using this priority has only calcification 64B emphasized, as shown in the lower diagram of FIG. 9 as an example.

[0063] On the other hand, the example shown in FIG. 10 is an example in which there are five calcifications 64 (64C to 64G) and only one mass 60. In this case, when it is desired to perform image interpretation focusing on the calcification 64 and when the perspective relationship is applied that the calcifications are three-dimensionally close to other structures of interest, the calcifications 64E, 64F, and 64G are close to each other in the three-dimensional space. Therefore, the priority of the calcifications 64E, 64F, and 64G is set higher than that of the other structures of interest. As a result, the composite two-dimensional image created using this priority has only the calcifications 64E, 64F, and 64G emphasized, as shown in the lower diagram of FIG. 10 as an example.

[0064] Furthermore, the example shown in FIG. 11 has two masses 60 (60A, 60B) and two calcifications 64 (64A, 64B), and only one spicule 62, which is the same as the example shown in FIG. 9. In this case, when it is desired to perform image interpretation focusing on the calcification 64, and when the relationship that the calcification 64 is three-dimensionally smaller than other structures of interest (in this embodiment, the volume is one-tenth or less of the largest structure of interest) is applied as the above-mentioned size relationship, the calcifications 64A and 64B are smaller than one-tenth of the mass 60A. Therefore, the priority of the calcifications 64A and 64B is set higher than the other structures of interest. As a result, the composite two-dimensional image created using this priority has only the calcifications 64A and 64B emphasized, as shown in the lower diagram of FIG. 11 as an example. In addition, the above-mentioned perspective relationship can also be exemplified by calcifications existing along the mammary gland.

[0065] In this embodiment, the priority determined by the priority determination unit 33 is reflected in the tomographic image Dj, and then a composite 2D image is created to emphasize the structure of interest having a higher priority. However, the present invention is not limited to this. For example, a composite 2D image may be created provisionally without considering the priority, and the composite 2D image may be processed to emphasize the structure of interest having a higher priority, to create a final composite 2D image. Examples of the method of emphasizing the structure of interest in this embodiment include a mode in which only the region of the structure of interest (hereinafter referred to as the "emphasis target region") is made brighter than the surrounding region of the emphasis target region, a mode in which only the emphasis target region is displayed in a different display state such as blinking or reverse display, and a mode in which only the emphasis target region is displayed surrounded by a circumscribing rectangular frame or a circumscribing circular frame.

[0066] Next, the processing performed in this embodiment will be described. Fig. 12 is a flowchart showing the image processing performed in this embodiment. Note that, here, it is assumed that a plurality of projection images Gi are acquired in advance and stored in the storage 23. Also, here, a case will be described in which information indicating the type of the structure of interest to be emphasized and information indicating the above-mentioned interrelationship are set in advance.

[0067] When the input device 25 receives an instruction from an operator to start processing, the image processing program 22 starts to be executed, and thus the image processing shown in FIG. 12 starts.

[0068] First, the image acquisition unit 30 acquires projection images Gi obtained by photographing a subject (in this embodiment, a breast M) from multiple angles using radiation emitted from the radiation source 16 by reading them from the storage 23 (step 100). Next, the tomographic image reconstruction unit 31 reconstructs multiple tomographic images Dj from the acquired projection images Gi (step 102).

[0069] Next, the structure of interest detection unit 32 detects a plurality of structures of interest from the reconstructed plurality of tomographic images Dj (step 104). Next, the priority determination unit 33 determines the priority of each of the detected structures of interest of the same type by using the interrelationship with other structures of interest (step 106).

[0070] Then, the image creation unit 34 creates a composite two-dimensional image from the multiple tomographic images Dj using the determined priorities (step 108). Next, the display control unit 35 displays the composite two-dimensional image on the display 24 (step 110), and the process ends.

[0071] As described above, the image processing device according to the present embodiment acquires projection images obtained by photographing a subject from multiple angles using radiation, reconstructs multiple tomographic images from the acquired projection images, detects multiple structures of interest from the reconstructed multiple tomographic images, and determines the priority of each of the detected structures of interest of the same type using the interrelationships with other structures of interest. Therefore, the superiority or inferiority of structures of interest of the same type can be determined taking into account the interrelationships between the structures of interest.

[0072] Moreover, according to the image processing device of the present embodiment, a composite 2D image of a subject is created using the determined priority, and therefore, the visibility of a structure of interest in the composite 2D image can be improved compared to a case where the determined priority is not used.

[0073] Moreover, according to the image processing device of this embodiment, the above-mentioned mutual relationship is at least one of a perspective relationship of a three-dimensional distance with another structure of interest and a size relationship with another structure of interest. Therefore, when the perspective relationship is applied as the mutual relationship, it is possible to obtain a priority of the structure of interest that should be prioritized in terms of the perspective relationship, and when the size relationship is applied as the mutual relationship, it is possible to obtain a priority of the structure of interest that should be prioritized in terms of the size relationship.

[0074] Moreover, according to the image processing device of the present embodiment, at least one of calcification, mass, spicula, and mammary gland is applied as a structure of interest. Therefore, for the applied type of structure of interest, the relative merits can be determined in consideration of the interrelationships between the structures of interest.

[0075] Furthermore, according to the image processing device of this embodiment, the detection of the structure of interest is performed using a machine learning model that has been trained in advance using information indicating a tomographic image as input information and information indicating a plurality of structures of interest as output information. Therefore, the structure of interest can be detected with higher accuracy than when the structure of interest is detected by pattern matching.

[0076] Furthermore, the image processing device according to the present embodiment also illustrates an aspect in which the detection of the structure of interest is performed by pattern matching using a template image of the target structure of interest. Therefore, according to this aspect, the structure of interest can be detected more easily than when the structure of interest is detected using a machine learning model.

[0077] In the above embodiment, the CPU 21 provided in the image processing device 4 is used as the processor of the technology of the present disclosure, but the present disclosure is not limited to this. For example, a CPU provided in any of the mammography device 1, the console 2, and the image storage system 3 may be used as the processor of the technology of the present disclosure.

[0078] In the above embodiment, the perspective relationship and the size relationship are applied independently, but the present invention is not limited to this. For example, the perspective relationship and the size relationship may be applied in combination. Examples of this embodiment include a form in which the priority is determined based on the perspective relationship first, and then the priority is determined based on the size relationship for multiple structures of interest that have the same priority, or a form in which the priority is determined based on the size relationship first, and then the priority is determined based on the perspective relationship for multiple structures of interest that have the same priority.

[0079] 7 to 11, the embodiment has been described with respect to the case where only calcification is emphasized as the structure of interest, but the present invention is not limited to this. Any one of a mass, a spicule, and a mammary gland, or a combination of a plurality of structures of interest including calcification may be applied as the structure of interest to be emphasized.

[0080] In the above embodiment, the case where a relatively small structure of interest is prioritized as the size relationship has been described, but the present invention is not limited to this. For example, a relationship in which a relatively large structure of interest is prioritized as the size relationship may be applied.

[0081] In the above embodiment, the case where the priority level has only two levels, namely, "preferred" and "non-preferred", is described, but the present invention is not limited to this. For example, the priority level may have three or more levels.

[0082] In the above embodiment, the following various processors can be used as the hardware structure of the processing unit that executes various processes, such as the image acquisition unit 30, the tomographic image reconstruction unit 31, the structure of interest detection unit 32, the priority determination unit 33, the image creation unit 34, and the display control unit 35. As described above, the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically for executing specific processes.

[0083] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0084] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as represented by computers such as client and server. Second, there is a form in which a processor is used to realize the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as represented by a system on chip (SoC). In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0085] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0086] In the above embodiment, the image processing program 22 is pre-stored (installed) in the storage 23 of the image processing device 4, but the present invention is not limited to this. The image processing program 22 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 22 may be downloaded from an external device via a network.

[0087] From the above description, the invention described in the following appendix can be understood.

[0088] [Appendix 1] at least one processor; The processor, A projection image is obtained by photographing a subject from multiple angles using radiation. Reconstructing a plurality of tomographic images from the acquired projection images; Detecting a plurality of structures of interest from the reconstructed tomographic images; determining a priority of each of the detected structures of interest of the same type using a correlation between the structures of interest and other structures of interest; Image processing device. [Appendix 2] The processor, generating a composite two-dimensional image of the subject using the determined priorities; 2. An image processing device as described in claim 1. [Appendix 3] The interrelationship is at least one of a three-dimensional perspective relationship with the other structure of interest and a size relationship with the other structure of interest. 3. An image processing device according to claim 1 or 2. [Appendix 4] The mutual relationship is both the perspective relationship and the size relationship. 4. The image processing device according to claim 3. [Appendix 5] The structure of interest is at least one of a calcification, a mass, a spicule, and a mammary gland. 5. An image processing device according to any one of claims 1 to 4. [Appendix 6] The processor, The detection of the structure of interest is performed using a machine learning model that has been machine-learned in advance, with information indicating the tomographic image as input information and information indicating the plurality of structures of interest as output information. 6. An image processing device according to any one of claims 1 to 5. [Appendix 7] The processor, The detection of the structure of interest is performed by pattern matching using a template image of the target structure of interest. 7. An image processing device according to any one of claims 1 to 6. [Appendix 8] An image processing device according to claim 1; a mammography device for acquiring projection images used by the image processing device; A radiation imaging system comprising: [Appendix 9] A projection image is obtained by photographing a subject from multiple angles using radiation. Reconstructing a plurality of tomographic images from the acquired projection images; Detecting a plurality of structures of interest from the reconstructed tomographic images; determining a priority of each of the detected structures of interest of the same type using a correlation between the structures of interest and other structures of interest; A program that causes a computer to carry out processing. [Explanation of symbols]

[0089] 1. Mammography equipment 2 Console 3. Image storage system 4. Image Processing Device 11 Rotation axis 12 Arm section 13. Shooting stand 14 Radiation Unit 15 Radiation detector 15A Detection surface 16 Radiation source 17 Compression plate 18 Support part 19 Moving mechanism 21 CPU 22 Image Processing Programs 23. Storage 24 Display 25 Input Devices 26 Memory 27 Network Interface 28 Bus 30 Image acquisition unit 31 Tomographic image reconstruction unit 32 Interesting Structure Detection Unit 32A Machine Learning Model 33 Priority determination section 34 Image Creation Department 35 Display control section 60, 60A, 60B mass 62 Spicula 64, 64A~64G Calcification 70 Fault 90 Radiation Imaging System Dj(j=1~m), Dj tomographic image Gi(i=1~n), Gi projection image M Breast Si(i=1~n) Source position Sc reference source position X0 optical axis

Claims

1. At least one processor; The processor, A projection image is obtained by photographing a subject from multiple angles using radiation. Reconstructing a plurality of tomographic images from the acquired projection images; Detecting a plurality of structures of interest from the reconstructed tomographic images; determining a priority of each of the detected structures of interest of the same type using a correlation between the structures of interest and other structures of interest; Image processing device.

2. The processor, generating a composite two-dimensional image of the subject using the determined priorities; The image processing device according to claim 1 .

3. The interrelationship is at least one of a three-dimensional perspective relationship with the other structure of interest and a size relationship with the other structure of interest. The image processing device according to claim 1 .

4. The mutual relationship is both the perspective relationship and the size relationship. The image processing device according to claim 3 .

5. the structure of interest is at least one of a calcification, a mass, a spicule, and a mammary gland; The image processing device according to claim 1 .

6. The processor, The detection of the structure of interest is performed using a machine learning model that has been machine-trained in advance, with information indicating the tomographic image as input information and information indicating the plurality of structures of interest as output information. The image processing device according to claim 1 .

7. The processor, The detection of the structure of interest is performed by pattern matching using a template image of the target structure of interest. The image processing device according to claim 1 .

8. The image processing device according to claim 1 ; a mammography device for acquiring projection images used by the image processing device; A radiation imaging system comprising:

9. A projection image is obtained by photographing a subject from multiple angles using radiation. Reconstructing a plurality of tomographic images from the acquired projection images; Detecting a plurality of structures of interest from the reconstructed tomographic images; determining a priority of each of the detected structures of interest of the same type using a correlation between the structures of interest and other structures of interest; A program that causes a computer to carry out processing.

Images