Image processing device, medical image imaging system, image processing method, and image processing program

Abstract

This image processing device includes a processor and the processor: acquires a radiographic image capturing a breast in a state compressed by a compression member; acquires a plurality of ultrasound images capturing the breast in a state compressed by the compression member in what can be considered the same state as in the radiographic image; and superimposes the contour of the breast included in a radiographic image for superimposition corresponding to the radiographic image, on an ultrasound image for display corresponding to the plurality of ultrasound images.

Description

Image processing device, medical image capturing system, image processing method, and image processing program

[0001] The present disclosure relates to an image processing device, a medical image capturing system, an image processing method, and an image processing program.

[0002] When observing a radiographic image of a breast, an ultrasound image of the breast is also referred to. For example, Japanese Patent Application Laid-Open No. 2005-125080 describes a method for observing a region of interest in a radiographic image and a region of interest in an ultrasound image.

[0003] Incidentally, there are cases where radiographic images and ultrasound images are interpreted in a manner that considers the compression state to be the same, such as when radiographic images and ultrasound images are taken consecutively while the breast is compressed by a compression member. However, ultrasound images may fail to capture the entire breast, resulting in missing images. For example, the breast may not be in contact with the compression member at the tip of the breast, resulting in missing images in the area where air exists between the ultrasound probe and the breast. Such missing images in the ultrasound images make it difficult to compare the radiographic and ultrasound images.

[0004] The present disclosure provides an image processing device, a medical image capturing system, an image processing method, and an image processing program that make it easy to compare a radiographic image with an ultrasound cross-sectional image even when there is a missing image in the ultrasound image.

[0005] An image processing device according to a first aspect of the present disclosure includes a processor, which acquires a radiographic image captured with the breast compressed by a compression member, acquires a plurality of ultrasound images captured with the breast compressed by the compression member in a compression state that can be considered identical to the radiographic image, and superimposes a contour of the breast included in a radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

[0006] An image processing device of a second aspect of the present disclosure is an image processing device of the first aspect, in which the processor accepts a specification of the breast depth, and the ultrasound image to be displayed is a cross-sectional image that is synthesized from multiple ultrasound images and is approximately parallel to the imaging table according to the accepted breast depth.

[0007] An image processing device of a third aspect of the present disclosure is an image processing device of the first aspect, in which the ultrasound image for display is any one of a plurality of ultrasound images, and is at least one of a first cross-sectional ultrasound image which is a cross-sectional image in a direction intersecting with the imaging table on which the breast is placed, a second cross-sectional ultrasound image which is synthesized from the plurality of ultrasound images and is a cross-sectional image in a direction intersecting with the imaging table and the first cross-sectional ultrasound image, and a third cross-sectional ultrasound image which is synthesized from the plurality of ultrasound images and is a cross-sectional image approximately parallel to the imaging table.

[0008] An image processing device of a fourth aspect of the present disclosure is the image processing device of the first aspect, wherein the radiological image is a projection image obtained by tomosynthesis imaging, and the radiological image to be superimposed is a radiological tomographic image at a height corresponding to the ultrasound image to be displayed, among a plurality of radiological tomographic images reconstructed from the projection images.

[0009] An image processing device according to a fifth aspect of the present disclosure is the image processing device according to the first aspect, wherein the processor also displays a radiation image for superimposition that includes the contour of the breast.

[0010] An image processing device of a sixth aspect of the present disclosure is the image processing device of the fifth aspect, wherein the processor displays the radiation image for superimposition with the contour of the breast contained in the ultrasound image for display superimposed thereon.

[0011] An image processing device of a seventh aspect of the present disclosure is an image processing device of the first aspect, in which the processor further displays a schematic diagram to which information representing the area that is the difference between the area of ​​the breast included in the ultrasound image for display and the area of ​​the breast included in the radiation image for superimposition has been added.

[0012] An image processing device of an eighth aspect of the present disclosure is the image processing device of the seventh aspect, wherein the information includes a numerical value representing the distance between the breast contour included in the ultrasound image for display and the breast contour included in the radiation image for superimposition.

[0013] A medical imaging system according to a ninth aspect of the present disclosure includes the image processing device of the present disclosure, a radiographic imaging device, and an ultrasonic imaging device.

[0014] In addition, an image processing method of a tenth aspect of the present disclosure includes a processor provided in an image processing device, which acquires a radiographic image of the breast compressed by a compression member, acquires a plurality of ultrasound images of the breast compressed by a compression member in a state that can be considered identical to the radiographic image, and superimposes the contour of the breast contained in a radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

[0015] In addition, the image processing program of an eleventh aspect of the present disclosure causes a processor provided in an image processing device to perform the following process: acquire a radiographic image captured with the breast compressed by a compression member; acquire multiple ultrasound images captured with the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image; and superimpose the outline of the breast contained in the radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the multiple ultrasound images.

[0016] According to the present disclosure, even when an image defect occurs in the ultrasound image, it is possible to easily compare the radiographic image with the ultrasound cross-sectional image.

[0017] 1 is a configuration diagram schematically illustrating an example of the overall configuration of an imaging system according to an embodiment; FIG. 2 is a side view illustrating an example of the appearance of a mammography apparatus according to an embodiment; FIG. 3 is a block diagram illustrating an example of the configuration of an ultrasound imaging apparatus according to an embodiment; FIG. 4 is a block diagram illustrating an example of the configuration of an image processing device according to an embodiment; FIG. 5 is a functional block diagram illustrating an example of the configuration of an image processing device according to an embodiment; FIG. 6 is a flowchart illustrating an example of the flow of image processing in the image processing device according to an embodiment; FIG. 7 is a diagram illustrating an example of the display state of an ultrasound cross-sectional image with a breast contour superimposed thereon and a radiation image according to an embodiment; FIG. 8 is a diagram illustrating an example of the display state of an ultrasound cross-sectional image with a breast contour superimposed thereon and a radiation tomographic image according to an embodiment;

[0018] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the embodiment.

[0019] First, an example of the overall configuration of the medical image capturing system of this embodiment will be described. Fig. 1 shows a configuration diagram illustrating an example of the overall configuration of a medical image capturing system 1 of this embodiment.

[0020] As shown in FIG. 1, the medical image capturing system 1 of this embodiment includes a radiation image capturing system 2, an ultrasonic image capturing device 16, an image processing device 18, and an image storage system 19.

[0021] First, a description will be given of the configuration of the radiation image capturing system 2. The radiation image capturing system 2 includes a mammography apparatus 10 and a console 12.

[0022] The mammography device 10 of this embodiment is a device that takes a radiographic image of a subject's breast by irradiating the breast with radiation R (e.g., X-rays). Note that the mammography device 10 may be a device that takes images of the subject's breast not only when the subject is standing (standing position), but also when the subject is sitting in a chair (including a wheelchair) or the like (seated position).

[0023] Figure 2 shows a side view of an example of the appearance of the mammography device 10 of this embodiment. Note that Figure 2 is a side view of the mammography device 10 as seen from the right side of the subject. As shown in Figure 2, the mammography device 10 includes a radiation source 36R, a radiation detector 30, an imaging table 40 disposed between the radiation source 36R and the radiation detector 30, and a compression member 34 that compresses the breast between the radiation source 36R and the imaging table 40.

[0024] The imaging stand 40 includes a control unit 20, a storage unit 22, an I / F (Interface) unit 24, an operation unit 26, and a radiation detector 30. The control unit 20 controls the overall operation of the mammography apparatus 10 in accordance with the control of the console 12. The control unit 20 includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc. (not shown). The ROM stores various programs in advance, including a program executed by the CPU for controlling radiographic imaging. The RAM temporarily stores various data.

[0025] The storage unit 22 stores image data of radiographic images and various other information. The storage unit 22 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. Note that, hereinafter, "image data of radiographic images" will be simply referred to as "radiographic images." Similarly, "image data of ultrasound images" will be simply referred to as "ultrasound images."

[0026] The I / F unit 24 communicates various types of information with the console 12 via wired or wireless communication. Specifically, the I / F unit 24 receives information related to the control of the mammography apparatus 10 from the console 12. The I / F unit 24 also transmits radiographic images to the console 12.

[0027] The operation unit 26 is a part provided on the imaging stand 40 or the like that can be operated by the user with the hands or feet, and is, for example, a switch, a button, a touch panel, etc. Furthermore, for example, the operation unit 26 may receive voice input from the user.

[0028] The radiation detector 30 is disposed inside the imaging table 40 and detects radiation R that has passed through the breast, which is the subject. When imaging is performed in the mammography device 10 of this embodiment, the subject's breast is positioned on the imaging surface 40A of the imaging table 40 by a user such as a doctor or technician. The imaging surface 40A, which comes into contact with the subject's breast, is made of, for example, carbon, from the viewpoints of radiation R transparency and strength.

[0029] The radiation detector 30 detects radiation R that has passed through the subject's breast and the imaging table 40, generates a radiographic image based on the detected radiation R, and outputs the generated radiographic image. The type of radiation detector 30 in this embodiment is not particularly limited, and may be, for example, an indirect conversion type radiation detector that converts radiation R into light and then converts the converted light into electric charges, or a direct conversion type radiation detector that directly converts radiation R into electric charges.

[0030] The radiation source 36R is provided in the radiation irradiation unit 36. As shown in FIG. 2, the radiation irradiation unit 36 ​​is provided on an arm 42 together with an imaging table 40 and a compression unit 46. As shown in FIG. 2, the mammography apparatus 10 of this embodiment includes the arm 42, a base 44, and a shaft 45. The arm 42 is held by the base 44 so as to be movable in the vertical direction (Z-axis direction). The shaft 45 connects the arm 42 to the base 44. The arm 42 is rotatable relative to the base 44, with the shaft 45 serving as the axis of rotation.

[0031] 2 , the compression member 34 is attached to the compression unit 46. The compression unit 46 and the arm 42 are rotatable independently relative to the base 44, with the shaft 45 as a rotation axis. In this embodiment, the shaft 45, the arm 42, and the compression unit 46 are each provided with a gear (not shown), and the arm 42 and the compression unit 46 are each connected to the shaft 45 by switching between an engaged state and a disengaged state of these gears. One or both of the arm 42 and the compression unit 46 connected to the shaft 45 rotates integrally with the shaft 45.

[0032] The compression member 34 in this embodiment is a plate-shaped member that moves up and down (in the Z-axis direction) by a compression plate driver (not shown) provided in the compression unit 46, and compresses the breast of the subject between the compression member 34 and the imaging table 40. As shown in Fig. 2, with regard to the direction of movement of the compression member 34, the direction in which the breast is compressed, in other words, the direction toward the imaging surface 40A, is referred to as the "compression direction," and the direction in which compression of the breast is released, in other words, the direction toward the radiation irradiation unit 36, is referred to as the "compression release direction."

[0033] The compression member 34 is preferably optically transparent to allow for easy alignment and confirmation of the compression state during breast compression, and is also formed from a material with excellent radiation R transparency. Furthermore, the compression member 34 is desirably formed from a material that easily transmits ultrasound waves transmitted from the ultrasound probe 55 (see FIG. 3 , described in detail below) of the ultrasound imaging device 16. Examples of materials that can be used for the compression member 34 include resins such as polymethylpentene, polycarbonate, acrylic, and polyethylene terephthalate. Polymethylpentene is particularly suitable for the compression member 34 because it has low rigidity, excellent stretchability and flexibility, and suitable values ​​for acoustic impedance, which affects the reflectivity of ultrasound, and attenuation coefficient, which affects the attenuation of ultrasound. The material constituting the compression member 34 is not limited to that described in this embodiment. For example, the compression member 34 may be a film-like material.

[0034] The compression member 34 is not limited to one that compresses the entire breast, but may also be one that compresses a portion of the breast. In other words, the compression member 34 may be smaller than the breast. For example, a compression member 34 used in so-called spot imaging, which captures a radiographic image of only the area where a lesion is present, is known as such a compression member 34.

[0035] On the other hand, the console 12 of this embodiment has the function of controlling the mammography apparatus 10 using imaging orders and various information acquired from a RIS (Radiology Information System) 5 or the like via a wireless communication LAN (Local Area Network) or the like, and instructions given by the user via an operation unit 56 or the like. The console 12 of this embodiment is, for example, a server computer.

[0036] Next, the configuration of the ultrasound imaging device 16 will be described. Fig. 3 shows a block diagram illustrating an example of the configuration of the ultrasound imaging device 16. The ultrasound imaging device 16 is a device that allows a user to capture ultrasound images of a subject's breast as an object, and is a so-called cart-type ultrasound imaging device. Note that the ultrasound imaging device 16 may also be a so-called compact or handheld type ultrasound imaging device other than the cart type.

[0037] 3, the ultrasonic imaging device 16 includes a control unit 50, a storage unit 52, an I / F unit 54, an ultrasonic probe 55, an operation unit 56, a position detection sensor 57, and a display unit 58. The control unit 50, the storage unit 52, the I / F unit 54, the ultrasonic probe 55, the operation unit 56, the position detection sensor 57, and the display unit 58 are connected via a bus 59 such as a system bus or a control bus so as to be able to exchange various information with each other.

[0038] The control unit 50 of this embodiment controls the overall operation of the ultrasound imaging device 16. The control unit 50 includes a CPU 50A, a ROM 50B, and a RAM 50C. The ROM 50B stores various programs to be executed by the CPU 50A in advance. The RAM 50C temporarily stores various data.

[0039] The captured ultrasound images and various other information are stored in the storage unit 52. Specific examples of the storage unit 52 include an HDD and an SSD.

[0040] The ultrasonic probe 55 is moved by the user along the upper surface 34A of the compression member 34 (see FIG. 2 , the surface opposite to the side that contacts the subject's breast) and scans the breast with ultrasound to obtain an ultrasound image of the breast. Specifically, when ultrasound imaging is performed, the ultrasonic probe 55 is moved by the user along the upper surface 34A of the compression member 34 with an acoustic matching member (not shown) such as echo jelly applied to the upper surface 34A of the compression member 34 or with an acoustic matching member attached to the ultrasonic probe 55.

[0041] The ultrasonic probe 55 includes a plurality of ultrasonic transducers (see ultrasonic transducer 55t in FIG. 11) arranged one-dimensionally or two-dimensionally. Each ultrasonic transducer transmits ultrasonic waves based on an applied drive signal, and outputs a reception signal by receiving ultrasonic echoes.

[0042] Each of the multiple ultrasonic transducers is composed of a vibrator made of a piezoelectric material (piezoelectric body), such as a piezoelectric ceramic, typically PZT (lead zirconate titanate), or a polymer piezoelectric element, typically PVDF (polyvinylidene difluoride), with electrodes formed on both ends. When a pulsed or continuous wave drive signal is sent to the electrodes of the vibrator to apply a voltage, the piezoelectric body expands and contracts. This expansion and contraction generates pulsed or continuous wave ultrasonic waves from each vibrator, and these ultrasonic waves are combined to form an ultrasonic beam. Furthermore, each vibrator expands and contracts upon receiving propagating ultrasonic waves, generating electrical signals. These electrical signals are output as ultrasonic reception signals and input to the main body (not shown) of the ultrasonic imaging device 16 via a cable (not shown).

[0043] The position detection sensor 57 is a sensor for detecting the position of the ultrasonic probe 55. In this embodiment, a six-axis sensor is used as the position detection sensor 57. The six-axis sensor detects the movement direction, orientation, and rotation of the ultrasonic probe 55 and further calculates the movement distance, movement speed, and the like. Specifically, a six-axis sensor is realized by combining an acceleration sensor that can detect three directions (front / back, left / right, and up / down) with a geomagnetic sensor that can detect north / south / east / west, or an acceleration sensor with a gyro sensor that can detect the speed of rotation. Note that the position detection sensor 57 is not limited to the six-axis sensor used in this embodiment. For example, a magnetic sensor such as that described in Japanese Patent Application Laid-Open No. 2011-167331 may also be used as the position detection sensor 57. Furthermore, in this embodiment, a method of using a sensor such as the position detection sensor 57 is described as a method of detecting the position of the ultrasonic probe 55, but the method of detecting the position of the ultrasonic probe 55 is not limited to the method using a sensor. For example, a marker that can detect the position in three axes in space may be attached to the ultrasonic probe 55, and the position of the ultrasonic probe 55 may be identified by analyzing an image captured of the marker. The position detection sensor 57 may be provided inside the ultrasonic probe 55 or may be provided outside the ultrasonic probe 55. Also, unlike the present embodiment, the position detection sensor 57 may be provided separately from the ultrasonic imaging device 16. In any case, as long as the position detection sensor 57 can detect the position of the ultrasonic probe 55, there are no limitations on its arrangement or configuration.

[0044] The control unit 50 identifies the current position of the ultrasound probe 55. For example, the control unit 50 identifies the position of the ultrasound probe 55 relative to the imaging surface 40A of the imaging table 40, the position of the ultrasound probe 55 relative to the upper surface 34A of the compression member 34, or the position of the ultrasound probe 55 relative to the body surface of the breast compressed by the compression member 34.

[0045] In the medical image capturing system 1 of this embodiment, it is possible to associate any position in the ultrasound image captured by the ultrasound imaging device 16 with any position pixel in the radiation image captured by the mammography device 10 based on the position of the ultrasound probe 55. Therefore, the position of the ultrasound probe 55 detected by the position detection sensor 57 is added to the captured ultrasound image and output.

[0046] The operation unit 56 is used by the user to input instructions and various information related to capturing ultrasound images, etc. The operation unit 56 is not particularly limited, and examples thereof include various switches, a touch panel, a touch pen, and a mouse. The display unit 58 displays various information and ultrasound images corresponding to the reception signals received from the ultrasound probe 55. The operation unit 56 and the display unit 58 may be integrated into one unit to form a touch panel display.

[0047] The I / F unit 54 communicates various types of information with the RIS 5 and the image storage system 19 via wireless or wired communication. The ultrasound images captured by the ultrasound imaging device 16 are transmitted to the image storage system 19 via the I / F unit 54 via wireless or wired communication.

[0048] Next, the image storage system 19 will be described. The image storage system 19 is a system that stores radiographic images captured by the radiographic imaging system 2 and ultrasound images captured by the ultrasound imaging device 16. The image storage system 19 is connected to each of the console 12 and the ultrasound imaging device 16 via wireless or wired communication. The image storage system 19 extracts images from the stored radiographic images and ultrasound images in response to requests from the console 12, the ultrasound imaging device 16, and other image reading devices (not shown), and transmits the images to the requesting devices. A specific example of the image storage system 19 is a PACS (Picture Archiving and Communication System).

[0049] Next, we will explain the image processing device 18. The image processing device 18 has a function of acquiring, from the image storage system 19, each of the radiographic images captured by the radiographic imaging system 2 and the ultrasound images captured by the ultrasound imaging device 16, and performing predetermined image processing.

[0050] Fig. 4 is a block diagram showing an example of the configuration of the image processing device 18. As shown in Fig. 4, the image processing device 18 includes a control unit 60, a storage unit 62, an I / F unit 64, an operation unit 66, and a display unit 68. The control unit 60, the storage unit 62, the I / F unit 64, the operation unit 66, and the display unit 68 are connected via a bus 69 such as a system bus or a control bus so as to be able to exchange various information with each other.

[0051] The control unit 60 of this embodiment controls the overall operation of the image processing device 18. The control unit 60 includes a CPU 60A, a ROM 60B, and a RAM 60C. The ROM 60B stores in advance various programs, including an image processing program 61 (described later), which are executed by the CPU 60A. The RAM 60C temporarily stores various data.

[0052] The storage unit 62 stores radiographic images, ultrasound images, and various other information acquired from the image storage system 19. Specific examples of the storage unit 62 include an HDD and an SSD.

[0053] The operation unit 66 is used by the user to input instructions and various information related to image processing. The operation unit 66 is not particularly limited, and examples thereof include various switches, a touch panel, a touch pen, and a mouse. The display unit 68 displays various information. The operation unit 66 and the display unit 68 may be integrated into a touch panel display.

[0054] The I / F unit 64 communicates radiological images, ultrasound images, and various types of information with the image storage system 19 via wireless or wired communication.

[0055] 5 is a functional block diagram showing an example of the functions of the image processing device 18. The image processing device 18 includes a radiographic image acquisition unit 70, an ultrasonic image acquisition unit 71, a breast contour detection unit 73, an ultrasonic cross-sectional image generation unit 74, and a display control unit 79. As an example, in the image processing device 18 of this embodiment, the CPU 60A of the control unit 60 executes the image processing program 61, so that the CPU 60A functions as the radiographic image acquisition unit 70, the ultrasonic image acquisition unit 71, the breast contour detection unit 73, the ultrasonic cross-sectional image generation unit 74, and the display control unit 79.

[0056] The radiographic image acquisition unit 70 has a function of acquiring a radiographic image X. In this embodiment, as an example, a set of radiographic images X and ultrasound images U obtained by continuous imaging, the details of which will be described later, that satisfy the display conditions input by the user via the operation unit 66 is acquired. Therefore, upon receiving the display conditions input by the user, the radiographic image acquisition unit 70 acquires the radiographic image X from the set of radiographic images X and ultrasound images U that corresponds to the received display conditions from the image storage system 19 via the I / F unit 64. Examples of the display conditions include identification information for identifying the subject and the breast, information indicating the date and time of imaging, and identification information assigned to each set of imaging. The radiographic image acquisition unit 70 outputs the acquired radiographic image X to the breast contour detection unit 73.

[0057] The ultrasonic image acquisition unit 71 has a function of acquiring an ultrasonic image U. As described above, the ultrasonic image acquisition unit 71 of this embodiment acquires an ultrasonic image U corresponding to the display conditions used when the radiographic image acquisition unit 70 acquires a radiographic image from the image storage system 19 via the I / F unit 64. In the ultrasonic imaging device 16 of this embodiment, an ultrasonic image U of the entire breast is acquired by repeating imaging multiple times while scanning with the ultrasonic probe 55. In other words, multiple ultrasonic images U of the entire breast are obtained by continuously capturing images of the entire breast. Therefore, the ultrasonic image acquisition unit 71 acquires multiple ultrasonic images U. In addition, for simplicity of explanation, these multiple ultrasonic images U may be simply referred to as "ultrasonic image U." The ultrasonic image acquisition unit 71 outputs the acquired ultrasonic image U to the ultrasonic cross-sectional image generation unit 74.

[0058] That is, in this embodiment, of a set of radiographic image X and ultrasound image U that are captured consecutively, the radiographic image acquisition unit 70 acquires the radiographic image X, and the ultrasound image acquisition unit 71 acquires the ultrasound image U. Note that the ultrasound image U in this embodiment is provided with a probe position detection result S. The probe position detection result S is the detection result of the position detection sensor 57 of the ultrasound imaging device 16. Specifically, each ultrasound image U is provided with the probe position detection result S, which is the detection result of the position of the ultrasound probe 55 when the ultrasound image U was captured, detected by the position detection sensor 57.

[0059] The breast contour detection unit 73 detects the contour of the breast included in the radiographic image X acquired by the radiographic image acquisition unit 70. Note that the method by which the breast contour detection unit 73 detects the image of the breast included in the radiographic image X is not particularly limited. For example, the breast contour detection unit 73 may examine the density of the radiographic image X, detect positions where the density step is equal to or greater than a predetermined value, and detect a group of pixels where this density step is equal to or greater than the predetermined value as the contour of the breast. In addition, for example, the breast contour detection unit 73 may perform segmentation on the breast region included in the radiographic image X to detect an area corresponding to the image of the breast. Note that the segmentation may be performed by detecting the contour of the breast using pattern matching, or may be performed using a machine learning model such as a CNN (Convolution Neural Network).

[0060] The breast contour detection unit 73 outputs the breast contour detected from the radiographic image X and the radiographic image X to the display control unit 79. Note that the radiographic image X of this embodiment is an example of a radiographic image to be superimposed according to the present disclosure.

[0061] The ultrasound cross-sectional image generating unit 74 has a function of combining multiple ultrasound images U acquired by the ultrasound image acquiring unit 71 to generate an ultrasound cross-sectional image of a plane substantially parallel to the imaging surface 40A of the imaging table 40. Typically, radiographic images captured by the mammography device 10 are images of a plane parallel to the imaging surface 40A of the imaging table 40. In this way, to facilitate comparison with radiographic images of a plane parallel to the imaging surface 40A of the imaging table 40, the ultrasound cross-sectional image generating unit 74 generates an ultrasound cross-sectional image of a plane parallel to the imaging surface 40A of the imaging table 40. In this embodiment, the term "substantially parallel" in the context of "a plane substantially parallel to the imaging surface 40A of the imaging table 40" refers to a plane that can be considered "parallel" if the bending and tilt of the compression member 34, the tilt of the ultrasound probe 55, etc., can be ignored. In this embodiment, the case where the bending and tilt of the compression member 34 and the tilt of the ultrasonic probe 55 are small enough to be ignored will be described, and therefore, hereinafter, the term "parallel" may be used. Note that, if the bending and tilt of the compression member 34 and the tilt of the ultrasonic probe 55 are large enough to be unignorable, these may be corrected to make the surface "substantially parallel to the imaging surface 40A of the imaging table 40."

[0062] When the upper surface 34A of the compression member 34 is parallel to the imaging surface 40A of the imaging table 40 and the inclination of the ultrasonic probe 55 during imaging is ignored, the ultrasonic cross-sectional image synthesized from the multiple ultrasonic images U becomes a so-called C-mode image, which is an image of the C-plane of the breast. Note that a C-mode image in ultrasonic imaging is an image of a cross-sectional plane perpendicular to the ultrasonic beam at a specific fixed diagnostic distance. Note that the method by which the ultrasonic cross-sectional image generator 74 generates the ultrasonic cross-sectional image is not limited, and for example, a known technique can be used to reconstruct a cross-sectional image of a plane parallel to the imaging surface 40A of the imaging table 40 by reconstructing the multiple ultrasonic images U.

[0063] In this embodiment, the ultrasonic cross-sectional image is described as a two-dimensional cross-sectional image that is a plane parallel to the imaging surface 40A of the imaging table 40, but the ultrasonic cross-sectional image is not limited to a two-dimensional image. For example, the ultrasonic cross-sectional image may be a three-dimensional image that has a cross section at any depth and has a thickness in the depth direction.

[0064] The ultrasound cross-sectional image generating unit 74 of this embodiment generates ultrasound cross-sectional images corresponding to a plurality of cross sections at any depth. That is, the ultrasound cross-sectional image generating unit 74 of this embodiment generates a plurality of ultrasound cross-sectional images. The ultrasound cross-sectional image generating unit 74 outputs the generated ultrasound cross-sectional images to the display control unit 79. Note that in this embodiment, the ultrasound cross-sectional images generated by the ultrasound cross-sectional image generating unit 74 are an example of an ultrasound image for display according to the present disclosure.

[0065] The display control unit 79 superimposes the breast contour detected by the breast contour detection unit 73 on the ultrasound cross-sectional image generated by the ultrasound cross-sectional image generation unit 74. The display control unit 79 displays the ultrasound cross-sectional image with the breast contour superimposed on it on the display unit 68. The display control unit 79 of this embodiment also displays the radiation image X on the display unit 68.

[0066] Next, the operation of the image processing device 18 of this embodiment will be described with reference to the drawings, but first, a description will be given of the flow of capturing a radiographic image X and an ultrasound image U using the medical image capturing system 1. Fig. 6 shows a flowchart illustrating an example of the flow of capturing a radiographic image X and an ultrasound image U using the medical image capturing system 1 of this embodiment.

[0067] First, the user positions the subject's breast, which will be the subject of the imaging, on the imaging surface 40A of the imaging table 40. Once positioning is complete, the user issues an instruction to compress the breast using the operation unit 26. Then, in step S10 of Figure 6, the control unit 20 of the mammography device 10 begins compressing the breast with the compression members 34. Specifically, upon receiving the instruction to compress the breast, the control unit 20 moves the compression members 34 in the compression direction, causing the breast to be compressed between the compression members 34 and the imaging surface 40A of the imaging table 40.

[0068] In the next step S12, a radiographic image X of the breast is captured by the mammography device 10. Specifically, the user operates an irradiation switch included in the operation unit 26 to cause the radiation source 36R to irradiate the breast with radiation R, and the radiation detector 30 captures the radiographic image X. The radiographic image X captured by the mammography device 10 is output to the console 12, and is then output from the console 12 to the image storage system 19 at a predetermined timing and stored in the image storage system 19.

[0069] In the next step S14, the technician scans the ultrasound probe 55 of the ultrasound imaging device 16 to capture multiple ultrasound images U of the breast compressed by the compression members 34. Specifically, after capturing the radiation image X, the user applies an acoustic matching material (not shown) such as echo jelly to the upper surface 34A of the compression members 34. The user then scans the upper surface 34A of the compression members 34, covered with the acoustic matching material, with ultrasound from the ultrasound probe 55 to capture multiple ultrasound images U.

[0070] The ultrasound image captured by the ultrasound imaging device 16 is displayed on the display unit 58. The ultrasound image U captured by the ultrasound imaging device 16 is temporarily stored in the storage unit 52, and then output from the ultrasound imaging device 16 to the image storage system 19 at a predetermined timing and stored in the image storage system 19.

[0071] After capturing the ultrasound image U, the compression of the breast by the compression members 34 is released in the next step S16. Specifically, the user issues an instruction to release the compression via the operation unit 26. Upon receiving the instruction to release the compression of the breast, the control unit 20 moves the compression members 34 in the compression release direction, moving the compression members 34 away from the imaging surface 40A of the imaging table 40, thereby releasing the compression of the breast by the compression members 34. When the processing of step S16 is completed in this manner, the continuous capture of the radiographic image X and the ultrasound image U is completed. In this embodiment, the continuous capture of the radiographic image X and the ultrasound image U while the breast W remains compressed by the compression members 34, as shown in FIG. 6 , is referred to as "continuous capture." The order of capturing the radiographic image X (step S12 in FIG. 6 ) and the ultrasound image U (step S14 in FIG. 6 ) is not limited. However, from the viewpoint of shortening the breast compression time, it is preferable to capture the radiographic image X first, as in this embodiment. Furthermore, the compression force applied to the breast by the compression member 34 may be reduced to the extent that it is considered that there is no change in the compression state of the breast between capturing the radiological image X and capturing the ultrasound image U. For example, the compression force applied by the compression member 34 may be reduced to the extent that it is considered that there is no change in the state of expansion of the mammary glands of the breast.

[0072] When the continuous capturing of the radiographic image X and the ultrasonic image U is completed in this manner, image processing is performed by the image processing device 18. As an example, when the image processing device 18 of this embodiment receives the display conditions for the radiographic image X and the ultrasonic image U to be displayed, input by the user via the operation unit 66 as described above, the CPU 60A of the control unit 60 executes the image processing program 61 stored in the ROM 60B, thereby performing the image processing, an example of which is shown in Fig. 7. Fig. 7 shows a flowchart illustrating an example of the flow of image processing in the image processing device 18 of this embodiment.

[0073] First, in step S100, the radiographic image acquisition unit 70 acquires a radiographic image X that satisfies the display conditions from the image storage system 19, as described above, and outputs it to the breast contour detection unit 73. In the next step S102, the ultrasound image acquisition unit 71 acquires a plurality of ultrasound images U that satisfy the display conditions from the image storage system 19, as described above, and outputs them to the ultrasound cross-sectional image generation unit 74. Through the processing of steps S100 and S102, the image processing device 18 acquires a set of a radiographic image X and an ultrasound image U obtained by continuous imaging.

[0074] In the next step S104, as described above, the ultrasound cross-sectional image generating unit 74 generates a plurality of ultrasound cross-sectional images corresponding to cross sections at any depth from the plurality of ultrasound images U acquired by the ultrasound image acquiring unit 71. The ultrasound cross-sectional image generating unit 74 outputs the generated ultrasound cross-sectional images to the display control unit 79.

[0075] In the next step S106, the breast contour detection unit 73 detects the contour of the breast included in the radiation image X as described above, and outputs the detection result and the radiation image X to the display control unit 79.

[0076] In the next step S108, the display control unit 79, as described above, causes the display unit 68 to display an ultrasound cross-sectional image on which the breast contour included in the radiation image X is superimposed. Note that here, the ultrasound cross-sectional image on which the breast contour is superimposed, i.e., the ultrasound cross-sectional image to be displayed on the display unit 68, is at least one of the plurality of ultrasound cross-sectional images generated by the ultrasound cross-sectional image generation unit 74. For example, the display control unit 79 receives a designation of the breast depth from the radiologist, selects an ultrasound cross-sectional image corresponding to the received breast depth from the plurality of ultrasound cross-sectional images generated by the ultrasound cross-sectional image generation unit 74, and displays the selected ultrasound cross-sectional image on the display unit 68 with the breast contour included in the radiation image X superimposed. Also, for example, the display control unit 79 causes the display unit 68 to display the ultrasound cross-sectional image containing the largest breast image area among the plurality of ultrasound cross-sectional images generated by the ultrasound cross-sectional image generation unit 74 with the breast contour included in the radiation image X superimposed.

[0077] In the next step S110, the display control unit 79 also displays the radiation image X corresponding to the contour of the breast superimposed on the ultrasound cross-sectional image on the display unit 68. That is, the display unit 68 also displays the radiation image X acquired in step S100. When the processing of step S110 ends, the image processing shown in FIG. 7 ends.

[0078] Fig. 8 shows an example of the display state of a radiographic image X and an ultrasound cross-sectional image D on which a breast contour C included in the radiographic image X is superimposed, which are displayed by the display control unit 79 on the display unit 68. In Fig. 8, the breast contour C included in the radiographic image X is larger than the contour of the breast image WG_d included in the ultrasound cross-sectional image D. The reason why the breast contour C included in the radiographic image X is larger than the contour of the breast image WG_d included in the ultrasound cross-sectional image D will be described with reference to Figs. 9 to 11.

[0079] FIG. 9 shows the state of the breast W compressed between the compression member 34 and the imaging table 40. As shown in FIG. 9, near the nipple, which is the tip of the breast, a space may be created between the underside 34B of the compression member 34 and the breast W, resulting in a region where the compression member 34 and the breast W are not in contact. In a region 90 where the compression member 34 and the breast W are not in contact, air is present between the ultrasound probe 55 scanning the compression member 34 and the breast W. As a result, the breast image in the region corresponding to region 90 is missing in the ultrasound image U. Specifically, as shown in FIG. 10, an image of a missing region 91 near the skin line of the breast is missing in the ultrasound image U.

[0080] 11 , in the ultrasonic probe 55, the housing 55h protrudes outward more than the ultrasonic transducer 55t. Therefore, the housing 55h may get in the way, preventing the ultrasonic transducer 55t from reaching the vicinity of the chest wall K. In such a case, an image of a missing region 92 near the chest wall K is missing in the ultrasonic image U. Specifically, as shown in FIGS. 9 and 10 , an image of a missing region 91 on the chest wall K side is missing in the ultrasonic image U. As a result, the image of the missing regions 91 and 92 is missing, and therefore the breast image WG_d in the ultrasonic cross-sectional image D is smaller than the breast image WG_x in the radiographic image X. Therefore, as shown in FIG. 8 , the breast contour C in the radiographic image X is larger than the contour of the breast image WG_d in the ultrasonic cross-sectional image D.

[0081] 8, the image processing device 18 of this embodiment displays the breast contour C included in the radiographic image X on the display unit 68 in a state where it is superimposed on the ultrasound cross-sectional image D. This allows the image reader to interpret the image while taking into consideration areas in the ultrasound cross-sectional image D that are missing images, so-called blind areas.

[0082] The present embodiment may be modified as follows: (Modification 1) In this modification, a case will be described in which the radiographic images X acquired by the radiographic image acquisition unit 70 are a plurality of radiographic images obtained by so-called tomosynthesis imaging.

[0083] The mammography apparatus 10 shown in FIG. 2 is capable of so-called tomosynthesis imaging. Tomosynthesis imaging is an imaging method in which radiation is irradiated onto a subject (a breast in this embodiment) at different irradiation angles, and a radiographic image X (so-called projection image) is captured for each irradiation angle. When performing tomosynthesis imaging, the arm 42 rotates to move the radiation source 36R of the radiation irradiator 36 to each of a plurality of irradiation positions with different irradiation angles (projection angles). For example, the radiation source 36R is moved to irradiation positions with different irradiation angles by a predetermined angle. In other words, the radiation source 36R is moved to positions where the angle of incidence of radiation R with respect to the detection surface of the radiation detector 30 is different. At each irradiation position, radiation R is irradiated from the radiation source 36R in response to instructions from the console 12, and a radiographic image X is captured by the radiation detector 30. In this way, tomosynthesis imaging obtains a plurality of radiographic images X corresponding to the number of irradiation positions. In addition, the mammography device capable of tomosynthesis imaging may be different from the mammography device 10 shown in Figure 2, and may be, for example, a mammography device in which a radiation source 36R is provided for each irradiation position, and the entire device is equipped with multiple radiation sources 36R.

[0084] As shown in FIG. 12, the image processing device 18 of this modified example further includes a radiation tomographic image generating unit 72 in addition to the components of the image processing device 18 of the embodiment (see FIG. 5).

[0085] The radiation image acquisition unit 70 of this modified example acquires a plurality of radiation images X obtained by tomosynthesis imaging, and outputs them to the radiation tomographic image generation unit 72 .

[0086] The radiological tomographic image generating unit 72 generates, from the radiological images X, a plurality of radiological tomographic images corresponding to a plurality of cross sections at different heights from the imaging surface 40A of the imaging table 40. The number of radiological tomographic images generated by the radiological tomographic image generating unit 72 may be the same as the number of ultrasonic cross sections generated by the ultrasonic cross section image generating unit 74. As described above, in an ultrasonic image, "depth" refers to the depth of the breast on the imaging table 40 from the surface that contacts the compression member 34 (see the lower surface 34B in FIG. 9 ). On the other hand, in a radiological tomographic image, the imaging surface 40A of the imaging table 40 is used as a reference to identify the position of the corresponding cross section in the height direction from the imaging surface 40A. In this case, the radiological tomographic image generating unit 72 converts the height in the radiological tomographic image into the depth in the ultrasonic image (ultrasonic cross section image). The farther away from the imaging table 40, the closer to the compression member 34. Therefore, the higher the height from the imaging surface 40A of the imaging table 40, the shallower the depth of the breast from the surface that contacts the compression member 34.

[0087] The method by which the radiation tomographic image generating unit 72 generates a radiation tomographic image from the plurality of radiation images X is not particularly limited, and any known method can be used. The radiation tomographic image generating unit 72 outputs the generated plurality of radiation tomographic images to the breast contour detecting unit 73.

[0088] The breast contour detection unit 73 selects a radiation tomographic image at a height corresponding to the ultrasound cross-sectional image to be displayed from among the multiple radiation tomographic images generated by the radiation tomographic image generation unit 72, detects the contour of the breast contained in the selected radiation tomographic image, and outputs the detection result and the selected radiation tomographic image to the display control unit 79.

[0089] In this modification, as shown in Fig. 13 , the display controller 79 displays the ultrasound cross-sectional image D on the display unit 68 with a breast contour C superimposed thereon, the breast contour C being included in the radiation tomographic image M at a depth corresponding to the ultrasound cross-sectional image D. The breast contour C superimposed on the ultrasound cross-sectional image D displayed on the display unit 68 is the same as the outline of the breast image WG_m included in the radiation tomographic image M. Note that, although the example shown in Fig. 13 describes a mode in which one ultrasound cross-sectional image D is displayed, a mode in which multiple ultrasound cross-sectional images D at different depths are displayed on the display unit 68 may also be used. In this case, the display controller 79 displays each ultrasound cross-sectional image D on the display unit 68 with a breast contour C being superimposed thereon, the breast contour C being included in the radiation tomographic image M at a height corresponding to the respective depth.

[0090] 14 , the display control unit 79 may further cause the display unit 68 to display a schematic diagram S to which information 95 and 96 indicating the difference between the breast region included in the ultrasound cross-sectional image D to be displayed on the display unit 68 and the breast region included in the radiation image X has been added. The information 95 corresponds to an image-missing region 91 in the skin line portion of the breast. The information 96 corresponds to an image-missing region 92 in the portion on the chest wall side.

[0091] Furthermore, the display control unit 79 may also display numerical information 99 that indicates the distance between the contour of the breast included in the ultrasound cross-sectional image D and the contour of the breast included in the radiation image X, as shown in FIG.

[0092] This modified example will be specifically described using an example in which the display control unit 79 displays a radiographic image X and an ultrasound cross-sectional image D with the contour C of the breast contained in the radiographic image X superimposed thereon, as shown in Figure 8.

[0093] First, a method for deriving the information 95 and the numerical information 99 related to the information 95 will be described. The breast contour detection unit 73 of this modified example also extracts the contour of the breast contained in the ultrasound cross-sectional image D. The breast contour detection unit 73 then derives the difference between the contour of the breast contained in the radiation image X and the contour of the breast contained in the ultrasound cross-sectional image D as the difference between the breast region contained in the ultrasound cross-sectional image D and the breast region contained in the radiation image X. The breast contour detection unit 73 outputs the difference in the skin line portion of the breast as information 95 and numerical information 99 ("2.0 cm" in FIG. 14 ) to the display control unit 79.

[0094] Next, a method for deriving the information 96 and the numerical information 99 related to the information 96 will be described. As described above, the image missing area 92 on the chest wall side occurs depending on the housing 55h of the ultrasound probe 55. Therefore, the display control unit 79 acquires information about the ultrasound probe 55 used to capture the ultrasound image U, specifically, the distance between the outline of the housing 55h of the ultrasound probe 55 and the position of the ultrasound transducer 55t. The acquired information about the ultrasound probe 55 is then output to the display control unit 79 as information 95 and numerical information 99 ("1.5 cm" in FIG. 14).

[0095] As shown in Figure 14, the display control unit 79 generates a breast schematic diagram S to which the information 95 and its numerical information 99 input from the breast contour detection unit 73 have been added, and displays it on the display control unit 79.

[0096] In this way, by displaying the schematic diagram S to which the information 95, 96 and the numerical information 99 have been added, the image reader can more easily consider the missing image area in the ultrasound image U.

[0097] (Variation 3) Furthermore, the ultrasonic cross-sectional image that the display control unit 79 causes the display unit 68 to display is not limited to the so-called C-mode image described above. It may be any of the ultrasonic images U obtained by imaging, and may be an ultrasonic cross-sectional image corresponding to so-called A-plane, which is a first cross-sectional ultrasonic image that is a cross-sectional image in a direction intersecting the imaging table 40 on which the breast is placed. It may also be an ultrasonic cross-sectional image corresponding to so-called B-plane, which is a second cross-sectional ultrasonic image that is synthesized from multiple ultrasonic images U and is a cross-sectional image in a direction intersecting the imaging table 40 and the ultrasonic images U. The type of ultrasonic cross-sectional image to be displayed may be any one of these three types, or multiple types.

[0098] 15 illustrates an example in which, in a configuration in which the display unit 68 displays the radiographic image X shown in FIG. 8 and the ultrasound cross-sectional image D in a state in which the contour C of the breast included in the radiographic image X is superimposed, an ultrasound cross-sectional image D_b of plane B of the ultrasound cross-sectional image D and an ultrasound cross-sectional image D_a of plane A of the ultrasound cross-sectional image D are further displayed. In the example shown in FIG. 15, the image reader can specify the positions of the ultrasound cross-sectional images D_b and D_a to be displayed by moving a guide line L provided on the ultrasound cross-sectional image D.

[0099] In this way, by superimposing the breast contour C on the ultrasound cross-sectional images D_b and D_a as well, it becomes easier for the image reader to take into consideration the missing image area in the ultrasound image U.

[0100] As described above, in the image processing device 18 of the above embodiment and each modified example, the radiographic image acquisition unit 70 acquires a radiographic image X captured while the breast is compressed by the compression member 34. The ultrasound image acquisition unit 71 acquires multiple ultrasound images U of the breast in a compressed state that can be considered to be the same as the radiographic image X.

[0101] The display control unit 79 superimposes, on the radiation image X, the contour of the breast contained in the ultrasound cross-sectional image that is synthesized from multiple ultrasound images and is parallel to the imaging table 40. Furthermore, the display control unit 79 displays the radiation image X with the contour of the breast superimposed.

[0102] Therefore, according to the image processing device 18 of the above embodiment and each of the modifications, the image interpreter can intuitively grasp the correspondence between the radiological image and the ultrasonic cross-sectional image.

[0103] When displaying the radiological image X in which the breast contour has been detected on the display unit 68, the display control unit 79 may superimpose the breast contour included in the ultrasound cross-sectional image D. In this case, the display control unit 79 may superimpose the breast contour included in the ultrasound cross-sectional image D that has the largest breast area among the multiple ultrasound cross-sectional images D on the radiological image X. Furthermore, when displaying multiple radiological sectional images M in a state in which the breast contour included in the ultrasound cross-sectional image D is superimposed, as in Modification Example 1, the display control unit 79 may display the breast contour included in the ultrasound cross-sectional image D at a depth corresponding to the height of each ultrasound tomographic image in a state in which it is superimposed.

[0104] In the above embodiment and each modification, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the radiographic image acquisition unit 70, the ultrasonic image acquisition unit 71, the radiographic tomographic image generation unit 72, the breast contour detection unit 73, the ultrasonic cross-sectional image generation unit 74, and the display control unit 79. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0105] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).Furthermore, multiple processing units may be configured with a single processor.

[0106] Examples of configuring multiple processing units with a single processor include: first, a configuration in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); and thus, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0107] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0108] In the above embodiment, the image processing program 61 is pre-stored (installed) in the control unit 60 of the image processing device 18, but this is not limiting. The image processing program 61 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 61 may also be downloaded from an external device via a network.

[0109] The present disclosure can also be applied to programs and program products. Specifically, the image processing program 61 in each of the above embodiments may be provided as a program product. The program product includes any type of product for providing a program. For example, the program product includes a program provided via a network such as the Internet, and a non-transitory computer-readable recording medium such as a CD-ROM or DVD on which the program is stored.

[0110] Furthermore, the configurations and operations of the medical image capturing system 1, the radiographic image capturing system 2, the mammography device 10, the ultrasound image capturing device 16, and the image processing device 18 described in the above embodiment and each modification are merely examples, and it goes without saying that they can be modified according to the circumstances without departing from the spirit of the present invention. It also goes without saying that the above embodiments can be combined as appropriate.

[0111] The following supplementary note is further disclosed regarding the above-described embodiment: (Supplementary Note 1) An image processing device including a processor, which acquires a radiographic image captured with the breast compressed by a compression member, acquires a plurality of ultrasound images captured with the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image, and superimposes a contour of the breast included in a radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

[0112] (Supplementary Note 2) The image processing device described in Supplementary Note 1, wherein the processor accepts a specification of the depth of the breast, and the ultrasound image for display is a cross-sectional image that is synthesized from the multiple ultrasound images and is approximately parallel to the imaging table according to the accepted depth of the breast.

[0113] (Supplementary Note 3) The image processing device described in Supplementary Note 1 or Supplementary Note 2, wherein the ultrasound image for display is any one of the plurality of ultrasound images and is at least one of a first cross-sectional ultrasound image which is a cross-sectional image in a direction intersecting with an imaging table on which the breast is placed, a second cross-sectional ultrasound image which is synthesized from the plurality of ultrasound images and is a cross-sectional image in a direction intersecting with the imaging table and the first cross-sectional ultrasound image, and a third cross-sectional ultrasound image which is synthesized from the plurality of ultrasound images and is a cross-sectional image approximately parallel to the imaging table.

[0114] (Supplementary Note 4) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 3, wherein the radiological image is a projection image obtained by tomosynthesis imaging, and the radiological image to be superimposed is a radiological tomographic image at a height corresponding to the ultrasound image to be displayed, among a plurality of radiological tomographic images reconstructed from the projection images.

[0115] (Supplementary Note 5) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 4, wherein the processor also displays the radiation image for superimposition including the contour of the breast.

[0116] (Supplementary Note 6) The image processing device according to Supplementary Note 5, wherein the processor displays the radiation image for superimposition with the contour of the breast included in the ultrasound image for display superimposed on it.

[0117] (Supplementary Note 7) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 6, wherein the processor further displays a schematic diagram to which information indicating a difference area between the area of ​​the breast included in the ultrasound image for display and the area of ​​the breast included in the radiation image for superimposition is added.

[0118] (Supplementary Note 8) The image processing device according to Supplementary Note 7, wherein the information includes a numerical value representing a distance between the contour of the breast included in the ultrasound image for display and the contour of the breast included in the radiation image for superimposition.

[0119] (Supplementary Note 9) A medical image capturing system including: the image processing device according to any one of Supplementary Notes 1 to 8; a radiographic image capturing device; and an ultrasonic image capturing device.

[0120] (Supplementary Note 10) An image processing method in which a processor included in an image processing device acquires a radiographic image captured with the breast compressed by a compression member, acquires a plurality of ultrasound images captured with the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image, and superimposes an outline of the breast included in a radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

[0121] (Supplementary Note 11) An image processing program for causing a processor included in an image processing device to execute the following processes: acquire a radiographic image captured with the breast compressed by a compression member; acquire a plurality of ultrasound images captured with the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image; and superimpose an outline of the breast included in a radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

[0122] The disclosure of Japanese Patent Application No. 2023-170679, filed on September 29, 2023, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards mentioned herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An image processing device comprising: a processor which acquires a radiological image of the breast compressed by a compression member, acquires a plurality of ultrasound images of the breast compressed by a compression member in a compression state that can be regarded as identical to the radiological image, and superimposes a contour of the breast contained in a radiological image for superimposition corresponding to the radiological image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

2. The image processing device according to claim 1, wherein the processor receives a specification of a depth of the breast, and the ultrasound image for display is a cross-sectional image that is synthesized from the multiple ultrasound images and is approximately parallel to the imaging table according to the received depth of the breast.

3. The image processing device of claim 1, wherein the ultrasound image for display is any one of the plurality of ultrasound images and is at least one of a first cross-sectional ultrasound image which is a cross-sectional image in a direction intersecting with an imaging table on which the breast is placed, a second cross-sectional ultrasound image which is synthesized from the plurality of ultrasound images and is a cross-sectional image in a direction intersecting with the imaging table and the first cross-sectional ultrasound image, and a third cross-sectional ultrasound image which is synthesized from the plurality of ultrasound images and is a cross-sectional image approximately parallel to the imaging table.

4. The image processing device according to claim 1, wherein the radiation image is a projection image obtained by tomosynthesis imaging, and the radiation image for superimposition is, among a plurality of radiation tomographic images reconstructed from the projection images, the radiation tomographic image at a height corresponding to the ultrasound image for display.

5. The image processing device according to claim 1, wherein the processor also displays the radiation image for superimposition, the radiation image including the contour of the breast.

6. The image processing device according to claim 5, wherein the processor displays the contour of the breast contained in the ultrasound image for display in a state where it is superimposed on the radiation image for superimposition.

7. The image processing device according to claim 1, wherein the processor further displays a schematic diagram to which is added information representing an area that is the difference between the area of ​​the breast included in the ultrasound image for display and the area of ​​the breast included in the radiation image for superimposition.

8. The image processing device according to claim 7, wherein the information includes a numerical value representing a distance between the contour of the breast included in the ultrasound image for display and the contour of the breast included in the radiation image for superimposition.

9. A medical image capturing system comprising: an image processing device according to any one of claims 1 to 8; a radiation image capturing device; and an ultrasonic image capturing device.

10. An image processing method in which a processor provided in an image processing device acquires a radiographic image taken with the breast compressed by a compression member, acquires a plurality of ultrasound images taken with the breast compressed by a compression member in a compression state that can be regarded as identical to the radiographic image, and superimposes the contour of the breast contained in a radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

11. An image processing program for causing a processor included in an image processing device to execute the following processes: acquire a radiographic image captured with the breast compressed by a compression member; acquire a plurality of ultrasound images captured with the breast compressed by a compression member in a compression state that can be regarded as identical to the radiographic image; and superimpose the contour of the breast contained in a radiographic image for superimposition corresponding to the radiographic image onto an ultrasound image for display corresponding to the plurality of ultrasound images.

Images