Image processing device, medical image capturing system, image processing method, and image processing program

Abstract

Provided is an image processing device comprising a processor. The processor: acquires a radiographic image captured while a breast is kept in a state of being compressed by a compression member; acquires a plurality of ultrasound images captured while the breast is kept in a state of being compressed by the compression member that can be considered the same compressed state as in the radiographic image; and superimposes, on a radiographic image for display corresponding to said radiographic image, a contour of the breast included in an ultrasound cross-sectional image that has been synthesized from the plurality of ultrasound images and is substantially parallel to an imaging table.

Description

Image processing device, medical image capturing system, image processing method, and image processing program

[0001] The present disclosure relates to an image processing device, a medical image capturing system, an image processing method, and an image processing program.

[0002] When observing a radiographic image of a breast, an ultrasound image of the breast is also referred to. For example, Japanese Patent Application Laid-Open No. 2005-125080 describes a method for observing a region of interest in a radiographic image and a region of interest in an ultrasound image.

[0003] However, even when radiographic images and ultrasound images are taken consecutively while the breast is compressed by a compression member, and the compression state is considered to be the same, the appearance of the breast in each image differs. For example, radiographic images are easy to detect calcifications, but become difficult to see when the breast density is high. On the other hand, ultrasound images are easy to detect tumors, and even when the breast density is high, they are not difficult to see like radiographic images. Because the appearance of radiographic images and ultrasound images differs, there is a problem that it may be difficult for the radiologist to understand the correspondence between the radiographic images and ultrasound images.

[0004] The present disclosure provides an image processing device, a medical image capturing system, an image processing method, and an image processing program that allow an image reader to intuitively grasp the correspondence between a radiological image and an ultrasonic cross-sectional image.

[0005] An image processing device of a first aspect of the present disclosure includes a processor that acquires a radiological image of the breast compressed by a compression member, acquires a plurality of ultrasound images of the breast compressed by a compression member in a state that can be considered identical to the radiological image, and superimposes the contour of the breast, which is synthesized from the plurality of ultrasound images and contained in an ultrasound cross-sectional image that is approximately parallel to the imaging table, onto a radiological image for display that corresponds to the radiological image.

[0006] An image processing device of a second aspect of the present disclosure is the image processing device of the first aspect, in which the processor accepts a specification of the breast depth and displays the breast contour contained in the ultrasound cross-sectional image corresponding to the accepted breast depth superimposed on the radiological image for display.

[0007] An image processing device of a third aspect of the present disclosure is the image processing device of the first aspect, in which a processor determines the contour of the breast from each of a plurality of ultrasound images based on image features indicating whether the area is one in which ultrasound echoes from the breast have been received.

[0008] An image processing device of a fourth aspect of the present disclosure is the image processing device of the first aspect, in which a processor displays the contour of the breast contained in a composite ultrasound image obtained by combining multiple ultrasound cross-sectional images of the breast at different depths in the depth direction, superimposed on a radiological image for display.

[0009] An image processing device of a fifth aspect of the present disclosure is the image processing device of the fourth aspect, in which a processor generates a composite ultrasound image by adding pixel values ​​in the depth direction for the breast region of the ultrasound cross-sectional image, and displays an ultrasound image averaged by the number of ultrasound cross-sectional images with pixel values ​​added in the depth direction at each position of the breast region as an ultrasound image for display.

[0010] An image processing device of a sixth aspect of the present disclosure is the image processing device of the fifth aspect, wherein the processor excludes an area from the breast area or breast contour if the number of added ultrasound cross-sectional images at each position is less than or equal to a threshold.

[0011] An image processing device of a seventh aspect of the present disclosure is the image processing device of the first aspect, in which the radiological image is a radiological image obtained by tomosynthesis imaging, the radiological image for display is each of a plurality of radiological tomographic images reconstructed from the radiological image, and the processor displays each of the plurality of radiological tomographic images with the contour of the breast included in the ultrasound cross-sectional image corresponding to the height of the radiological tomographic image superimposed thereon.

[0012] An image processing device according to an eighth aspect of the present disclosure is the image processing device according to the first aspect, wherein the processor also displays an ultrasound cross-sectional image including the contour of the breast.

[0013] An image processing device of a ninth aspect of the present disclosure is an image processing device of the first aspect, in which the processor also displays an ultrasound cross-sectional image including a breast contour with the breast contour included in the radiation image for display superimposed thereon.

[0014] A medical imaging system according to a tenth aspect of the present disclosure includes the image processing device of the present disclosure, a radiographic imaging device, and an ultrasonic imaging device.

[0015] In addition, an image processing method of an eleventh aspect of the present disclosure includes a processor provided in an image processing device, which acquires a radiographic image of the breast compressed by a compression member, acquires multiple ultrasound images of the breast compressed by a compression member in a state that can be considered identical to the radiographic image, and superimposes the contour of the breast contained in an ultrasound cross-sectional image that is synthesized from the multiple ultrasound images and is approximately parallel to the imaging table onto a radiographic image for display that corresponds to the radiographic image.

[0016] In addition, the image processing program of a twelfth aspect of the present disclosure causes a processor provided in an image processing device to acquire a radiographic image captured with the breast compressed by a compression member, acquire multiple ultrasound images captured with the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image, and superimpose the contour of the breast contained in an ultrasound cross-sectional image that is synthesized from the multiple ultrasound images and is approximately parallel to the imaging table onto a radiographic image for display that corresponds to the radiographic image.

[0017] According to the present disclosure, an image reader can intuitively grasp the correspondence between a radiological image and an ultrasound cross-sectional image.

[0018] 1 is a configuration diagram schematically illustrating an example of the overall configuration of an imaging system according to an embodiment; FIG. 2 is a side view illustrating an example of the appearance of a mammography apparatus according to an embodiment; FIG. 3 is a block diagram illustrating an example of the configuration of an ultrasound imaging apparatus according to an embodiment; FIG. 4 is a block diagram illustrating an example of the configuration of an image processing device according to an embodiment; FIG. 5 is a functional block diagram illustrating an example of the configuration of an image processing device according to an embodiment; FIG. 6 is a diagram for explaining differences in breast contours according to depth; FIG. 7 is a flowchart illustrating an example of the flow of capturing radiographic images and ultrasound images using an imaging system according to an embodiment; FIG. 8 is a flowchart illustrating an example of the flow of image processing in an image processing device according to an embodiment; FIG. 9 is a diagram illustrating an example of the display state of a radiographic image with a breast contour superimposed thereon and an ultrasound cross-sectional image according to an embodiment; FIG. 10 is a functional block diagram illustrating an example of the configuration of an image processing device according to a first modification; FIG. 11 is a diagram illustrating an example of the display state of a radiographic image with a breast contour superimposed thereon and a composite ultrasound image according to the first modification;

[0019] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the embodiment.

[0020] First, an example of the overall configuration of the medical image capturing system of this embodiment will be described. Fig. 1 shows a configuration diagram illustrating an example of the overall configuration of a medical image capturing system 1 of this embodiment.

[0021] As shown in FIG. 1, the medical image capturing system 1 of this embodiment includes a radiation image capturing system 2, an ultrasonic image capturing device 16, an image processing device 18, and an image storage system 19.

[0022] First, a description will be given of the configuration of the radiation image capturing system 2. The radiation image capturing system 2 includes a mammography apparatus 10 and a console 12.

[0023] The mammography device 10 of this embodiment is a device that takes a radiographic image of a subject's breast by irradiating the breast with radiation R (e.g., X-rays). Note that the mammography device 10 may be a device that takes images of the subject's breast not only when the subject is standing (standing position), but also when the subject is sitting in a chair (including a wheelchair) or the like (seated position).

[0024] Figure 2 shows a side view of an example of the appearance of the mammography device 10 of this embodiment. Note that Figure 2 is a side view of the mammography device 10 as seen from the right side of the subject. As shown in Figure 2, the mammography device 10 includes a radiation source 36R, a radiation detector 30, an imaging table 40 disposed between the radiation source 36R and the radiation detector 30, and a compression member 34 that compresses the breast between the radiation source 36R and the imaging table 40.

[0025] The imaging stand 40 includes a control unit 20, a storage unit 22, an I / F (Interface) unit 24, an operation unit 26, and a radiation detector 30. The control unit 20 controls the overall operation of the mammography apparatus 10 in accordance with the control of the console 12. The control unit 20 includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc. (not shown). The ROM stores various programs in advance, including a program executed by the CPU for controlling radiographic imaging. The RAM temporarily stores various data.

[0026] The storage unit 22 stores image data of radiographic images and various other information. The storage unit 22 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. Note that, hereinafter, "image data of radiographic images" will be simply referred to as "radiographic images." Similarly, "image data of ultrasound images" will be simply referred to as "ultrasound images."

[0027] The I / F unit 24 communicates various types of information with the console 12 via wired or wireless communication. Specifically, the I / F unit 24 receives information related to the control of the mammography apparatus 10 from the console 12. The I / F unit 24 also transmits radiographic images to the console 12.

[0028] The operation unit 26 is a part provided on the imaging stand 40 or the like that can be operated by the user with the hands or feet, and is, for example, a switch, a button, a touch panel, etc. Furthermore, for example, the operation unit 26 may receive voice input from the user.

[0029] The radiation detector 30 is disposed inside the imaging table 40 and detects radiation R that has passed through the breast, which is the subject. When imaging is performed in the mammography device 10 of this embodiment, the subject's breast is positioned on the imaging surface 40A of the imaging table 40 by a user such as a doctor or technician. The imaging surface 40A, which comes into contact with the subject's breast, is made of, for example, carbon, from the viewpoints of radiation R transparency and strength.

[0030] The radiation detector 30 detects radiation R that has passed through the subject's breast and the imaging table 40, generates a radiographic image based on the detected radiation R, and outputs the generated radiographic image. The type of radiation detector 30 in this embodiment is not particularly limited, and may be, for example, an indirect conversion type radiation detector that converts radiation R into light and then converts the converted light into electric charges, or a direct conversion type radiation detector that directly converts radiation R into electric charges.

[0031] The radiation source 36R is provided in the radiation irradiation unit 36. As shown in FIG. 2, the radiation irradiation unit 36 ​​is provided on an arm 42 together with an imaging table 40 and a compression unit 46. As shown in FIG. 2, the mammography apparatus 10 of this embodiment includes the arm 42, a base 44, and a shaft 45. The arm 42 is held by the base 44 so as to be movable in the vertical direction (Z-axis direction). The shaft 45 connects the arm 42 to the base 44. The arm 42 is rotatable relative to the base 44, with the shaft 45 serving as the axis of rotation.

[0032] 2 , the compression member 34 is attached to the compression unit 46. The compression unit 46 and the arm 42 are rotatable independently relative to the base 44, with the shaft 45 as a rotation axis. In this embodiment, the shaft 45, the arm 42, and the compression unit 46 are each provided with a gear (not shown), and the arm 42 and the compression unit 46 are each connected to the shaft 45 by switching between an engaged state and a disengaged state of these gears. One or both of the arm 42 and the compression unit 46 connected to the shaft 45 rotates integrally with the shaft 45.

[0033] The compression member 34 in this embodiment is a plate-shaped member that moves up and down (in the Z-axis direction) by a compression plate driver (not shown) provided in the compression unit 46, and compresses the breast of the subject between the compression member 34 and the imaging table 40. As shown in Fig. 2, with regard to the direction of movement of the compression member 34, the direction in which the breast is compressed, in other words, the direction toward the imaging surface 40A, is referred to as the "compression direction," and the direction in which compression of the breast is released, in other words, the direction toward the radiation irradiation unit 36, is referred to as the "compression release direction."

[0034] The compression member 34 is preferably optically transparent to allow for easy alignment and confirmation of the compression state during breast compression, and is also formed from a material with excellent radiation R transparency. Furthermore, the compression member 34 is desirably formed from a material that easily transmits ultrasound waves transmitted from the ultrasound probe 55 (see FIG. 3 , described in detail below) of the ultrasound imaging device 16. Examples of materials that can be used for the compression member 34 include resins such as polymethylpentene, polycarbonate, acrylic, and polyethylene terephthalate. Polymethylpentene is particularly suitable for the compression member 34 because it has low rigidity, excellent stretchability and flexibility, and suitable values ​​for acoustic impedance, which affects the reflectivity of ultrasound, and attenuation coefficient, which affects the attenuation of ultrasound. The material constituting the compression member 34 is not limited to that described in this embodiment. For example, the compression member 34 may be a film-like material.

[0035] The compression member 34 is not limited to one that compresses the entire breast, but may also be one that compresses a portion of the breast. In other words, the compression member 34 may be smaller than the breast. For example, a compression member 34 used in so-called spot imaging, which captures a radiographic image of only the area where a lesion is present, is known as such a compression member 34.

[0036] On the other hand, the console 12 of this embodiment has the function of controlling the mammography apparatus 10 using imaging orders and various information acquired from a RIS (Radiology Information System) 5 or the like via a wireless communication LAN (Local Area Network) or the like, and instructions given by the user via an operation unit 56 or the like. The console 12 of this embodiment is, for example, a server computer.

[0037] Next, the configuration of the ultrasound imaging device 16 will be described. Fig. 3 shows a block diagram illustrating an example of the configuration of the ultrasound imaging device 16. The ultrasound imaging device 16 is a device that allows a user to capture ultrasound images of a subject's breast as an object, and is a so-called cart-type ultrasound imaging device. Note that the ultrasound imaging device 16 may also be a so-called compact or handheld type ultrasound imaging device other than the cart type.

[0038] 3, the ultrasonic imaging device 16 includes a control unit 50, a storage unit 52, an I / F unit 54, an ultrasonic probe 55, an operation unit 56, a position detection sensor 57, and a display unit 58. The control unit 50, the storage unit 52, the I / F unit 54, the ultrasonic probe 55, the operation unit 56, the position detection sensor 57, and the display unit 58 are connected via a bus 59 such as a system bus or a control bus so as to be able to exchange various information with each other.

[0039] The control unit 50 of this embodiment controls the overall operation of the ultrasound imaging device 16. The control unit 50 includes a CPU 50A, a ROM 50B, and a RAM 50C. The ROM 50B stores various programs to be executed by the CPU 50A in advance. The RAM 50C temporarily stores various data.

[0040] The captured ultrasound images and various other information are stored in the storage unit 52. Specific examples of the storage unit 52 include an HDD and an SSD.

[0041] The ultrasonic probe 55 is moved by the user along the upper surface 34A of the compression member 34 (see FIG. 2 , the surface opposite to the side that contacts the subject's breast) and scans the breast with ultrasound to obtain an ultrasound image of the breast. Specifically, when ultrasound imaging is performed, the ultrasonic probe 55 is moved by the user along the upper surface 34A of the compression member 34 with an acoustic matching member (not shown) such as echo jelly applied to the upper surface 34A of the compression member 34 or with an acoustic matching member attached to the ultrasonic probe 55.

[0042] The ultrasonic probe 55 includes a plurality of ultrasonic transducers (not shown) arranged one-dimensionally or two-dimensionally. Each ultrasonic transducer transmits ultrasonic waves based on an applied drive signal, and receives ultrasonic echoes to output a reception signal.

[0043] Each of the multiple ultrasonic transducers is composed of a vibrator made of a piezoelectric material (piezoelectric body), such as a piezoelectric ceramic, typically PZT (lead zirconate titanate), or a polymer piezoelectric element, typically PVDF (polyvinylidene difluoride), with electrodes formed on both ends. When a pulsed or continuous wave drive signal is sent to the electrodes of the vibrator to apply a voltage, the piezoelectric body expands and contracts. This expansion and contraction generates pulsed or continuous wave ultrasonic waves from each vibrator, and these ultrasonic waves are combined to form an ultrasonic beam. Furthermore, each vibrator expands and contracts upon receiving propagating ultrasonic waves, generating electrical signals. These electrical signals are output as ultrasonic reception signals and input to the main body (not shown) of the ultrasonic imaging device 16 via a cable (not shown).

[0044] The position detection sensor 57 is a sensor for detecting the position of the ultrasonic probe 55. In this embodiment, a six-axis sensor is used as the position detection sensor 57. The six-axis sensor detects the movement direction, orientation, and rotation of the ultrasonic probe 55 and further calculates the movement distance, movement speed, and the like. Specifically, a six-axis sensor is realized by combining an acceleration sensor that can detect three directions (front / back, left / right, and up / down) with a geomagnetic sensor that can detect north / south / east / west, or an acceleration sensor with a gyro sensor that can detect the speed of rotation. Note that the position detection sensor 57 is not limited to the six-axis sensor used in this embodiment. For example, a magnetic sensor such as that described in Japanese Patent Application Laid-Open No. 2011-167331 may also be used as the position detection sensor 57. Furthermore, in this embodiment, a method of using a sensor such as the position detection sensor 57 is described as a method of detecting the position of the ultrasonic probe 55, but the method of detecting the position of the ultrasonic probe 55 is not limited to the method using a sensor. For example, a marker that can detect the position in three axes in space may be attached to the ultrasonic probe 55, and the position of the ultrasonic probe 55 may be identified by analyzing an image captured of the marker. The position detection sensor 57 may be provided inside the ultrasonic probe 55 or may be provided outside the ultrasonic probe 55. Also, unlike the present embodiment, the position detection sensor 57 may be provided separately from the ultrasonic imaging device 16. In any case, as long as the position detection sensor 57 can detect the position of the ultrasonic probe 55, there are no limitations on its arrangement or configuration.

[0045] The control unit 50 identifies the current position of the ultrasound probe 55. For example, the control unit 50 identifies the position of the ultrasound probe 55 relative to the imaging surface 40A of the imaging table 40, the position of the ultrasound probe 55 relative to the upper surface 34A of the compression member 34, or the position of the ultrasound probe 55 relative to the body surface of the breast compressed by the compression member 34.

[0046] In the medical image capturing system 1 of this embodiment, it is possible to associate any position in the ultrasound image captured by the ultrasound imaging device 16 with any position pixel in the radiation image captured by the mammography device 10 based on the position of the ultrasound probe 55. Therefore, the position of the ultrasound probe 55 detected by the position detection sensor 57 is added to the captured ultrasound image and output.

[0047] The operation unit 56 is used by the user to input instructions and various information related to capturing ultrasound images, etc. The operation unit 56 is not particularly limited, and examples thereof include various switches, a touch panel, a touch pen, and a mouse. The display unit 58 displays various information and ultrasound images corresponding to the reception signals received from the ultrasound probe 55. The operation unit 56 and the display unit 58 may be integrated into one unit to form a touch panel display.

[0048] The I / F unit 54 communicates various types of information with the RIS 5 and the image storage system 19 via wireless or wired communication. The ultrasound images captured by the ultrasound imaging device 16 are transmitted to the image storage system 19 via the I / F unit 54 via wireless or wired communication.

[0049] Next, the image storage system 19 will be described. The image storage system 19 is a system that stores radiographic images captured by the radiographic imaging system 2 and ultrasound images captured by the ultrasound imaging device 16. The image storage system 19 is connected to each of the console 12 and the ultrasound imaging device 16 via wireless or wired communication. The image storage system 19 extracts images from the stored radiographic images and ultrasound images in response to requests from the console 12, the ultrasound imaging device 16, and other image reading devices (not shown), and transmits the images to the requesting devices. A specific example of the image storage system 19 is a PACS (Picture Archiving and Communication System).

[0050] Next, we will explain the image processing device 18. The image processing device 18 has a function of acquiring, from the image storage system 19, each of the radiographic images captured by the radiographic imaging system 2 and the ultrasound images captured by the ultrasound imaging device 16, and performing predetermined image processing.

[0051] Fig. 4 is a block diagram showing an example of the configuration of the image processing device 18. As shown in Fig. 4, the image processing device 18 includes a control unit 60, a storage unit 62, an I / F unit 64, an operation unit 66, and a display unit 68. The control unit 60, the storage unit 62, the I / F unit 64, the operation unit 66, and the display unit 68 are connected via a bus 69 such as a system bus or a control bus so as to be able to exchange various information with each other.

[0052] The control unit 60 of this embodiment controls the overall operation of the image processing device 18. The control unit 60 includes a CPU 60A, a ROM 60B, and a RAM 60C. The ROM 60B stores in advance various programs, including an image processing program 61 (described later), which are executed by the CPU 60A. The RAM 60C temporarily stores various data.

[0053] The storage unit 62 stores radiographic images, ultrasound images, and various other information acquired from the image storage system 19. Specific examples of the storage unit 62 include an HDD and an SSD.

[0054] The operation unit 66 is used by the user to input instructions and various information related to image processing. The operation unit 66 is not particularly limited, and examples thereof include various switches, a touch panel, a touch pen, and a mouse. The display unit 68 displays various information. The operation unit 66 and the display unit 68 may be integrated into a touch panel display.

[0055] The I / F unit 64 communicates radiological images, ultrasound images, and various types of information with the image storage system 19 via wireless or wired communication.

[0056] 5 is a functional block diagram showing an example of the functions of the image processing device 18. The image processing device 18 includes a radiographic image acquisition unit 70, an ultrasonic image acquisition unit 71, an ultrasonic cross-sectional image generation unit 74, a breast contour detection unit 76, and a display control unit 79. As an example, in the image processing device 18 of this embodiment, the CPU 60A of the control unit 60 executes the image processing program 61, so that the CPU 60A functions as the radiographic image acquisition unit 70, the ultrasonic image acquisition unit 71, the ultrasonic cross-sectional image generation unit 74, the breast contour detection unit 76, and the display control unit 79.

[0057] The radiographic image acquisition unit 70 has a function of acquiring a radiographic image X. In this embodiment, as an example, a set of radiographic images X and ultrasound images U obtained by continuous imaging, the details of which will be described later, that satisfy the display conditions input by the user via the operation unit 66 is acquired. Therefore, when the radiographic image acquisition unit 70 receives the display conditions input by the user, it acquires the radiographic image X from the set of radiographic images X and ultrasound images U that corresponds to the received display conditions from the image storage system 19 via the I / F unit 64. Examples of the display conditions include identification information for identifying the subject and the breast, information indicating the date and time of imaging, and identification information assigned to each set of imaging. The radiographic image acquisition unit 70 outputs the acquired radiographic image X to the display control unit 79.

[0058] The ultrasonic image acquisition unit 71 has a function of acquiring an ultrasonic image U. As described above, the ultrasonic image acquisition unit 71 of this embodiment acquires an ultrasonic image U corresponding to the display conditions used when the radiographic image acquisition unit 70 acquires a radiographic image from the image storage system 19 via the I / F unit 64. In the ultrasonic imaging device 16 of this embodiment, an ultrasonic image U of the entire breast is acquired by repeating imaging multiple times while scanning with the ultrasonic probe 55. In other words, multiple ultrasonic images U of the entire breast are obtained by continuously capturing images of the entire breast. Therefore, the ultrasonic image acquisition unit 71 acquires multiple ultrasonic images U. In addition, for simplicity of explanation, these multiple ultrasonic images U may be simply referred to as "ultrasonic image U." The ultrasonic image acquisition unit 71 outputs the acquired ultrasonic image U to the ultrasonic cross-sectional image generation unit 74.

[0059] That is, in this embodiment, of a set of radiographic image X and ultrasound image U that are captured consecutively, the radiographic image acquisition unit 70 acquires the radiographic image X, and the ultrasound image acquisition unit 71 acquires the ultrasound image U. Note that the ultrasound image U in this embodiment is provided with a probe position detection result S. The probe position detection result S is the detection result of the position detection sensor 57 of the ultrasound imaging device 16. Specifically, each ultrasound image U is provided with the probe position detection result S, which is the detection result of the position of the ultrasound probe 55 when the ultrasound image U was captured, detected by the position detection sensor 57.

[0060] The ultrasound cross-sectional image generating unit 74 has a function of combining multiple ultrasound images U acquired by the ultrasound image acquiring unit 71 to generate an ultrasound cross-sectional image of a plane substantially parallel to the imaging surface 40A of the imaging table 40. Typically, radiographic images captured by the mammography device 10 are images of a plane parallel to the imaging surface 40A of the imaging table 40. In this way, to facilitate comparison with radiographic images of a plane parallel to the imaging surface 40A of the imaging table 40, the ultrasound cross-sectional image generating unit 74 generates an ultrasound cross-sectional image of a plane parallel to the imaging surface 40A of the imaging table 40. In this embodiment, the term "substantially parallel" in the context of "a plane substantially parallel to the imaging surface 40A of the imaging table 40" refers to a plane that can be considered "parallel" if the bending and tilt of the compression member 34, the tilt of the ultrasound probe 55, etc., can be ignored. In this embodiment, the case where the bending and tilt of the compression member 34 and the tilt of the ultrasonic probe 55 are small enough to be ignored will be described, and therefore, hereinafter, the term "parallel" may be used. Note that, if the bending and tilt of the compression member 34 and the tilt of the ultrasonic probe 55 are large enough to be unignorable, these may be corrected to make the surface "substantially parallel to the imaging surface 40A of the imaging table 40."

[0061] When the upper surface 34A of the compression member 34 is parallel to the imaging surface 40A of the imaging table 40 and the inclination of the ultrasonic probe 55 during imaging is ignored, the ultrasonic cross-sectional image synthesized from the multiple ultrasonic images U becomes a so-called C-mode image, which is an image of the C-plane of the breast. Note that a C-mode image in ultrasonic imaging is an image of a cross-sectional plane perpendicular to the ultrasonic beam at a specific fixed diagnostic distance. Note that the method by which the ultrasonic cross-sectional image generator 74 generates the ultrasonic cross-sectional image is not limited, and for example, a known technique can be used to reconstruct a cross-sectional image of a plane parallel to the imaging surface 40A of the imaging table 40 by reconstructing the multiple ultrasonic images U.

[0062] In this embodiment, the ultrasonic cross-sectional image is described as a two-dimensional cross-sectional image that is a plane parallel to the imaging surface 40A of the imaging table 40, but the ultrasonic cross-sectional image is not limited to a two-dimensional image. For example, the ultrasonic cross-sectional image may be a three-dimensional image that has a cross section at any depth and has a thickness in the depth direction.

[0063] The ultrasound cross-sectional image generating unit 74 of this embodiment generates ultrasound cross-sectional images corresponding to a plurality of cross sections at any depth. That is, the ultrasound cross-sectional image generating unit 74 of this embodiment generates a plurality of ultrasound cross-sectional images. The ultrasound cross-sectional image generating unit 74 outputs the generated ultrasound cross-sectional images to the breast contour detecting unit 76 and the display control unit 79.

[0064] The breast contour detection unit 76 detects the contour of the breast contained in each of the multiple ultrasound cross-sectional images generated by the ultrasound cross-sectional image generation unit 74. Specifically, the breast contour detection unit 76 detects the image of the breast contained in each ultrasound cross-sectional image. The method by which the breast contour detection unit 76 detects the image of the breast contained in the ultrasound cross-sectional image is not particularly limited. For example, the breast contour detection unit 76 may examine the density of the ultrasound cross-sectional image, detect positions where the density step is equal to or greater than a predetermined value, and detect a group of pixels where this density step is equal to or greater than the predetermined value as the contour of the breast. The contour of the breast may be determined from each ultrasound cross-sectional image based on image features that indicate whether or not the area is a region where ultrasound echoes from the breast are received. For example, the breast contour detection unit 76 may segment the breast area contained in the ultrasound cross-sectional image to detect an area corresponding to the breast image. The segmentation may be performed by detecting the contour of the breast using pattern matching, or may be performed using a machine learning model such as a convolution neural network (CNN).

[0065] Note that the size of the breast included in ultrasound cross-sectional images taken at different depths may vary depending on the depth, etc. FIG. 6 shows the state of the breast W compressed between the compression member 34 and the imaging table 40. In the example shown in FIG. 6, the distances L1 to L3 from the chest wall K to the tip of the breast W vary depending on the depth of the breast W, i.e., the distance from the lower surface 34B of the compression member 34 toward the imaging surface 40A of the imaging table 40. The distance L2 from the chest wall W to the tip at a depth approximately halfway through the thickness of the breast W is longer than the distance L1 from the chest wall K to the tip of the breast W that contacts the lower surface 34B of the compression member 34, and is also longer than the distance L3 from the chest wall K to the tip of the breast W that contacts the imaging surface 40A of the imaging table 40. Therefore, the area of ​​the contour C2 of the breast W corresponding to distance L2 is larger than the area of ​​the contour C1 of the breast W corresponding to distance L1, and is also larger than the area of ​​the contour C3 of the breast W corresponding to distance L3.

[0066] The breast contour detection unit 76 outputs the breast contour detected from each ultrasonic cross-sectional image to the display control unit 79 .

[0067] The display control unit 79 superimposes the breast contour detected by the breast contour detection unit 76 on the radiographic image X acquired by the radiographic image acquisition unit 70. The display control unit 79 displays the radiographic image X with the breast contour superimposed on it on the display unit 68. In this embodiment, the radiographic image X is an example of a radiographic image for display according to the present disclosure.

[0068] Next, the operation of the image processing device 18 of this embodiment will be described with reference to the drawings, but first, a description will be given of the flow of capturing a radiographic image X and an ultrasound image U using the medical image capturing system 1. Fig. 7 shows a flowchart illustrating an example of the flow of capturing a radiographic image X and an ultrasound image U using the medical image capturing system 1 of this embodiment.

[0069] First, the user positions the subject's breast, which will be the subject of the imaging, on the imaging surface 40A of the imaging table 40. Once positioning is complete, the user issues an instruction to compress the breast using the operation unit 26. Then, in step S10 of Figure 7, the control unit 20 of the mammography device 10 begins compressing the breast with the compression members 34. Specifically, upon receiving the instruction to compress the breast, the control unit 20 moves the compression members 34 in the compression direction, causing the breast to be compressed between the compression members 34 and the imaging surface 40A of the imaging table 40.

[0070] In the next step S12, a radiographic image X of the breast is captured by the mammography device 10. Specifically, the user operates an irradiation switch included in the operation unit 26 to cause the radiation source 36R to irradiate the breast with radiation R, and the radiation detector 30 captures the radiographic image X. The radiographic image X captured by the mammography device 10 is output to the console 12, and is then output from the console 12 to the image storage system 19 at a predetermined timing and stored in the image storage system 19.

[0071] In the next step S14, the technician scans the ultrasound probe 55 of the ultrasound imaging device 16 to capture multiple ultrasound images U of the breast compressed by the compression members 34. Specifically, after capturing the radiation image X, the user applies an acoustic matching material (not shown) such as echo jelly to the upper surface 34A of the compression members 34. The user then scans the upper surface 34A of the compression members 34, covered with the acoustic matching material, with ultrasound from the ultrasound probe 55 to capture multiple ultrasound images U.

[0072] The ultrasound image captured by the ultrasound imaging device 16 is displayed on the display unit 58. The ultrasound image U captured by the ultrasound imaging device 16 is temporarily stored in the storage unit 52, and then output from the ultrasound imaging device 16 to the image storage system 19 at a predetermined timing and stored in the image storage system 19.

[0073] After capturing the ultrasound image U, the compression of the breast by the compression members 34 is released in the next step S16. Specifically, the user issues an instruction to release the compression via the operation unit 26. Upon receiving the instruction to release the compression of the breast, the control unit 20 moves the compression members 34 in the compression release direction, moving the compression members 34 away from the imaging surface 40A of the imaging table 40, thereby releasing the compression of the breast by the compression members 34. When the processing of step S16 is completed in this manner, the continuous capture of the radiographic image X and the ultrasound image U is completed. In this embodiment, the continuous capture of the radiographic image X and the ultrasound image U while the breast W remains compressed by the compression members 34, as shown in FIG. 7 , is referred to as "continuous capture." The order of capturing the radiographic image X (step S12 in FIG. 7 ) and the ultrasound image U (step S14 in FIG. 7 ) is not limited. However, from the viewpoint of shortening the breast compression time, it is preferable to capture the radiographic image X first, as in this embodiment. Furthermore, the compression force applied to the breast by the compression member 34 may be reduced to the extent that it is considered that there is no change in the compression state of the breast between capturing the radiological image X and capturing the ultrasound image U. For example, the compression force applied by the compression member 34 may be reduced to the extent that it is considered that there is no change in the state of expansion of the mammary glands of the breast.

[0074] When the continuous capturing of the radiographic image X and the ultrasonic image U is completed in this manner, image processing is performed by the image processing device 18. As an example, when the image processing device 18 of this embodiment receives the display conditions for the radiographic image X and the ultrasonic image U to be displayed, input by the user via the operation unit 66 as described above, the CPU 60A of the control unit 60 executes the image processing program 61 stored in the ROM 60B, thereby performing the image processing, an example of which is shown in Fig. 8. Fig. 8 shows a flowchart illustrating an example of the flow of image processing in the image processing device 18 of this embodiment.

[0075] First, in step S100, the radiographic image acquisition unit 70 acquires a radiographic image X that satisfies the display conditions from the image storage system 19, as described above, and outputs it to the display control unit 79. In the next step S102, the ultrasound image acquisition unit 71 acquires a plurality of ultrasound images U that satisfy the display conditions from the image storage system 19, as described above, and outputs them to the ultrasound cross-sectional image generation unit 74. Through the processing of steps S100 and S102, the image processing device 18 acquires a set of a radiographic image X and an ultrasound image U obtained by continuous imaging.

[0076] In the next step S104, as described above, the ultrasound cross-sectional image generating unit 74 generates a plurality of ultrasound cross-sectional images corresponding to cross sections at any depth from the plurality of ultrasound images U acquired by the ultrasound image acquiring unit 71. The ultrasound cross-sectional image generating unit 74 outputs the generated ultrasound cross-sectional images to the breast contour detecting unit 76 and the display control unit 79.

[0077] In the next step S106, the breast contour detection unit 76 detects the contour of the breast contained in each ultrasonic cross-sectional image as described above, and outputs it to the display control unit 79.

[0078] In the next step S108, the display control unit 79 causes the display unit 68 to display the radiographic image X on which the breast contour included in the ultrasound cross-sectional image is superimposed, as described above. As described above, the size of the breast contour included in the ultrasound cross-sectional image may vary depending on the depth of the ultrasound cross-sectional image. Therefore, the display control unit 79 may arbitrarily select which ultrasound cross-sectional image to superimpose the breast contour included in the ultrasound cross-sectional image on the radiographic image X, and various modifications are possible. For example, as shown in FIG. 9 , the display control unit 79 may cause the display unit 68 to display the breast contour C2 (see also FIG. 6 ) having the largest area superimposed on the radiographic image X. Furthermore, for example, the display control unit 79 may receive a breast depth specification from the radiologist and superimpose the breast contour included in the ultrasound cross-sectional image corresponding to the received breast depth on the radiographic image X. Figure 9 shows an example of when a depth corresponding to the imaging surface 40A of the imaging table 40 is specified, and shows the state in which the display control unit 79 displays the breast contour C3 (see also Figure 6) on the display unit 68 with it superimposed on the radiation image X.

[0079] The display control unit 79 may also sequentially switch and display the breast area to be superimposed on the radiation image X at predetermined time intervals.

[0080] In the next step S110, the display control unit 79 also displays on the display unit 68 an ultrasound cross-sectional image D corresponding to the outline of the breast superimposed on the radiation image X, as shown in Fig. 9. When the processing of step S110 is completed, the image processing shown in Fig. 8 is also completed.

[0081] In this manner, in the image processing device 18 of this embodiment, as shown in FIG. 9, the breast contour (C2, C3 in FIG. 9) contained in the ultrasound cross-sectional image D is displayed on the display unit 68 in a state where it is superimposed on the radiation image X.

[0082] The display control unit 79 may also display the ultrasound cross-sectional image D including the contour of the breast with the contour of the breast included in the radiation image X superimposed thereon.

[0083] The present embodiment may be modified as follows: (Modification 1) In the above embodiment, the form in which the contour of the breast included in the ultrasound cross-sectional image is superimposed on the radiation image X has been described, but in this modification, the contour of the breast included in a composite ultrasound image obtained by combining multiple ultrasound cross-sectional images in the depth direction is superimposed on the radiation image X.

[0084] As shown in FIG. 10 , the image processing device 18 of this modified example further includes a composite ultrasound image generation unit 75. The composite ultrasound image generation unit 75 generates a composite ultrasound image by combining, in the depth direction, multiple ultrasound cross-sectional images generated by the ultrasound cross-sectional image generation unit 74 at different depth positions. As an example, the composite ultrasound image generation unit 75 generates a composite ultrasound image by adding pixel values ​​in the depth direction for the breast region of each ultrasound cross-sectional image. The composite ultrasound image generation unit 75 also outputs an ultrasound image averaged in the depth direction by the number of ultrasound cross-sectional images whose pixel values ​​have been added at each position in the breast region to the display control unit 79 as an ultrasound image for display. Note that, at each position, if the number of added ultrasound cross-sectional images is equal to or less than a threshold, the composite ultrasound image generation unit 75 preferably excludes that region from the breast region or breast contour. In this way, if the number of added ultrasound cross-sectional images is equal to or less than the threshold, noise can be excluded from the composite ultrasound image by excluding that region from the breast region or breast contour. The composite ultrasound image generating unit 75 outputs the generated composite ultrasound image to the breast contour detecting unit 76 and the display control unit 79 .

[0085] The breast contour detection unit 76 of this modified example detects the contour of the breast included in the composite ultrasound image and outputs the detection result to the display control unit 79 .

[0086] 11 , the display control unit 79 of this modified example causes the display unit 68 to display the breast contour C_g included in the composite ultrasound image G superimposed on the radiographic image X. Furthermore, the display control unit 79 also causes the display unit 68 to display the composite ultrasound image G corresponding to the breast contour superimposed on the radiographic image X, as shown in FIG.

[0087] (Modification 2) In this modification, a case will be described in which the radiographic images X acquired by the radiographic image acquisition unit 70 are a plurality of radiographic images obtained by so-called tomosynthesis imaging.

[0088] The mammography apparatus 10 shown in FIG. 2 is capable of so-called tomosynthesis imaging. Tomosynthesis imaging is an imaging method in which radiation is irradiated onto a subject (a breast in this embodiment) at different irradiation angles, and a radiographic image X (so-called projection image) is captured for each irradiation angle. When performing tomosynthesis imaging, the arm 42 rotates to move the radiation source 36R of the radiation irradiator 36 to each of a plurality of irradiation positions with different irradiation angles (projection angles). For example, the radiation source 36R is moved to irradiation positions with different irradiation angles by a predetermined angle. In other words, the radiation source 36R is moved to positions where the angle of incidence of radiation R with respect to the detection surface of the radiation detector 30 is different. At each irradiation position, radiation R is irradiated from the radiation source 36R in response to instructions from the console 12, and a radiographic image X is captured by the radiation detector 30. In this way, tomosynthesis imaging obtains a plurality of radiographic images X corresponding to the number of irradiation positions. In addition, the mammography device capable of tomosynthesis imaging may be different from the mammography device 10 shown in Figure 2, and may be, for example, a mammography device in which a radiation source 36R is provided for each irradiation position, and the entire device is equipped with multiple radiation sources 36R.

[0089] As shown in FIG. 12, the image processing device 18 of this modified example further includes a radiation tomographic image generating unit 73 in addition to the components of the image processing device 18 of the embodiment (see FIG. 5).

[0090] The radiation image acquisition unit 70 of this modified example acquires a plurality of radiation images X obtained by tomosynthesis imaging, and outputs them to the radiation tomographic image generation unit 73 .

[0091] The radiation tomographic image generating unit 73 generates, from the multiple radiation images X, multiple radiation tomographic images corresponding to multiple cross sections at different heights from the imaging surface 40A of the imaging table 40. The number of radiation tomographic images generated by the radiation tomographic image generating unit 73 may be the same as the number of ultrasound cross sections generated by the ultrasound cross section image generating unit 74, for example. As described above, in an ultrasound image, "depth" refers to the depth of the breast on the imaging table 40 from the surface that contacts the compression member 34. On the other hand, the radiation tomographic image may specify the position of the corresponding cross section in the height direction from the imaging surface 40A of the imaging table 40, using the imaging surface 40A of the imaging table 40 as a reference. In this case, the radiation tomographic image generating unit 73 converts the height in the radiation tomographic image into the depth in the ultrasound image (ultrasound cross section image).

[0092] The method by which the radiation tomographic image generating unit 73 generates a radiation tomographic image from the plurality of radiation images X is not particularly limited, and any known method can be used. The radiation tomographic image generating unit 73 outputs the generated plurality of radiation tomographic images to the display control unit 79.

[0093] In this modified example, as shown in FIG. 13, the display control unit 79 displays multiple (six in FIG. 13) radiation sectional images M_1 to M_6 side by side on the display unit 68, with breast contours C_1 to C_6 ​​contained in ultrasound sectional images corresponding to the cross sections at the corresponding depths superimposed on each of the radiation sectional images M_1 to M_6.

[0094] In this manner, in the image processing device 18 of this modified example, breast contours C-1 to C-6 contained in the corresponding ultrasound cross-sectional images are superimposed on each of the radiation tomographic images M_1 to M_ at different heights (or depths). Therefore, the image processing device 18 of this modified example can facilitate comparison between the radiation tomographic images and the ultrasound cross-sectional images.

[0095] As described above, in the image processing device 18 of the above embodiment and each modified example, the radiographic image acquisition unit 70 acquires a radiographic image X captured while the breast is compressed by a compression member, and the ultrasound image acquisition unit 71 acquires multiple ultrasound images U of the breast in a compressed state that can be considered to be identical to the radiographic image X, captured while scanning with the ultrasound probe.

[0096] The display control unit 79 superimposes, on the radiation image X, the contour of the breast contained in the ultrasound cross-sectional image that is synthesized from multiple ultrasound images and is parallel to the imaging table 40. Furthermore, the display control unit 79 displays the radiation image X with the contour of the breast superimposed.

[0097] Therefore, according to the image processing device 18 of the above embodiment and each of the modifications, the image interpreter can intuitively grasp the correspondence between the radiological image and the ultrasonic cross-sectional image.

[0098] The method by which the breast contour detection unit 76 determines the breast contour from each of the multiple ultrasound images is not limited to the method described above. Alternatively, the breast contour may be determined from each of the multiple ultrasound images U based on image features that indicate whether or not the area received ultrasound echoes from the breast. Areas where no breast is present do not receive ultrasound echoes from the breast and exhibit only multiple reflections. Therefore, as shown in FIG. 14 , the image features differ between the left and right sides of the boundary with the non-contact area in the ultrasound image U. In FIG. 14 , the left side of the boundary with the non-contact area is the area where the breast image UW is present, while the right side of the boundary with the non-contact area is the area of ​​multiple reflections, with the two areas exhibiting different image features. In the area of ​​multiple reflections, brightness peaks occur at regular intervals in areas deeper than the compression member 34, and the multiple reflections gradually attenuate. The breast contour detection unit 76 may determine the breast contour from each of the multiple ultrasound images based on the features of such areas of multiple reflections, i.e., areas other than the breast image UW. As described above, in areas of multiple reflection, positions where the brightness peaks are repeated at regular intervals, so applying a smoothing filter in the horizontal direction of the ultrasound image U can improve the accuracy of determining the breast contour.

[0099] In the above embodiment and each modification, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the radiographic image acquisition unit 70, the ultrasonic image acquisition unit 71, the radiographic tomographic image generation unit 73, the ultrasonic cross-sectional image generation unit 74, the composite ultrasonic image generation unit 75, the breast contour detection unit 76, the scanning position image generation unit 78, and the display control unit 79. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0100] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).Furthermore, multiple processing units may be configured with a single processor.

[0101] Examples of configuring multiple processing units with a single processor include: first, a configuration in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor functions as multiple processing units; second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); and thus, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0102] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0103] In the above embodiment, the image processing program 61 is pre-stored (installed) in the control unit 60 of the image processing device 18, but this is not limiting. The image processing program 61 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 61 may also be downloaded from an external device via a network.

[0104] The present disclosure can also be applied to programs and program products. Specifically, the image processing program 61 in each of the above embodiments may be provided as a program product. The program product includes any type of product for providing a program. For example, the program product includes a program provided via a network such as the Internet, and a non-transitory computer-readable recording medium such as a CD-ROM or DVD on which the program is stored.

[0105] Furthermore, the configurations and operations of the medical image capturing system 1, the radiographic image capturing system 2, the mammography device 10, the ultrasound image capturing device 16, and the image processing device 18 described in the above embodiment and each modification are merely examples, and it goes without saying that they can be modified according to the circumstances without departing from the spirit of the present invention. It also goes without saying that the above embodiments can be combined as appropriate.

[0106] The following supplementary notes are further disclosed regarding the above-described embodiments: (Supplementary Note 1) An image processing device comprising a processor, which acquires radiographic images captured with the breast compressed by a compression member, acquires a plurality of ultrasound images captured with the breast compressed by a compression member in a compression state that can be considered identical to the radiographic images, and superimposes, onto a radiographic image for display corresponding to the radiographic images, a contour of the breast included in an ultrasound cross-sectional image that is synthesized from the plurality of ultrasound images and is substantially parallel to an imaging table.

[0107] (Supplementary Note 2) The image processing device according to Supplementary Note 1, wherein the processor receives a designation of a depth of the breast, and displays a contour of the breast included in the ultrasound cross-sectional image corresponding to the received depth of the breast superimposed on the radiation image for display.

[0108] (Supplementary Note 3) The image processing device according to Supplementary Note 1 or Supplementary Note 2, wherein the processor determines the contour of the breast from each of the plurality of ultrasound images based on image features that indicate whether or not the area is a region that has received ultrasound echoes from the breast.

[0109] (Supplementary Note 4) The image processing device described in any one of Supplementary Note 1 to Supplementary Note 3, wherein the processor displays a composite ultrasound image obtained by combining multiple ultrasound cross-sectional images of the breast at different depths in the depth direction, the composite ultrasound image including the contour of the breast superimposed on the radiation image for display.

[0110] (Supplementary Note 5) The image processing device described in Supplementary Note 4, wherein the processor generates the composite ultrasound image by adding pixel values ​​in the depth direction for the breast region of the ultrasound cross-sectional image, and displays an ultrasound image averaged by the number of ultrasound cross-sectional images whose pixel values ​​have been added in the depth direction at each position of the breast region as an ultrasound image for display.

[0111] (Supplementary Note 6) The image processing device according to Supplementary Note 5, wherein the processor excludes a region from the region of the breast or the contour of the breast when the number of the added ultrasound cross-sectional images at each position is equal to or less than a threshold.

[0112] (Supplementary Note 7) The image processing device described in any one of Supplementary Note 1 to Supplementary Note 6, wherein the radiological image is a radiological image obtained by tomosynthesis imaging, the radiological image for display is each of a plurality of radiological tomographic images reconstructed from the radiological image, and the processor displays, for each of the plurality of radiological tomographic images, a contour of the breast included in the ultrasound cross-sectional image corresponding to a height of the radiological tomographic image superimposed thereon.

[0113] (Supplementary Note 8) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 7, wherein the processor also displays the ultrasound cross-sectional image including an outline of the breast.

[0114] (Supplementary Note 9) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 8, wherein the processor also displays the ultrasound cross-sectional image including the contour of the breast with the contour of the breast included in the radiation image for display superimposed thereon.

[0115] (Supplementary Note 10) A medical image capturing system comprising: the image processing device according to any one of Supplementary Notes 1 to 9; a radiographic image capturing device; and an ultrasonic image capturing device.

[0116] (Supplementary Note 11) An image processing method in which a processor included in an image processing device acquires a radiographic image captured while the breast is compressed by a compression member, acquires a plurality of ultrasound images captured while the breast is compressed by a compression member in a compression state that can be considered identical to the radiographic image, and superimposes an outline of the breast contained in an ultrasound cross-sectional image that is synthesized from the plurality of ultrasound images and is approximately parallel to an imaging table onto a radiographic image for display that corresponds to the radiographic image.

[0117] (Supplementary Note 12) An image processing program for causing a processor included in an image processing device to execute the following processes: acquire a radiographic image captured with the breast compressed by a compression member; acquire a plurality of ultrasound images captured with the breast compressed by a compression member in a compression state that can be considered identical to the radiographic image; and superimpose, onto a radiographic image for display corresponding to the radiographic image, a contour of the breast that is synthesized from the plurality of ultrasound images and is included in an ultrasound cross-sectional image that is approximately parallel to the imaging table.

[0118] The disclosure of Japanese Patent Application No. 2023-170616, filed on September 29, 2023, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards mentioned herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An image processing device comprising: a processor which acquires a radiological image of the breast compressed by a compression member, acquires a plurality of ultrasound images of the breast compressed by a compression member in a state that can be regarded as identical to the radiological image, and superimposes a contour of the breast contained in an ultrasound cross-sectional image that is synthesized from the plurality of ultrasound images and is approximately parallel to an imaging table onto a radiological image for display corresponding to the radiological image.

2. The image processing device according to claim 1, wherein the processor receives a designation of the depth of the breast and displays the contour of the breast contained in the ultrasound cross-sectional image corresponding to the received depth of the breast superimposed on the radiation image for display.

3. The image processing device according to claim 1, wherein the processor determines the contour of the breast from each of the plurality of ultrasound images based on image features indicating whether or not the area is one that has received ultrasound echoes from the breast.

4. The image processing device according to claim 1, wherein the processor displays the contour of the breast contained in a composite ultrasound image obtained by combining multiple ultrasound cross-sectional images of the breast at different depths in the depth direction, superimposed on the radiation image for display.

5. The image processing device according to claim 4, wherein the processor generates the composite ultrasound image by adding pixel values ​​in the depth direction for the region of the breast in the ultrasound cross-sectional image, and displays an ultrasound image averaged by the number of ultrasound cross-sectional images whose pixel values ​​have been added in the depth direction at each position of the region of the breast as an ultrasound image for display.

6. The image processing device according to claim 5, wherein the processor excludes an area from the breast area or the breast contour when the number of added ultrasound cross-sectional images at each position is equal to or less than a threshold value.

7. The image processing device according to claim 1, wherein the radiological image is a radiological image obtained by tomosynthesis imaging, the radiological image for display is each of a plurality of radiological tomographic images reconstructed from the radiological image, and the processor displays, for each of the plurality of radiological tomographic images, a contour of the breast included in the ultrasound cross-sectional image corresponding to a height of the radiological tomographic image superimposed thereon.

8. The image processing device according to claim 1, wherein the processor also displays the ultrasound cross-sectional image including the contour of the breast.

9. The image processing device according to claim 1, wherein the processor also displays the ultrasound cross-sectional image including the contour of the breast with the contour of the breast included in the radiation image for display superimposed thereon.

10. A medical image capturing system comprising: an image processing device according to any one of claims 1 to 9; a radiation image capturing device; and an ultrasound image capturing device.

11. An image processing method comprising the steps of: acquiring a radiological image captured by a processor equipped in an image processing device with the breast compressed by a compression member; acquiring a plurality of ultrasound images captured by a processor equipped in an image processing device with the breast compressed by a compression member in a state that can be regarded as identical to the radiological image; and superimposing a contour of the breast contained in an ultrasound cross-sectional image that is synthesized from the plurality of ultrasound images and is approximately parallel to the imaging table onto a radiological image for display corresponding to the radiological image.

12. An image processing program for causing a processor included in an image processing device to execute the following processes: acquire a radiographic image of the breast compressed by a compression member, acquire a plurality of ultrasound images of the breast compressed by a compression member in a state that can be regarded as identical to the radiographic image, and superimpose the contour of the breast contained in an ultrasound cross-sectional image that is synthesized from the plurality of ultrasound images and is approximately parallel to the imaging table onto a radiographic image for display corresponding to the radiographic image.

Images