ULTRASONIC DIAGNOSTIC APPARATUS AND METHOD FOR CONTROLLING ULTRASONIC DIAGNOSTIC APPARATUS

Abstract

This ultrasonic diagnostic device (1) comprises: an ultrasonic probe (2); an image generation unit (22) that generates an ultrasonic image which includes a region of interest of a breast of a subject in a radiographic image; an image adjustment unit (27) that adjusts the radiographic image and the ultrasonic image such that the region of interest in the radiographic image and the region of interest in the ultrasonic image are in the same orientation as each other on the basis of radiographic image orientation information stored in a tag of the radiographic image and probe orientation information of the ultrasonic probe (2) when the ultrasonic image was taken; and a monitor (24) that displays the adjusted radiographic image and ultrasonic image which have been adjusted by the image adjustment unit (27).

Description

[Technical field]

[0001] The present invention relates to an ultrasonic diagnostic apparatus for examining a breast of a subject and a method for controlling the ultrasonic diagnostic apparatus. [Background technology]

[0002] Conventionally, examinations of lesions and the like in a subject have been performed using an ultrasound diagnostic device. Prior to an examination using such an ultrasound diagnostic device, examinations of the inside of the subject are often performed in advance using an imaging diagnostic device other than the ultrasound diagnostic device, such as a CT (Computed Tomography) device. In this case, a user such as a doctor often observes both an ultrasound image captured by the ultrasound diagnostic device and a medical image captured by another imaging diagnostic device to diagnose lesions and the like in the subject.

[0003] In order to improve the accuracy of diagnosis using two different medical images, an ultrasound diagnostic device has been developed, for example, as disclosed in Patent Document 1. Patent Document 1 discloses that a two-dimensional CT cross-sectional image showing a cross section corresponding to an ultrasound image is selected based on three-dimensional data of a subject obtained by CT imaging, and the selected CT cross-sectional image and the ultrasound image are displayed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-39877 A Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, when an examination is performed on the breast of a subject using an ultrasonic diagnostic apparatus, an examination called mammography is often performed prior to the examination using the ultrasonic diagnostic apparatus.

[0006] In both the examination using an ultrasound diagnostic device and the examination using a CT device as disclosed in Patent Document 1, the examination is performed with the subject lying on his / her back on an examination table or the like. As a result, in the examination using an ultrasound diagnostic device and the examination using a CT device, the subject's breast is imaged from the same direction, and the shape of the subject's breast at the time of imaging can be made the same. Therefore, when an examination using a CT device is performed prior to the examination using an ultrasound diagnostic device, an ultrasound image and a CT cross-sectional image that are easy for a user to compare can be easily obtained.

[0007] However, when radiological images of a subject's breast are captured by mammography, the subject is photographed in a standing position, and the breast is compressed between a compression plate and an imaging table, so that the shape of the subject's breast in mammography differs from the shape of the subject's breast in an examination using an ultrasound diagnostic device. This makes it difficult to capture a radiological image that shows the same cross-section as the ultrasound image, and there is a problem that it is difficult for users such as doctors to compare the ultrasound image with the radiological image.

[0008] The present invention has been made to solve such conventional problems, and has an object to provide an ultrasound diagnostic apparatus and a control method for an ultrasound diagnostic apparatus that enable a user to easily compare ultrasound images and radiation images and improve the accuracy of diagnosis for a subject. [Means for solving the problem]

[0009] The ultrasound diagnostic device of the present invention is characterized by comprising an ultrasound probe, an image generation unit that generates an ultrasound image including a region of interest of the subject's breast imaged in the radiation image by transmitting and receiving an ultrasound beam to and from the subject using the ultrasound probe, an image adjustment unit that adjusts the radiation image and the ultrasound image so that the region of interest imaged in the radiation image and the region of interest imaged in the ultrasound image are oriented in the same direction based on radiation image orientation information stored in a tag of the radiation image and probe orientation information of the ultrasound probe at the time the ultrasound image was captured, and a monitor that displays the adjusted radiation image and ultrasound image adjusted by the image adjustment unit.

[0010] The probe orientation information is preferably position information of the ultrasound probe designated by a user, or position information detected by a position sensor mounted on the ultrasound probe. The image adjustment section can generate adjusted radiographic images and ultrasound images by performing at least one of rotation processing and inversion processing on at least one of the entire radiographic image and the entire ultrasound image.

[0011] Alternatively, a region of interest extraction unit may be provided which extracts a region of interest from each of the radiological image and the ultrasound image, in which case the image adjustment unit can generate adjusted radiological images and ultrasound images by performing at least one of a rotation process and an inversion process on at least one of the region of interest extracted from the radiological image and the region of interest extracted from the ultrasound image.

[0012] Furthermore, the image adjustment unit can superimpose a subject orientation mark indicating the orientation of the subject on the adjusted radiation image and ultrasound image. Furthermore, the image adjustment unit can generate adjusted radiological images and ultrasound images so that the area of ​​interest imaged in the radiological image and the area of ​​interest imaged in the ultrasound image are the same size. In this case, it is preferable that the image adjustment section determines a ratio of the sizes of the adjusted radiographic image and ultrasound image based on the inter-pixel distance of the radiographic image and the inter-pixel distance of the ultrasound image.

[0013] In addition, when a radiological image is acquired with the radiation source positioned in a direction inclined relative to the vertical direction, it is preferable that the image adjustment unit adjusts the radiological image and the ultrasound image after rotationally converting the radiological image based on the rotation angle of the radiation source.

[0014] In addition, the tag of the radiation image includes radiation image breast information indicating whether the subject's breast imaged in the radiation image is the left or right breast, and the image adjustment unit can adjust the radiation image and the ultrasound image based on the radiation image breast information and information input by the user indicating whether the subject's breast imaged in the ultrasound image is the left or right breast when the subject's breast imaged in the radiation image and the subject's breast imaged in the ultrasound image match each other. Furthermore, the image adjustment unit can further adjust already adjusted radiation images and ultrasound images based on readjustment information input by the user.

[0015] The control method for an ultrasound diagnostic apparatus according to the present invention is characterized in that it generates an ultrasound image including a region of interest of the subject's breast imaged in a radiation image by transmitting and receiving an ultrasound beam to and from the subject using an ultrasound probe, adjusts the radiation image and the ultrasound image based on radiation image orientation information stored in a tag of the radiation image and probe orientation information of the ultrasound probe at the time the ultrasound image was captured so that the region of interest imaged in the radiation image and the region of interest imaged in the ultrasound image are oriented in the same direction, and displays the adjusted radiation image and ultrasound image on a monitor. Effect of the Invention

[0016] According to the present invention, an ultrasound diagnostic device includes an image adjustment unit that adjusts a radiological image and an ultrasound image based on radiological image orientation information stored in a tag of the radiological image and probe orientation information of the ultrasound probe at the time the ultrasound image was captured so that the region of interest imaged in the radiological image and the region of interest imaged in the ultrasound image have the same orientation, and a monitor that displays the adjusted radiological image and ultrasound image adjusted by the image adjustment unit. This allows a user to easily compare the ultrasound image and the radiological image, thereby improving the accuracy of diagnosis for the subject. [Brief description of the drawings]

[0017] [Figure 1] 1 is a block diagram showing a configuration of an ultrasound diagnostic apparatus according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram of an example of a radiation image stored in a server in the first embodiment of the present invention. [Diagram 3] FIG. 1 is a schematic diagram of an ultrasonic probe according to a first embodiment of the present invention. [Figure 4] 1 is a block diagram showing a configuration of a transmission / reception circuit in a first embodiment of the present invention. [Diagram 5] FIG. 2 is a block diagram showing a configuration of an image generating unit according to the first embodiment of the present invention. [Figure 6] FIG. 2 is a schematic diagram illustrating an example of an ultrasound image according to the first embodiment of the present invention. [Figure 7] 3A to 3C are schematic diagrams illustrating an example of a radiological image adjusted in the first embodiment of the present invention. [Figure 8] 5 is a flowchart showing the operation of the ultrasound diagnostic apparatus according to the first embodiment of the present invention. [Figure 9] 2A to 2C are schematic diagrams illustrating an example of a radiological image and an ultrasonic image displayed on a monitor in the first embodiment of the present invention. [Figure 10] 5A to 5C are schematic diagrams illustrating other examples of a radiological image and an ultrasonic image displayed on the monitor in the first embodiment of the present invention. [Figure 11] FIG. 4 is a schematic diagram of another example of a radiological image adjusted in accordance with the first embodiment of the present invention. [Figure 12] 3A to 3C are schematic diagrams illustrating an example of a subwindow on a radiation image and a subwindow on an ultrasound image displayed on a monitor in accordance with the first embodiment of the present invention. [Figure 13] FIG. 2 is a schematic diagram of an example of a region of interest displayed in a subwindow in the first embodiment of the present invention. [Figure 14] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a second embodiment of the present invention. [Figure 15] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The following description of the components will be given based on a representative embodiment of the present invention, but the present invention is not limited to such an embodiment. In this specification, a numerical range expressed using "to" means a range that includes the numerical values ​​before and after "to" as the lower and upper limits. In this specification, the terms "same" and "identical" include a generally acceptable margin of error in the technical field.

[0019] First embodiment 1 shows the configuration of an ultrasonic diagnostic device 1 according to a first embodiment of the present invention. The ultrasonic diagnostic device 1 includes an ultrasonic probe 2 and a diagnostic device main body 3. The ultrasonic probe 2 and the diagnostic device main body 3 are connected to each other. The diagnostic device main body 3 is also connected to an external server 4 via a network NW.

[0020] The ultrasonic probe 2 includes a transducer array 11, to which a transmission / reception circuit 12 is connected in sequence.

[0021] The diagnostic device main body 3 includes an image generating unit 22, which is connected to the transmission / reception circuit 12 of the ultrasound probe 2. A display control unit 23 and a monitor 24 are connected to the image generating unit 22 in this order. A memory 25 is connected to the image generating unit 22. The diagnostic device main body 3 includes a communication unit 21, which is connected to the server 4 via the network NW. The memory 25 is connected to the communication unit 21. An image adjustment unit 27 is connected to the memory 25. The image adjustment unit 27 is connected to the display control unit 23.

[0022] A main body control unit 29 is connected to the transmission / reception circuit 12, the communication unit 21, the image generation unit 22, the display control unit 23, the memory 25, and the image adjustment unit 27 of the ultrasonic probe 2. An input device 30 is also connected to the main body control unit 29. The communication unit 21, the image generation unit 22, the display control unit 23, the image adjustment unit 27 and the main body control unit 29 form a processor 31.

[0023] The server 4 is installed, for example, in a hospital or the like, at a remote location relative to the location where the diagnostic apparatus main body 3 is installed. The server 4 manages image data, and can be used, for example, in a so-called PACS (Picture Archiving and Communication System: medical image management system).

[0024] The server 4 stores in advance a radiation image T1 as shown in Fig. 2, which is captured by a radiation diagnostic device (not shown). The radiation image T1 stored in the server 4 includes a region of interest A1 suspected of being a lesion. The radiation image T1 also has a tag for storing information about the radiation image T1. The tag stores radiation image orientation information, which is information about the orientation of the subject in the radiation image T1, such as the so-called A (Anterior) direction, P (Posterior) direction, R (Right) direction, L (Left) direction, H (Head) direction, F (Foot) direction, etc.

[0025] As the tag of the radiation image T1, a tag conforming to the so-called Digital Imaging and Communications in Medicine (DICOM) standard can be used.

[0026] In addition, the example shown in Figure 2 shows a radiological image T1 in which a subject's breast is photographed from the so-called CC (Cranio Caudal) direction, and four subject orientation marks indicating the orientation of the subject are arranged on the radiological image T1: an R direction mark D1 indicating the R direction, an L direction mark D2 indicating the L direction, an A direction mark D3 indicating the A direction, and a P direction mark D4 indicating the P direction.

[0027] As shown in Fig. 3, the ultrasonic probe 2 has a housing J made of resin or the like and incorporating various electric circuits and the like. The housing J has a gripping portion J1 for a user who performs an examination on a subject to grip the ultrasonic probe 2, and a tip portion J2 where the transducer array 11 is located, and a protruding marker M is formed near the tip portion J2 on one side of the housing J. The user can grasp the orientation of the ultrasonic probe 2 based on the orientation of the marker M. In addition, the orientation of the subject in the ultrasonic image is set to the A direction, P direction, R direction, L direction, H direction, F direction, or the like based on the orientation of the marker M.

[0028] The transducer array 11 of the ultrasonic probe 2 shown in Fig. 1 has a plurality of ultrasonic transducers arranged one-dimensionally or two-dimensionally. Each of these ultrasonic transducers transmits ultrasonic waves in accordance with a drive signal supplied from a transmission / reception circuit 12, receives ultrasonic echoes from a subject, and outputs a signal based on the ultrasonic echoes. Each ultrasonic transducer is configured by forming electrodes on both ends of a piezoelectric body made of, for example, a piezoelectric ceramic represented by PZT (Lead Zirconate Titanate), a polymeric piezoelectric element represented by PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal represented by PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0029] The transmission / reception circuit 12, under the control of the probe control unit 15, transmits ultrasonic waves from the transducer array 11 and generates sound ray signals based on reception signals acquired by the transducer array 11. As shown in Fig. 4, the transmission / reception circuit 12 has a pulser 16 connected to the transducer array 11, and an amplifier unit 17, an AD (Analog-to-Digital) conversion unit 18, and a beamformer 19, which are connected in series from the transducer array 11 in sequence.

[0030] The pulser 16 includes, for example, a plurality of pulse generators, and adjusts the delay amount of each drive signal and supplies it to the plurality of ultrasonic transducers of the transducer array 11 so that ultrasonic waves transmitted from the plurality of ultrasonic transducers form ultrasonic beams based on the transmission delay pattern selected in response to the control signal from the probe control unit 15. In this way, when a pulsed or continuous wave voltage is applied to the electrodes of the ultrasonic transducers of the transducer array 11, the piezoelectric body expands and contracts, and each ultrasonic transducer generates a pulsed or continuous wave ultrasonic wave, and an ultrasonic beam is formed from the composite wave of those ultrasonic waves.

[0031] The transmitted ultrasonic beam is reflected by an object such as a part of the subject, and propagates toward the transducer array 11 of the ultrasonic probe 2. The ultrasonic echo propagating toward the transducer array 11 in this manner is received by each ultrasonic transducer constituting the transducer array 11. At this time, each ultrasonic transducer constituting the transducer array 11 expands and contracts upon receiving the propagating ultrasonic echo, generating a received signal which is an electrical signal, and outputs this received signal to the amplifier 17.

[0032] The amplifier 17 amplifies the reception signals input from each ultrasonic transducer constituting the transducer array 11, and transmits the amplified reception signals to the AD converter 18. The AD converter 18 converts the reception signals transmitted from the amplifier 17 into digital reception data. The beamformer 19 performs so-called reception focusing processing by adding each piece of reception data received from the AD converter 18 with a respective delay. By this reception focusing processing, each piece of reception data converted by the AD converter 18 is phased and added, and a sound ray signal in which the focus of the ultrasonic echo is narrowed is acquired.

[0033] The communication unit 21 is configured with a circuit including an antenna for transmitting and receiving radio waves, a circuit for connecting to a LAN (Local Area Network), and communicates with the server 4 via the network NW under the control of the main body control unit 29. The communication unit 21 can receive radiographic images T1, etc. from the server 4 via the network NW.

[0034] As shown in FIG. 5, the image generating unit 22 has a configuration in which a signal processing unit 32, a DSC (Digital Scan Converter) 33, and an image processing unit 34 are connected in series.

[0035] The signal processing unit 32 performs correction for attenuation due to distance on the sound ray signals sent from the transmission / reception circuit 12 of the ultrasonic probe 2 using the sound velocity value set by the main body control unit 29 in accordance with the depth of the reflection position of the ultrasonic waves, and then performs envelope detection processing to generate a B-mode image signal, which is tomographic image information on the tissue in the subject.

[0036] The DSC 33 converts (raster converts) the B-mode image signal generated by the signal processor 32 into an image signal conforming to a normal television signal scanning method. The image processing unit 34 performs various necessary image processing such as gradation processing on the B-mode image signal input from the DSC 33, and then sends the B-mode image signal to the display control unit 23 and the memory 25. Hereinafter, the B-mode image signal that has been subjected to image processing by the image processing unit 34 will be referred to as an ultrasound image.

[0037] The memory 25 is for storing the ultrasound image generated by the image generation unit 22, the radiation image T1 transmitted from the server 4 to the communication unit 21 via the network NW, etc. The ultrasound image stored in the memory 25 is read out under the control of the main body control unit 29 and sent to the display control unit 23 and the image adjustment unit 27. In addition, the radiation image T1 stored in the memory 25 is read out under the control of the main body control unit 29 and sent to the image adjustment unit 27.

[0038] Examples of the memory 25 that can be used include recording media such as a flash memory, an HDD (Hard Disc Drive), an SSD (Solid State Drive), an FD (Flexible Disc), an MO disk (Magneto-Optical disc), an MT (Magnetic Tape), a RAM (Random Access Memory), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SD card (Secure Digital card), and a USB memory (Universal Serial Bus memory).

[0039] The image adjustment unit 27 adjusts the radiation image T1 and the ultrasound image based on the radiation image orientation information stored in the tag of the radiation image T1 saved in the memory 25 and the probe orientation information of the ultrasound probe 2 at the time the ultrasound image was captured, so that the area of ​​interest A1 imaged in the radiation image T1 and the area of ​​interest imaged in the ultrasound image have the same orientation.

[0040] Here, the probe orientation information is information on the orientation of the subject in the ultrasound image, such as the A direction, P direction, R direction, L direction, H direction, and F direction. The image adjustment unit 27 sets the probe orientation information based on information input by the user's input operation via the input device 30. As a result, for example, as shown in FIG. 6, four subject orientation marks representing the orientation of the subject are added to the ultrasound image U1, including an R direction mark D5 representing the R direction, an L direction mark D6 representing the L direction, an A direction mark D7 representing the A direction, and a P direction mark D8 representing the P direction. Note that the ultrasound image U1 shown in FIG. 6 is an image of the subject's breast.

[0041] For example, the image adjustment unit 27 adjusts the radiation image T1 shown in Figure 2 and the ultrasound image U1 shown in Figure 6 so that the R direction mark D1 and the R direction mark D5, the L direction mark D2 and the L direction mark D6, the A direction mark D3 and the A direction mark D7, and the P direction mark D4 and the P direction mark D8 each face in the same direction as each other.

[0042] In this case, the image adjustment unit 27 can adjust the radiation image T1 shown in FIG. 2 to an orientation as shown in FIG. 7 by, for example, rotating the radiation image T1 shown in FIG. 2 by 90 degrees clockwise, i.e., tilting the R direction mark D1 side toward the P direction mark D4 side, and then inverting it left-right, i.e., inverting the R direction mark D1 side and the L direction mark D2 side.

[0043] In the ultrasound image U1 shown in Figure 6 and the radiation image T1 shown in Figure 7, the R direction mark D1 and the R direction mark D5, the L direction mark D2 and the L direction mark D6, the A direction mark D3 and the A direction mark D7, and the P direction mark D4 and the P direction mark D8 each face in the same direction, and the area of ​​interest A1 in the radiation image T1 and the area of ​​interest A2 in the ultrasound image U1 also face in the same direction.

[0044] The main body control unit 29 controls each part of the diagnostic device main body 3 according to a prerecorded program or the like. Under the control of the main body control unit 29, the display control unit 23 performs predetermined processing on the ultrasound image U1 generated by the image generation unit 22 and the radiation image T1 transmitted from the server 4 to the communication unit 21 via the network NW, and displays the images on the monitor 24.

[0045] The monitor 24 performs various displays under the control of the display control unit 23. The monitor 24 includes a display device such as an LCD (Liquid Crystal Display) or an organic EL display (Organic Electroluminescence Display).

[0046] The input device 30 of the diagnostic device main body 3 is for a user to perform input operations. The input device 30 is configured by devices such as a keyboard, a mouse, a trackball, a touch pad, and a touch panel for a user to perform input operations.

[0047] The processor 31 including the communication unit 21, image generation unit 22, display control unit 23, image adjustment unit 27 and main body control unit 29 is composed of a CPU (Central Processing Unit) and a control program for causing the CPU to perform various processes, but may also be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), or other ICs (Integrated Circuits), or may be composed of a combination of these.

[0048] Furthermore, the communication section 21, image generation section 22, display control section 23, image adjustment section 27 and main body control section 29 may be configured as being partially or entirely integrated into a single CPU or the like.

[0049] The operation of the ultrasound diagnostic device 1 according to the embodiment of the present invention will be described below. First, in step S1, based on an input operation by a user via the input device 30, a radiation image T1 stored in the server 4 is transmitted to the communication unit 21 via the network NW, and the radiation image T1 is stored in the memory 25. As shown in Fig. 2, the radiation image T1 includes a region of interest A1 that is suspected to be a lesion. Furthermore, radiation image orientation information is stored in the tag of the radiation image T1, and an R-direction mark D1, an L-direction mark D2, an A-direction mark D3, and a P-direction mark D4 are arranged in the radiation image T1 based on the radiation image orientation information.

[0050] Next, in step S2, a plurality of frames of ultrasound images U are captured by the user with the ultrasound probe 2 in contact with the body surface of the subject. At this time, the transmission / reception circuit 12 performs reception focusing processing using a preset sound velocity value under the control of the probe control unit 15 to generate a sound ray signal. The sound ray signal generated by the transmission / reception circuit 12 in this manner is sent to the image generation unit 22. The image generation unit 22 generates an ultrasound image U1 as shown in FIG. 6 using the sound ray signal sent from the transmission / reception circuit 12. The generated ultrasound image U1 is stored in the memory 25.

[0051] In the next step S3, the image adjustment unit 27 sets probe orientation information based on an input operation by the user via the input device 30. At this time, the user inputs information regarding the orientation of the subject in the ultrasound image U1 while checking, for example, the direction of the marker M formed on the ultrasound probe 2 as shown in Fig. 3 and the orientation of the subject. As a result, for example, as shown in Fig. 6, an R-direction mark D5, an L-direction mark D6, an A-direction mark D7, and a P-direction mark D8 are arranged in the ultrasound image U1.

[0052] In step S4, the image adjustment unit 27 adjusts the radiographic image T1 and the ultrasound image U1 based on the radiographic image orientation information stored in the tag of the radiographic image T1 saved in step S1 and the probe orientation information set in step S3 so that the area of ​​interest A1 imaged in the radiographic image T1 and the area of ​​interest A2 imaged in the ultrasound image U1 have the same orientation.

[0053] For example, the image adjustment unit 27 rotates the radiation image T1 shown in Fig. 2 clockwise and then flips it left to right to adjust it to the orientation shown in Fig. 7. This allows the orientation of the region of interest A1 in the radiation image T1 shown in Fig. 7 and the orientation of the region of interest A2 in the ultrasound image U1 shown in Fig. 6 to be adjusted to be the same orientation. This makes it easier for the user to compare the region of interest A1 in the radiation image T1 with the region of interest A2 in the ultrasound image U1.

[0054] Finally, in step S5, the radiation image T1 and the ultrasound image U1 adjusted in step S4 are displayed on the monitor 24, as shown in FIG. In this manner, the operation of the ultrasound diagnostic apparatus 1 according to the first embodiment shown in the flowchart of FIG. 8 is completed.

[0055] As described above, according to the ultrasound diagnostic device 1 of the first embodiment of the present invention, the radiation image T1 and the ultrasound image U1 are adjusted based on the radiation image orientation information stored in the tag of the radiation image T1 and the probe orientation information of the ultrasound probe 2 when the ultrasound image U1 was captured, so that the orientation of the region of interest A1 in the radiation image T1 and the orientation of the region of interest A2 in the ultrasound image U1 are the same. This allows the user to easily compare the region of interest A1 in the radiation image T1 with the region of interest A2 in the ultrasound image U1, thereby improving the accuracy of diagnosis for the regions of interest A1 and A2.

[0056] In the ultrasonic diagnostic device 1, the image generating section 22 is provided in the diagnostic device main body 3, but it may be provided in the ultrasonic probe 2 instead of in the diagnostic device main body 3.

[0057] Further, although the ultrasonic probe 2 and the diagnostic apparatus body 3 are described as being connected to each other by wired communication, the ultrasonic probe 2 and the diagnostic apparatus body 3 can also be connected to each other by wireless communication.

[0058] Furthermore, although the diagnostic device main body 3 is provided with one memory 25, it may be provided with a plurality of memories depending on the application, etc.

[0059] Also, although it has been described that the radiation image T1 is transmitted from the server 4 to the diagnostic apparatus body 3 via the network NW, the radiation image T1 is not limited to being transmitted from the server 4. For example, the radiation image T1 can also be transmitted to the diagnostic apparatus body 3 from a radiation diagnostic apparatus (not shown).

[0060] 3, a protruding marker M is shown, but the shape of the marker M is not particularly limited as long as it can indicate the direction of the ultrasonic probe 2. The marker M may have, for example, a recessed shape or a flat shape with a pattern.

[0061] Also, although it has been described that the R-direction mark D1, the L-direction mark D2, the A-direction mark D3, and the P-direction mark D4 are arranged on the radiation image T1, the form of the marks indicating the directions is not particularly limited to this. For example, a so-called schema for diagrammatically representing a breast may be arranged on the radiation image T1, and a mark indicating the direction of the radiation source when capturing the radiation image T1 may be arranged on the schema, thereby indicating the directions in the radiation image T1.

[0062] Also, for the ultrasound image U1, the form of the mark indicating the direction is not limited to the R direction mark D5, the L direction mark D6, the A direction mark D7, and the P direction mark D8, etc. For example, a schema is placed on the ultrasound image U1, and a so-called probe mark indicating the position and orientation of the ultrasound probe 2 when capturing the ultrasound image U1 is placed on the schema, thereby making it possible to indicate the direction in the ultrasound image U1.

[0063] Furthermore, the position and orientation of the probe mark superimposed on the schema on the ultrasound image U1 can be set by an input operation of the user via the input device 30. In this case, the image adjustment unit 27 can set the probe orientation information based on the orientation of the probe mark input by the user.

[0064] Furthermore, the mark indicating the direction in the radiological image T1 and the mark indicating the direction in the ultrasound image U1 do not need to be displayed on the monitor 24. However, by displaying these marks on the monitor 24, the user can easily understand that the orientation of the region of interest A1 in the radiological image T1 and the orientation of the region of interest A2 in the ultrasound image U1 are the same.

[0065] In addition, although an example in which the image adjustment unit 27 performs left-right inversion on the radiographic image T1 has been described, the inversion process is not particularly limited to left-right inversion. For example, the image adjustment unit 27 can also perform a so-called up-down inversion process in which the R-direction mark D1 side and the L-direction mark D2 side of the radiographic image T1 shown in Fig. 2 are inverted. In addition, the inversion process performed by the image adjustment unit 27 is not limited to the process of inverting the radiographic image T1 about an axis that divides the radiographic image T1 in half in the left-right direction or an axis that divides the radiographic image T1 in half in the up-down direction, but can also set an arbitrary axis and perform a process of inverting the radiographic image T1 about the axis.

[0066] In addition, an example has been described in which rotation and inversion processing is performed on the radiological image T1 to make the orientation of the area of ​​interest A1 in the radiological image T1 the same as the orientation of the area of ​​interest A2 in the ultrasound image U1. However, the rotation and inversion processing may be performed on the ultrasound image U1 instead of the radiological image T1, or the rotation and inversion processing may be performed on both the radiological image T1 and the ultrasound image U1.

[0067] Also, although an example of adjusting a radiographic image T1 captured in the CC direction and an ultrasonic image U1 captured in the corresponding direction has been described, the capturing directions of the radiographic image T1 and the ultrasonic image U1 are not particularly limited. For example, as shown in Fig. 10, a radiographic image T2 captured of a subject's breast in a so-called MLO (Medio Lateral Oblique) direction and an ultrasonic image U2 captured in the corresponding direction are rotated and inverted, so that the orientation of a region of interest A3 in the radiographic image T2 and the orientation of a region of interest A4 in the ultrasonic image U2 can be made the same.

[0068] In the example of Figure 10, in the radiation image T2 and the ultrasound image U2, the HR (Head Right) direction mark C1 and the H direction mark C5, the FL (Foot Left) direction mark C2 and the F direction mark C6, the A direction mark C3 and the A direction mark C7, and the P direction mark C4 and the P direction mark C8 each point in the same direction.

[0069] Furthermore, for example, when imaging a subject's breast from the MLO direction, imaging of the breast is performed with the radiation source rotated at an arbitrary rotation angle. Therefore, when a radiation image T2 is acquired by imaging from the MLO direction with the radiation source rotated at a rotation angle Q, the image adjustment unit 27 can adjust the radiation image T2 and the ultrasound image U2 after rotationally converting the radiation image T1 based on the rotation angle Q of the radiation source, as shown in Fig. 11 . The image adjustment unit 27 can adjust the radiation image T2 and the ultrasound image U2 not only when the subject's breast is imaged from the MLO direction, but also when the radiation image T1 is acquired with the radiation source positioned in a direction inclined relative to the vertical direction. This allows the user to more easily compare the region of interest A3 in the radiation image T2 with the region of interest A4 in the ultrasound image U2, thereby improving the accuracy of diagnosis for the subject.

[0070] In addition, the tag of the radiation image T1 may include radiation image breast information indicating whether the subject's breast imaged in the radiation image T1 is the left or right breast, and further, ultrasound image breast information indicating whether the subject's breast imaged in the ultrasound image U1 is the left or right breast may be input by a user, for example, via the input device 30.

[0071] In this state, the image adjustment unit 27 can adjust the radiographic image T1 and the ultrasound image U1 based on the radiographic image breast information and the ultrasound image breast information so that the orientation of the area of ​​interest A1 in the radiographic image T1 and the orientation of the area of ​​interest A2 in the ultrasound image U1 are the same only if the subject's breast imaged in the radiographic image T1 and the subject's breast imaged in the ultrasound image U1 coincide with each other. This allows the user to easily compare the region of interest A1 and the region of interest A2 in the radiation image T1 and the ultrasound image U1 obtained by capturing the same breast, thereby improving the accuracy of diagnosis for the subject.

[0072] Furthermore, the image adjustment unit 27 can further adjust the radiation image T1 and ultrasound image U1 that have already been adjusted and are displayed on the monitor 24, based on readjustment information input by the user via the input device 30. This makes it possible to readjust the radiation image T1 and ultrasound image U1 so that they can be easily compared by the user, and further improves the accuracy of diagnosis for the subject.

[0073] In addition, instead of rotating and inverting the entire radiographic image T1 and the entire ultrasound image U1, the main body control unit 29 can also rotate and invert a partial area in the radiographic image T1 that includes the area of ​​interest A1 and a partial area in the ultrasound image U1 that includes the area of ​​interest A2 specified by the user, and display them on the monitor 24.

[0074] For example, as shown in FIG. 12, a sub-window W1 can be placed on a radiological image T1 and a sub-window W2 can be placed on an ultrasound image U1, and further, a region of interest B1 in the radiological image T1, which has been adjusted so that the orientations of the regions of interest A1 and A2 are the same, can be displayed in the sub-window W1 on the radiological image T1, and a region of interest B2 in the ultrasound image U1, which has been adjusted so that the orientations of the regions of interest A1 and A2 are the same, can be displayed in the sub-window W2 on the ultrasound image U1. This allows the user to easily compare the region of interest B1 and the region of interest B2 which face in the same direction.

[0075] Furthermore, the main body control unit 29 can enlarge and display the region of interest B1 or B2 so that the region of interest B1 in the subwindow W1 and the region of interest B2 in the subwindow W2 are the same in size based on the pixel distance in the radiation image T1 and the pixel distance in the ultrasound image U1, for example. This allows the user to more easily compare the region of interest B1 and the region of interest B2. The main body control unit 29 may enlarge both the region of interest B1 in the sub-window W1 and the region of interest B2 in the sub-window W2 so that the sizes of the region of interest B1 in the sub-window W1 and the region of interest B2 in the sub-window W2 are the same.

[0076] Here, the pixel distance between the radiographic image T1 and the ultrasonic image U1 refers to the actual length per pixel in each of the radiographic image T1 and the ultrasonic image U1. For example, if the pixel distance between the radiographic image T1 is 0.1 mm and the pixel distance between the ultrasonic image U1 is 0.5 mm, the size ratio of the radiographic image T1 to the size of the ultrasonic image U1 can be determined as 0.1 mm / 0.5 mm=0.2.

[0077] Information regarding the inter-pixel distance of the ultrasound image U1 is stored in advance, for example, in the image adjustment unit 27. Information regarding the inter-pixel distance of the radiation image T1 is stored in advance, for example, in the tag of the radiation image T1.

[0078] Furthermore, the main body control unit 29 can display marks indicating directions in the radiation image T1, such as an R direction mark F1, an L direction mark F2, an A direction mark F3, and a P direction mark F4, in the sub-window W1, as shown in Fig. 13. This allows the user to easily understand that the region of interest B1 in the sub-window W1 has been adjusted by rotation, inversion, etc. For the same reason, the main body control unit 29 can also display a mark indicating a direction in the ultrasound image U1 in the sub-window W2 on the ultrasound image U1.

[0079] Embodiment 2 In embodiment 1, an example is described in which a region of interest A2 in an ultrasound image U1 is set based on a user's input operation via an input device 30, but the region of interest A2 can also be extracted by performing image analysis on the ultrasound image U1.

[0080] As shown in FIG. 14, an ultrasonic diagnostic apparatus 1A according to the second embodiment includes a diagnostic apparatus main body 3A instead of the diagnostic apparatus main body 3 in the ultrasonic diagnostic apparatus 1 of the first embodiment shown in FIG. The diagnostic device body 3A is configured by adding a region of interest extraction unit 41 to the diagnostic device body 3 in the first embodiment, and by providing a body control unit 29A instead of the body control unit 29. Also, a processor 31A including the region of interest extraction unit 41 is provided instead of the processor 31.

[0081] In the diagnostic apparatus main body 3A, a region of interest extraction section 41 is connected to the memory 25. In addition, the region of interest extraction section 41 is connected to the image adjustment section 27 and a main body control section 29A.

[0082] The region of interest extraction unit 41 performs image analysis on the ultrasound image U1 stored in the memory 25 to extract a region of interest A2 within the ultrasound image U1.

[0083] For example, the region of interest extraction unit 41 pre-stores typical pattern data of the region of interest A2 as a template, calculates the similarity to the pattern data while searching the ultrasound image U1, and can determine that the region of interest A2 is present at a location where the similarity is greater than or equal to a threshold and is at its maximum.

[0084] In addition to the method of using simple template matching, the method of extracting the region of interest A2 can also use, for example, the machine learning method described in Csurka et al.: Visual Categorization with Bags of Keypoints, Proc. of ECCV Workshop on Statistical Learning in Computer Vision, pp.59-74 (2004), or the general image recognition method using deep learning described in Krizhevsk et al.: ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems 25, pp.1106-1114 (2012).

[0085] The image adjuster 27 adjusts the radiological image T1 and the ultrasonic image U1 so that the orientation of the region of interest A1 in the radiological image T1 and the orientation of the region of interest A2 in the ultrasonic image U1 become the same.

[0086] From the above, even when the region of interest A2 in the ultrasound image U1 is extracted by image analysis, as in embodiment 1, the radiological image T1 and the ultrasound image U1 are adjusted so that the orientation of the region of interest A1 in the radiological image T1 and the orientation of the region of interest A2 in the ultrasound image U1 are the same, so that the user can easily compare the region of interest A1 in the radiological image T1 and the region of interest A2 in the ultrasound image U1, thereby improving the diagnostic accuracy for the regions of interest A1, A2.

[0087] In addition, instead of rotating and inverting the entire radiological image T1 and the entire ultrasound image U1, the main body control unit 29 can also rotate and invert a partial area including the region of interest A1 in the radiological image T1 and a partial area including the region of interest A2 extracted by the region of interest extraction unit 41 in the ultrasound image U1, as shown in FIG. 12, and display them on the monitor 24.

[0088] At this time, the main body control unit 29 can enlarge and display the region of interest B1 or B2 so that the size of the region of interest B1 in the subwindow W1 and the size of the region of interest B2 in the subwindow W2 are the same based on the pixel distance of the radiation image T1 and the pixel distance of the ultrasound image U1, for example. This allows the user to more easily compare the region of interest B1 and the region of interest B2.

[0089] Furthermore, the main body control unit 29 can display marks indicating directions in the radiation image T1, such as an R direction mark F1, an L direction mark F2, an A direction mark F3, and a P direction mark F4, in the sub-window W1, as shown in Fig. 13. This allows the user to easily understand that the region of interest B1 in the sub-window W1 has been adjusted by rotation, inversion, etc. For the same reason, the main body control unit 29 can also display a mark indicating a direction in the ultrasound image U1 in the sub-window W2 on the ultrasound image U1.

[0090] In addition, the region of interest extraction unit 41 performs image analysis on the ultrasound image U1 to extract a region of interest A2 from within the ultrasound image U1, but can also perform image analysis on the radiation image T1 to extract a region of interest A1 from within the radiation image T1.

[0091] Third embodiment In the first embodiment, it is described that the probe orientation information is set based on the user's input operation via the input device 30. However, for example, the orientation of the ultrasonic probe 2 may be detected, and the probe orientation information may be set based on the detected orientation of the ultrasonic probe 2.

[0092] As shown in FIG. 15, an ultrasonic diagnostic apparatus 1B according to the third embodiment includes an ultrasonic probe 2B instead of the ultrasonic probe 2 in the ultrasonic diagnostic apparatus 1 according to the first embodiment shown in FIG. An ultrasonic probe 2B in the third embodiment is configured by adding a position sensor 42 to the ultrasonic probe 2 in the first embodiment. The position sensor 42 is connected to the memory 25 and the main body control unit 29 of the diagnostic apparatus main body 3.

[0093] The position sensor 42 is a sensor for detecting position information including the orientation of the ultrasonic probe 2B, and may include, for example, a magnetic sensor, an optical position sensor, an acceleration sensor, a gyro sensor, a GPS (Global Positioning System) sensor, etc. The position information of the ultrasonic probe 2B detected by the position sensor 42 is sent to the memory 25 of the diagnostic device main body 3, and under the control of the main body control unit 29, is stored in the memory 25 in association with the ultrasonic image U1 every time the image generation unit 22 generates the ultrasonic image U1.

[0094] The image adjustment unit 27 receives the ultrasound image U1 and the radiation image T1 from the memory 25, sets probe orientation information based on the position information of the ultrasound probe 2B associated with the ultrasound image U1, and adjusts the radiation image T1 and the ultrasound image U1 based on the probe orientation information and the radiation image orientation information of the radiation image T1 so that the orientation of the area of ​​interest A1 in the radiation image T1 and the orientation of the area of ​​interest A2 in the ultrasound image U1 are the same.

[0095] From the above, even when the probe orientation information is set based on the position information of the ultrasound probe 2B detected by the position sensor 42, as in embodiment 1, the radiological image T1 and the ultrasound image U1 are adjusted so that the orientation of the region of interest A1 in the radiological image T1 and the orientation of the region of interest A2 in the ultrasound image U1 are the same, so that the user can easily compare the region of interest A1 in the radiological image T1 and the region of interest A2 in the ultrasound image U1, thereby improving the diagnostic accuracy for the regions of interest A1, A2. [Explanation of symbols]

[0096] 1,1A,1B Ultrasound diagnostic device, 2,2B Ultrasound probe, 3,3A Diagnostic device main body, 4 Server, 11 Transducer array, 12 Transmitting / receiving circuit, 15 Probe control unit, 16 Pulser, 17 Amplification unit, 18 AD conversion unit, 19 Beamformer, 21 Communication unit, 22 Image generation unit, 23 Display control unit, 24 Monitor, 25 Memory, 27 Image adjustment unit, 29,29A Main body control unit, 30 Input device, 31,31A Processor, 32 Signal processing unit, 33 DSC, 34 Image processing unit, 41 Region of interest extraction unit, 42 Position sensor, A1 to A4, B1, B2 Region of interest, C1 HR direction mark, C2 FL direction mark, C3, C7, D3, D7, F3 A direction mark, C4, C8, D4, D8, F4 P direction mark, C5 H direction mark, C6 F direction mark, D1,D5,F1 R direction mark, D2,D6,F2 L direction mark, J housing, J1 gripping part, J2 tip, M mark, NW network, Q rotation angle, T1,T2 radiological image, U1,U2 ultrasound image, W1,W2 sub-window.

Claims

1. An ultrasonic probe; an image generating unit that generates a two-dimensional ultrasound image including a region of interest of the breast of the subject captured in a two-dimensional radiation image by transmitting and receiving an ultrasound beam to and from the subject using the ultrasound probe; an image adjustment unit that adjusts at least one of the radiological image and the ultrasonic image so that the region of interest imaged in the radiological image and the region of interest imaged in the ultrasonic image are oriented in the same direction, based on radiological image orientation information, which is information that represents a correspondence relationship between a plurality of directions in the radiological image and a plurality of reference directions based on the subject, and probe orientation information, which is information that represents a correspondence relationship between a plurality of directions in the ultrasonic image and the plurality of reference directions based on the subject, both stored in a tag of the radiological image; a monitor for displaying the radiographic image and the ultrasonic image that have been adjusted by the image adjustment unit; An ultrasound diagnostic device comprising:

2. The ultrasonic diagnostic apparatus according to claim 1 , wherein the image adjustment unit sets the probe orientation information based on information input by a user or position information detected by a position sensor mounted on the ultrasonic probe.

3. 3. The ultrasound diagnostic apparatus according to claim 1, wherein the image adjustment unit generates the adjusted radiation image and the adjusted ultrasound image by performing at least one of a rotation process and a flip process on at least one of the entire radiation image and the entire ultrasound image.

4. a region of interest extraction unit that extracts the region of interest from each of the radiation image and the ultrasound image, 3. The ultrasound diagnostic apparatus according to claim 1, wherein the image adjustment unit generates the adjusted radiation image and the adjusted ultrasound image by performing at least one of a rotation process and a flip process on at least one of the region of interest extracted from the radiation image and the region of interest extracted from the ultrasound image.

5. 5. The ultrasonic diagnostic apparatus according to claim 1, wherein the image adjustment section displays a subject orientation mark, which indicates an orientation of the subject, superimposed on the adjusted radiation image and the adjusted ultrasonic image.

6. 6. The ultrasound diagnostic apparatus according to claim 1, wherein the image adjustment unit generates the adjusted radiological image and the adjusted ultrasound image such that the region of interest imaged in the radiological image and the region of interest imaged in the ultrasound image are the same size.

7. The ultrasound diagnostic apparatus according to claim 6 , wherein the image adjustment unit determines a ratio of sizes of the adjusted radiation image and the adjusted ultrasound image based on a distance between pixels of the radiation image and a distance between pixels of the ultrasound image.

8. 8. The ultrasound diagnostic apparatus of claim 1, wherein, when the radiation image is acquired with a radiation source disposed in a direction inclined with respect to a vertical direction, the image adjustment unit adjusts at least one of the radiation image and the ultrasound image after rotationally converting the radiation image based on a rotation angle of the radiation source.

9. the tag of the radiation image includes radiation image breast information indicating whether the breast of the subject captured in the radiation image is the left or right breast; 3. The ultrasound diagnostic device of claim 2, wherein the image adjustment unit adjusts at least one of the radiographic image and the ultrasound image based on the radiographic image breast information and information input by the user indicating whether the subject's breast depicted in the ultrasound image is the left or right breast, when the subject's breast depicted in the radiographic image and the subject's breast depicted in the ultrasound image match each other.

10. The ultrasonic diagnostic apparatus according to claim 2 or 9, wherein the image adjustment unit further adjusts at least one of the radiation image and the ultrasonic image, which have already been adjusted, based on readjustment information input by the user.

11. generating a two-dimensional ultrasound image including a region of interest of the breast of the subject captured in the two-dimensional radiation image by transmitting and receiving an ultrasound beam to and from the subject using an ultrasound probe; adjusting at least one of the radiological image and the ultrasound image so that the region of interest imaged in the radiological image and the region of interest imaged in the ultrasound image are oriented in the same direction based on radiological image orientation information, which is information representing a correspondence relationship between a plurality of directions in the radiological image and a plurality of reference directions based on the subject, and probe orientation information, which is information representing a correspondence relationship between a plurality of directions in the ultrasound image and the plurality of reference directions based on the subject, stored in a tag of the radiological image; The adjusted radiographic image and the adjusted ultrasound image are displayed on a monitor. A method for controlling an ultrasonic diagnostic device.

Images