ULTRASONIC SYSTEM AND METHOD FOR CONTROLLING ULTRASONIC SYSTEM - Patent application

Abstract

To provide an ultrasonic system and an ultrasonic system control method capable of determining whether or not a region of interest in a breast included in a radiation image and a region of interest in a breast included in an ultrasonic image are identical.SOLUTION: In an ultrasonic system and an ultrasonic system control method, a second region-of-interest identification unit identifies a second region of interest included in an ultrasonic image, and an identity determination unit determines whether or not a first region of interest included in a radiation image and the second region of interest are identical to each other.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an ultrasound system having a function of determining whether a region of interest of a breast included in a radiological image is the same as a region of interest of a breast included in an ultrasound image, and a method for controlling the ultrasound system. [Background technology]

[0002] If a mammography examination identifies a breast region of interest that may be a lesion, a subsequent ultrasound examination may identify a breast region of interest in the ultrasound image that corresponds to the breast region of interest in the mammography image, and a qualitative diagnosis may be made as to whether the breast region of interest in the ultrasound image is a lesion.

[0003] Prior art documents that serve as references for the present invention include, for example, Patent Documents 1 and 2.

[0004] In the treatment system described in Patent Document 1, three-dimensional volume data of multi-slice images is acquired using an imaging diagnostic device such as an X-ray CT (Computed Tomography) device or an MRI (Magnetic Resonance Imaging) device, and a treatment region is set and marked on the volume data. A position sensor that detects the three-dimensional position and tilt of the ultrasound probe is attached to the treatment probe, providing information for determining which region of the subject the ultrasound probe is located at. From the volume data of the imaging diagnostic device, tomographic image data of the same cross section as the ultrasound image corresponding to the position of the treatment probe is extracted and used as a reference image. The marked treatment region is superimposed and displayed on the reference image. The reference image and ultrasound image are then displayed on a monitor in real time. When the marked treatment region appears on the reference image, the treatment probe is held in that position and treatment is performed.

[0005] Patent Document 2 relates to an image diagnosis support device that supports the diagnosis of a subject using multiple images obtained by photographing the same subject. This image diagnosis support device calculates, for each lesion on a plurality of images, a vector from a reference point on the image to the lesion, using the position of the lesion on the image. Next, the similarity between each vector on the plurality of images is calculated. As a result, if the similarity is equal to or greater than a predetermined threshold, the lesions corresponding to the vectors are determined to be the same lesion, and if the similarity is less than the threshold, the lesions are determined to be different lesions. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-146863 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-123528 Summary of the Invention [Problem to be solved by the invention]

[0007] However, mammography and ultrasound examinations differ in the posture of the subject during the examination due to the difference in examination method. For example, in mammography, a mammography image of the breast is generated while the subject is standing and the breast is compressed by a compression paddle. On the other hand, in ultrasound examinations, an ultrasound image of the breast is generated while the subject is lying supine. As a result, the shape and size of the breast region of interest in the mammography image and the breast region of interest in the ultrasound image appear different, which makes it difficult to identify the breast region of interest in the ultrasound image that corresponds to the breast region of interest in the mammography image.

[0008] An object of the present invention is to provide an ultrasound system and a method for controlling an ultrasound system that can determine whether a region of interest in a breast included in a radiological image is the same as a region of interest in a breast included in an ultrasound image. [Means for solving the problem]

[0009] In order to achieve the above object, the present invention provides an ultrasonic probe, an image generating unit that generates an ultrasound image including a second region of interest of the subject's breast corresponding to a first region of interest of the subject's breast included in the radiation image from a received signal obtained by transmitting and receiving an ultrasound beam to and from the subject's breast using an ultrasound probe; a second region of interest identifying unit that identifies a second region of interest included in the ultrasound image; an identity determination unit that determines whether a first region of interest and a second region of interest included in a radiographic image are identical to each other; an input device for receiving instructions input by a user; Equipped with The identity determination unit provides an ultrasound system that starts determination based on an instruction to freeze an ultrasound image input by a user.

[0010] Here, an ultrasound diagnostic device is provided, The ultrasound diagnostic apparatus preferably includes an ultrasound probe, an image generating unit, a second region of interest specifying unit, and an identity determining unit.

[0011] Also, an ultrasound diagnostic device, a remote computer connected to the ultrasound diagnostic device via a network, The ultrasound diagnostic device includes an ultrasound probe and an image generating unit. The remote computer preferably includes an identity determining unit.

[0012] The ultrasound diagnostic apparatus further includes a monitor and a display control unit. The display control unit preferably displays the radiographic image and the ultrasound image side by side on the monitor.

[0013] Also, the remote computer is a workstation, A monitor and a display control unit are provided, The display control unit preferably displays the radiographic image and the ultrasound image side by side on the monitor.

[0014] Also, the remote computer is a workstation, a client connected to the workstation via a network; The client includes a monitor and a display control unit, The display control unit preferably displays the radiographic image and the ultrasound image side by side on the monitor.

[0015] Also, the remote computer is a medical image management system; A medical image management system includes a monitor and a display control unit, The display control unit preferably displays the radiographic image and the ultrasound image side by side on the monitor.

[0016] Preferably, the display control unit enlarges and displays on the monitor the first region of interest included in the radiographic image and the second region of interest included in the ultrasound image.

[0017] It is also preferable that the display control unit displays the result of the determination made by the identity determination unit on a monitor.

[0018] Preferably, the identity determination unit makes the determination based on a radiographic image from one direction including the first region of interest, or on radiographic images from two different directions including the first region of interest.

[0019] Furthermore, it is preferable that the identity determination unit makes the determination based on an ultrasound image of one cross section including the second region of interest, or ultrasound images of two cross sections that are orthogonal to each other and that include the second region of interest.

[0020] Preferably, the identity determination unit makes the determination based on volume data consisting of ultrasound images of a plurality of different cross sections including the second region of interest, or a three-dimensional image reconstructed using the volume data.

[0021] Preferably, the identity determination unit identifies the second region of interest included in the ultrasound image based on an instruction input by the user to designate the second region of interest.

[0022] The second region of interest identification unit has a first determination model that uses a training ultrasound image of the subject's breast as first training data and learns the relationship between the training ultrasound image and the region of interest included in the training ultrasound image for a plurality of first training data; The first determination model preferably receives an ultrasound image as an input and identifies a second region of interest included in the ultrasound image.

[0023] Furthermore, the ultrasound image is a moving image. The first determination model preferably receives a moving image as an input and identifies a second region of interest included in the moving image.

[0024] The identity determination unit also has a second determination model that has learned, using a set of second teacher data consisting of a training radiographic image including a region of interest of the subject's breast and a training ultrasound image including a region of interest of the subject's breast that is identical to the region of interest included in the training radiographic image, a relationship between the training radiographic image and the training ultrasound image, and whether the region of interest included in the training radiographic image and the region of interest included in the training ultrasound image are identical, for a plurality of second teacher data; The second determination model preferably receives a pair of inputs, a radiographic image and an ultrasound image, and outputs a determination result as to whether a first region of interest included in the radiographic image and a second region of interest included in the ultrasound image are the same.

[0025] Furthermore, it is preferable that the identity determination unit starts determination based on an instruction to specify a second region of interest input by the user after the ultrasound image is generated based on an instruction to freeze the ultrasound image input by the user.

[0026] Preferably, the identity determination unit identifies the second region of interest included in the radiological image based on information on the first region of interest in the radiological image obtained from an external source.

[0027] The present invention also provides a method for generating an ultrasound image including a second region of interest of the subject's breast corresponding to a first region of interest of the subject's breast included in a radiation image from received signals obtained by transmitting and receiving ultrasound beams to and from the subject's breast using an ultrasound probe; Identifying a second region of interest in the ultrasound image; A control method for an ultrasound system is provided that starts determining whether a first region of interest and a second region of interest included in a radiation image are identical to each other based on an instruction to freeze an ultrasound image input by a user. [Effects of the Invention]

[0028] According to the present invention, a second region of interest of the breast included in the ultrasound image can be identified, and a determination of whether the first region of interest of the breast included in the radiographic image and the second region of interest of the breast are identical can be initiated based on an instruction to freeze the ultrasound image input by a user, thereby preventing the user from confusing the first region of interest of the breast included in the radiographic image with the second region of interest of the breast included in the ultrasound image. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a block diagram illustrating a configuration of an ultrasound system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram of an embodiment showing a configuration in which an ultrasound diagnostic apparatus acquires mammography images from a mammography apparatus. [Figure 3] FIG. 2 is a block diagram illustrating a configuration of a transmission / reception circuit according to an embodiment. [Figure 4] FIG. 2 is a block diagram illustrating a configuration of an image generating unit according to an embodiment. [Figure 5] FIG. 2 is a block diagram illustrating a configuration of a region of interest processing unit according to an embodiment. [Figure 6] 1 is a block diagram showing the configuration of a mammography apparatus according to an embodiment of the present invention; [Figure 7]1 is a flowchart illustrating one embodiment of the operation of an ultrasound system when capturing an ultrasound image. [Figure 8] 10 is a flowchart illustrating one embodiment of the operation of an ultrasound system when performing an identity determination of a region of interest for a right breast. [Figure 9] FIG. 2 is a conceptual diagram illustrating a display screen of a monitor according to an embodiment. [Figure 10] FIG. 1 is a block diagram of an embodiment showing a configuration in which an ultrasound diagnostic apparatus acquires mammography images from a remote computer. [Figure 11] FIG. 2 is a block diagram of one embodiment illustrating the configuration of a remote computer. DETAILED DESCRIPTION OF THE INVENTION

[0030] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An ultrasound system and a method for controlling an ultrasound system according to the present invention will be described in detail below based on preferred embodiments shown in the accompanying drawings.

[0031] Figure 1 is a block diagram showing the configuration of an embodiment of an ultrasound system according to the present invention. The ultrasound system shown in Figure 1 includes an ultrasound diagnostic device 20 having an ultrasound probe 1 and a device main body 3 connected to the ultrasound probe 1. As shown in Figure 2, the ultrasound diagnostic device 20 is connected to a mammography device 30 via a network 40, such as a local network within a hospital, which allows data to be exchanged in both directions.

[0032] The ultrasonic probe 1 scans a subject with an ultrasonic beam and outputs a sound ray signal corresponding to an ultrasonic image. As shown in Fig. 1, the ultrasonic probe 1 includes a transducer array 11 and a transmission / reception circuit 14. The transducer array 11 and the transmission / reception circuit 14 are bidirectionally connected. In addition, the transmission / reception circuit 14 is connected to a device control unit 36, which will be described later.

[0033] The transducer array 11 has a plurality of ultrasound transducers arranged one-dimensionally or two-dimensionally. Each of these transducers transmits ultrasound waves in accordance with a drive signal supplied from the transmission / reception circuit 14, and receives reflected waves from the subject and outputs an analog reception signal. Each vibrator is constructed using an element in which electrodes are formed on both ends of a piezoelectric body made of, for example, a piezoelectric ceramic such as PZT (Lead Zirconate Titanate), a polymer piezoelectric element such as PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal such as PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0034] The transmission / reception circuit 14, under the control of the device control unit 36, causes the transducer array 11 to transmit ultrasonic waves and generates sound ray signals by performing reception focusing processing on reception signals output from the transducer array 11 that have received ultrasonic echoes. As shown in Fig. 3, the transmission / reception circuit 14 has a pulser 51 connected to the transducer array 11, and an amplifier 52, an AD (Analog-to-Digital) converter 53, and a beamformer 54 that are connected in series from the transducer array 11 in this order.

[0035] The pulser 51 includes, for example, a plurality of pulse generators, and adjusts the delay amount of each drive signal and supplies it to the plurality of transducers in the transducer array 11 so that the ultrasonic waves transmitted from the plurality of transducers form an ultrasonic beam based on the transmission delay pattern selected by the device control unit 36. In this way, when a pulsed or continuous wave voltage is applied to the electrodes of the transducers in the transducer array 11, the piezoelectric material expands and contracts, and pulsed or continuous wave ultrasonic waves are generated from each transducer, and an ultrasonic beam is formed from the composite wave of these ultrasonic waves.

[0036] The transmitted ultrasonic beam is reflected by an object such as a part of the subject, and propagates toward the transducer array 11 of the ultrasonic probe 1. Each transducer constituting the transducer array 11 expands and contracts upon receiving the ultrasonic echo propagating toward the transducer array 11 in this manner, generating received signals which are electrical signals, and outputs these received signals to the amplifier unit 52.

[0037] The amplifier 52 amplifies the signals input from the respective transducers constituting the transducer array 11 and transmits the amplified signals to the AD converter 53. The AD converter 53 converts the analog signals transmitted from the amplifier 52 into digital received data and outputs the received data to the beamformer 54.

[0038] The beam former 54 performs so-called reception focusing processing by delaying and adding each piece of reception data converted by the AD conversion unit 53 according to the sound speed or sound speed distribution set based on the reception delay pattern selected by the device control unit 36. By this reception focusing processing, each piece of reception data converted by the AD conversion unit 53 is phased and added, and a sound ray signal in which the focus of the ultrasonic echo is narrowed is generated.

[0039] Next, the device main body 3 displays an ultrasound image based on the sound ray signals generated by the ultrasound probe 1. As shown in FIG. 1 , the device main body 3 includes an image generation unit 31, an image memory 32, a region of interest processing unit 35, a display control unit 33, a device control unit 36, a monitor (display unit) 34, and an input device 37.

[0040] A display control unit 33 and a monitor 34 are connected in series to the image generation unit 31. An image memory 32 and a region of interest processing unit 35 are connected to the image generation unit 31, and the image memory 32 and the region of interest processing unit 35 are connected to the display control unit 33. An apparatus control unit 36 is connected to the aforementioned transmission / reception circuit 14, image generation unit 31, display control unit 33, and region of interest processing unit 35, and an input device 37 is connected to the apparatus control unit 36.

[0041] The image generating unit 31 generates an ultrasound image (ultrasound image signal) based on the sound ray signal generated by the transmitting and receiving circuit 14 under the control of the device control unit 36. As shown in Fig. 4, the image generating unit 31 has a configuration in which a signal processing unit 16, a DSC (Digital Scan Converter) 18, and an image processing unit 17 are connected in series.

[0042] The signal processing unit 16 generates image information data corresponding to an ultrasound image based on the sound ray signals generated by the transmitting / receiving circuit 14. More specifically, the signal processing unit 16 performs signal processing on the sound ray signals generated by the beam former 54 of the transmitting / receiving circuit 14, for example, performs correction for attenuation caused by the propagation distance in accordance with the depth of the position where the ultrasound is reflected, and then performs envelope detection processing to generate image information data representing tomographic image information regarding tissue within the subject.

[0043] The DSC 18 raster-converts the image information data generated by the signal processing unit 16 into an image signal that conforms to the scanning method of a normal television signal.

[0044] The image processing unit 17 performs various image processing such as brightness correction, tone correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction on the image signal input from the DSC 18 in accordance with the display format of the monitor 34 to generate an ultrasound image (ultrasound image signal), and outputs the generated ultrasound image to the display control unit 33 and the image memory 32.

[0045] In this embodiment, the image generation unit 31 generates an ultrasound image including a second region of interest of the subject's breast corresponding to a first region of interest of the subject's breast included in the radiation image from received signals obtained by transmitting and receiving ultrasound beams to and from the subject's breast using the ultrasound probe 1 (more precisely, the transducer array 11), or more specifically, from sound ray signals generated from the received signals by the transmission / reception circuit 14.

[0046] The image memory 32 is a memory that holds a series of multiple frames of ultrasound images (ultrasound image signals) generated for each diagnosis by the image generation unit 31. As the image memory 32, a recording medium such as a flash memory, an HDD (Hard Disc Drive), an SSD (Solid State Drive), an FD (Flexible Disc), an MO disk (Magneto-Optical disc), an MT (Magnetic Tape), a RAM (Random Access Memory), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SD card (Secure Digital card), or a USB memory (Universal Serial Bus memory), or a server, etc., can be used.

[0047] Region of interest processing unit 35 performs processing such as identifying a second region of interest and determining whether the first region of interest and the second region of interest are identical to a first region of interest of the subject's breast included in a mammography image and a second region of interest of the subject's breast included in an ultrasound image under the control of device control unit 36. As shown in FIG. 5 , region of interest processing unit 35 includes second region of interest identification unit 44 and identity determination unit 46.

[0048] The second region of interest specifying unit 44 has a first determination model, and specifies a second region of interest of the subject's breast included in the ultrasound image using the first determination model.

[0049] The first judgment model is a trained model that uses a training ultrasound image of an arbitrary subject's breast as first training data and has learned the relationship between the training ultrasound image and the region of interest of the breast contained in this training ultrasound image for multiple first training data. The first determination model receives an ultrasound image to be determined as an input and outputs a determination result (prediction result) of the second region of interest of the breast included in the ultrasound image based on the learning result. In other words, the first determination model identifies the second region of interest of the breast included in the ultrasound image. The first determination model may learn normal breast data (data without a region of interest) as the first training data, and when abnormal data (data with a region of interest) is input, output a determination result that the data is not normal, i.e., that a second region of interest is present. Alternatively, the first determination model may learn abnormal breast data as the first training data, and when normal data is input, output a determination result that the data is normal, i.e., that a second region of interest is not present. The determination result may also be output together with the coordinates and position of the extracted second region of interest.

[0050] It is not essential that second region of interest identifying unit 44 has the first determination model, and the second region of interest of the breast included in the ultrasound image may be identified based on, for example, an instruction input by a user to specify the second region of interest of the breast by surrounding it with a rectangular frame. Alternatively, an image analysis unit that analyzes the ultrasound image may be provided, and second region of interest identifying unit 44 may identify the second region of interest of the breast included in the ultrasound image based on the result of analysis of the ultrasound image by the image analysis unit.

[0051] The identity determination unit 46 has a second determination model, and uses the second determination model to determine whether a first region of interest of the breast contained in the mammography image and a second region of interest of the breast contained in the ultrasound image are identical to each other, and outputs the result of the determination.

[0052] The second judgment model is a trained model that has learned the relationship between the training mammography image and the training ultrasound image and whether the region of interest in the breast included in the training mammography image is the same as the region of interest in the breast included in the training ultrasound image, using multiple sets of second training data, each of which is a training mammography image including a region of interest in the breast of an arbitrary subject and a training ultrasound image including a region of interest in the breast of the same subject as the region of interest in the breast included in the training mammography image. The training mammography images and training ultrasound images may be images that include only the region of interest of the breast. The second determination model, based on its learning results, receives a set of a mammography image and an ultrasound image to be determined as inputs, and outputs a determination result (prediction result) as to whether a first region of interest of the breast included in the mammography image and a second region of interest of the breast included in the ultrasound image are identical. In other words, the second determination model determines whether a first region of interest of the breast included in the radiographic image and a second region of interest of the breast included in the ultrasound image are identical to each other.

[0053] The second judgment model may output a judgment result of whether they are identical or not, or may output a judgment result of what percentage of the breast is identical between a first region of interest in the breast contained in a mammography image and a second region of interest in the breast contained in an ultrasound image.

[0054] The display control unit 33, under the control of the device control unit 36, causes various types of information to be displayed on the monitor 34. For example, the display control unit 33 performs predetermined processing on the ultrasound image stored in the image memory 32, and causes the processed ultrasound image to be displayed on the monitor 34. The display control unit 33 also causes the monitor 34 to display a mammography image and an ultrasound image side by side, and causes the monitor 34 to display the results of the determination by the identity determination unit 46.

[0055] The device control unit 36 controls each unit of the device main body 3 based on a pre-stored program and user instructions input from the input device 37. More specifically, the device control unit 36 controls the display control unit 33 to display the ultrasound image on the monitor 34. The device control unit 36 also controls the region of interest processing unit 35 to determine whether the region of interest of the breast included in the mammography image is the same as the region of interest of the breast included in the ultrasound image.

[0056] The image generating unit 31, the display control unit 33, the region of interest processing unit 35, and the device control unit 36 constitute a processor 39 for the ultrasound diagnostic device 20.

[0057] Monitor 34 displays various types of information under the control of display control unit 33. Monitor 34 displays mammography images, ultrasound images, and results of identity determination, etc. Examples of monitor 34 include an LCD (Liquid Crystal Display) and an organic EL (Electro-Luminescence) display.

[0058] The input device 37 receives various instructions input by the user, and includes, for example, various buttons including a judgment start button and a freeze button, and a touch panel that the user touches to input various instructions.

[0059] Next, Figure 6 is a block diagram showing the configuration of one embodiment of a mammography apparatus. The mammography apparatus 30 shown in Figure 6 includes an X-ray source 60 and an X-ray detector 62. In a mammography examination, the breast of a subject in an upright position is compressed by a compression paddle, and X-rays are irradiated from the X-ray source 60 onto the breast compressed by the compression paddle. The X-rays that pass through the breast are then detected by the X-ray detector 62, and a mammography image of the breast is generated from the detection signal detected by the X-ray detector 62.

[0060] In a mammography examination, for example, mammography images (R_MLO image and R_CC image) are generated in the MLO (Miolateral-Oblique) and CC (Cranio-Caudal) directions of a subject's right breast. Similarly, mammography images (L_MLO image and L_CC image) are generated in the MLO and CC directions of a subject's left breast. That is, an R_MLO image, an R_CC image, an L_MLO image, and an L_CC image are generated.

[0061] A first region of interest of the breast included in the mammography image is identified by a first region of interest identification unit. The first region of interest identification unit has a function of extracting the first region of interest from the mammography image and is a computer-aided detection (CAD) performed in the mammography device 30, a computer associated with the mammography device 30, or a remote computer (described later). The first region of interest identification unit has a third determination model and uses the third determination model to identify the first region of interest of the subject's breast included in the mammography image.

[0062] The third judgment model is a trained model that uses a training mammography image of an arbitrary subject's breast as third training data and has learned the relationship between the training mammography image and the breast region of interest contained in this training mammography image for multiple third training data. Based on the learning result, the third determination model inputs a mammography image to be determined and outputs a determination result (prediction result) of the first region of interest of the breast included in the mammography image. In other words, the third determination model identifies the first region of interest of the breast included in the mammography image. The third determination model may learn normal breast data (data without a region of interest) as third training data, and when abnormal data (data with a region of interest) is input, output a determination result that the data is not normal, i.e., that the first region of interest is present. Alternatively, the third determination model may learn abnormal breast data as third training data, and when normal data is input, output a determination result that the data is normal, i.e., that the first region of interest is not present. The determination result may be output together with the coordinates and position of the extracted first region of interest.

[0063] It is not essential that the first region of interest identifying unit has the third determination model, and the first region of interest of the breast included in the mammography image may be identified based on, for example, an instruction input by a user to specify the first region of interest of the breast by surrounding it with a rectangular frame. Also, an image analysis unit that analyzes the mammography image may be provided, and the first region of interest identifying unit may identify the first region of interest of the breast included in the mammography image based on the results of the analysis of the mammography image by the image analysis unit.

[0064] Information about the first region of interest included in the mammography image is supplied to the ultrasound diagnostic device 20 directly from the mammography device via the network 40 or via a PACS (Picture Archiving and Communication Systems: medical image management system) connected to the network 40. Alternatively, the ultrasound diagnostic device 20 can also identify the first region of interest of the breast included in the mammography image based on an instruction input by the user to specify the first region of interest of the breast by enclosing it in a rectangular frame.

[0065] Next, the operation of the ultrasound system when capturing an ultrasound image will be described with reference to the flowchart of FIG.

[0066] First, with the ultrasonic probe 1 in contact with the surface skin of the breast of the subject, the transmission / reception circuit 14 starts transmitting ultrasonic waves under the control of the device control unit 36, and a sound ray signal is generated (step S1).

[0067] That is, ultrasonic beams are transmitted from the plurality of transducers of the transducer array 11 to the breast in accordance with the drive signals from the pulser 51 . The ultrasonic echo from the breast based on the ultrasonic beam transmitted from the pulser 51 is received by each transducer of the transducer array 11, and a received signal, which is an analog signal, is output from each transducer of the transducer array 11 that receives the ultrasonic echo. The received signal, which is an analog signal output from each transducer of the transducer array 11, is amplified by the amplifier 52 and AD converted by the AD converter 53 to obtain received data. The beamformer 54 performs reception focus processing on this reception data, thereby generating sound ray signals.

[0068] Subsequently, under the control of the device control unit 36, the image generation unit 31 generates an ultrasound image of the breast (ultrasound image signal) based on the sound ray signals generated by the beamformer 54 of the transmission / reception circuit 14 (step S2).

[0069] That is, the sound ray signals generated by the beam former 54 are subjected to various signal processing by the signal processing unit 16, and image information data representing tomographic image information relating to tissues within the subject is generated. The image information data generated by the signal processing unit 16 is raster converted by the DSC 18, and then subjected to various image processing by the image processing unit 17, thereby generating an ultrasound image (ultrasound image signal). The ultrasound image generated by the image processing unit 17 is stored in the image memory 32.

[0070] Subsequently, under the control of the device control unit 36, the display control unit 33 performs predetermined processing on the ultrasound image held in the image memory 32 and displays it on the monitor 34 (step S3).

[0071] Next, the operation of the ultrasound system when determining whether the regions of interest in the right breast are identical will be described with reference to the flowchart of FIG.

[0072] First, in response to an instruction from a user, the ultrasound diagnostic device 20 causes the device control unit 36 to acquire a mammography image and information on the first region of interest from the mammography device 30 via the network 40, preferably via a PACS on the network 40 (step S20). In this embodiment, the ultrasound diagnostic device 20 acquires four mammography images, consisting of an R_MLO image, an R_CC image, an L_MLO image, and an L_CC image, from the mammography device 30.

[0073] Next, the display control unit 33 displays, for example, the R_MLO image in the upper left display area of the display screen of the monitor 34, as shown in FIG. Next, the display control unit 33 enlarges the first region of interest of the right breast included in the R_MLO image based on the information of the first region of interest in the R_MLO image, and displays it in the window area at the lower right of the upper left display area.

[0074] Similarly, the display control unit 33 displays, for example, an R_CC image in the display area at the bottom left of the display screen of the monitor 34, as shown in FIG. Next, the display control unit 33 enlarges the first region of interest of the right breast included in the R_CC image based on the information of the first region of interest in the R_CC image, and displays it in the window region in the upper right corner of the lower left display region.

[0075] As shown in FIG. 9, the first region of interest of the right breast included in each of the R_MLO image displayed in the upper left display area and the R_CC image displayed in the lower left display area is displayed surrounded by a circle.

[0076] Next, the image generation unit generates an ultrasound image of a first cross section including a second region of interest of the right breast of the subject lying on his / her back, for example, an ultrasound image of a cross section in the lateral (left-right) direction of the subject lying on his / her back (step S23), and the display control unit 33 displays the ultrasound image of the first cross section in the upper right display area of the display screen of the monitor 34. Next, the second region of interest identification unit 44 identifies a second region of interest of the right breast included in the ultrasound image of the first cross section using the first judgment model (step S24), and the display control unit 33 enlarges the second region of interest of the right breast included in the ultrasound image of the first cross section and displays it within a window area in the center left of the upper right display area.

[0077] 9, a schematic diagram of the right breast is displayed in the lower right portion of the upper right display area, indicating the cross-sectional position corresponding to the ultrasound image of the first cross section and the orientation of the ultrasound probe 1. In addition, in the second region of interest of the right breast included in the ultrasound image of the first cross section displayed in the upper right display area, a line indicating, for example, the first diameter and the second diameter perpendicular to the first diameter, and the distance therebetween, are displayed.

[0078] Next, the image generation unit generates an ultrasound image of a second cross section including a second region of interest of the right breast of the subject who is also lying on his back, for example, an ultrasound image of a vertical (up-down) cross section of the subject who is lying on his back (step S25), and the display control unit 33 displays the ultrasound image of the second cross section in the display area at the bottom right of the display screen of monitor 34. Various types of ultrasonic probes are used as the ultrasonic probe 1, such as a 1D probe in which an array of transducers is arranged one-dimensionally, a 2D probe in which an array of transducers is arranged two-dimensionally, or a 1.5D probe. As a method for acquiring ultrasonic images of the first cross section and the second cross section, for example, there is a method for rotating the ultrasonic probe 1 by 90 degrees. As another acquisition method, it is also possible to acquire images using an ultrasonic probe in which an array of transducers is arranged two-dimensionally and which can be imaged without rotation. Next, the second region of interest identification unit 44 uses the first judgment model to identify a second region of interest of the right breast included in the ultrasound image of the second cross section (step S26), and the display control unit 33 enlarges the second region of interest of the right breast included in the ultrasound image of the second cross section and displays it within a window area in the center left of the lower right display area.

[0079] 9, a schematic diagram of the right breast is displayed in the lower right portion of the lower right display area, indicating the cross-sectional position corresponding to the ultrasound image of the second cross section and the orientation of the ultrasound probe 1. In addition, in the second region of interest of the right breast included in the ultrasound image of the second cross section displayed in the lower right display area, a straight line indicating, for example, a third diameter perpendicular to the first diameter and the second diameter, and the distance thereto, are displayed.

[0080] Next, the user presses the determination start button. In response to this, the identity determination unit 46 uses the second determination model to determine whether the first region of interest of the right breast included in the R_CC image identified based on information on the first region of interest in the R_CC image acquired externally, for example, and the second region of interest of the right breast included in the ultrasound image of the first cross section are identical to each other (step S27).

[0081] Then, the display control unit 33 displays the result of the determination by the identity determination unit 46 on the monitor 34 (step S28). For example, the result of the determination by the identity determination unit 46 may be displayed as "Match!". The user can check the determination result displayed on the monitor 34 to see whether the second region of interest of the right breast included in the ultrasound image of the first cross section is the same as the first region of interest of the right breast included in the R_CC image, and can make a diagnosis on the second region of interest of the right breast included in the ultrasound image of the first cross section.

[0082] In this way, the ultrasound system can identify the second region of interest of the breast included in the ultrasound image and automatically determine whether the first region of interest of the breast included in the mammography image and the second region of interest of the breast included in the ultrasound image are identical to each other. This prevents the user from confusing the first region of interest of the breast included in the mammography image with the second region of interest of the breast included in the ultrasound image, and can prevent, for example, performing a biopsy on the wrong region of interest of the breast.

[0083] The ultrasound images of the first and second cross sections may be moving images of the first and second cross sections. In this case, the second region of interest identifying unit 44 uses the first determination model to input the moving images and identifies the second region of interest of the breast included in the moving images.

[0084] Furthermore, the ultrasound image of the first cross section and the enlarged display of the second region of interest of the right breast may be hidden until the second region of interest of the right breast included in the ultrasound image of the second cross section is identified and enlarged and displayed. In other words, after the second region of interest of the right breast included in the ultrasound image of the second cross section is identified, the ultrasound images of the first cross section and the second cross section may be simultaneously displayed on the monitor 34, and the second region of interest of each right breast may be enlarged and displayed on the monitor 34.

[0085] Although an example has been described in which a first region of interest of the right breast included in an R_CC image and a second region of interest of the right breast included in an ultrasound image of a first cross section are determined to be identical, the present invention is not limited to this example, and the identity determination unit 46 may perform the determination based on a radiological image of one direction including the first region of interest, or radiological images of two different directions including the first region of interest. Similarly, the identity determination unit 46 may perform the determination based on an ultrasound image of one cross section including the second region of interest, or ultrasound images of two cross sections that are orthogonal to each other and including the second region of interest. The same applies to the left breast.

[0086] The identity determination unit 46 can perform identity determination using at least one mammography image and one ultrasound image of the same breast. However, as in this embodiment, the R_CC image and the ultrasound image of the first cross section are preferable because they are observed in the same direction. On the other hand, performing identity determination using mammography images from two directions and ultrasound images from two cross sections can further improve the accuracy of identity determination by the identity determination unit 46. Furthermore, by viewing the mammography images from two directions and the ultrasound images from two cross sections, the user can easily predict the three-dimensional images of the first and second regions of interest, which in turn makes it easier to predict the results of identity determination.

[0087] Furthermore, the identity determination unit 46 may perform the determination based on volume data consisting of ultrasound images of multiple different cross sections including the second region of interest of the breast, or a three-dimensional image reconstructed using the volume data, thereby further improving the accuracy of the identity determination by the identity determination unit 46.

[0088] In this embodiment, the identity determination unit 46 starts the determination based on an instruction to start the determination input by the user, such as the user pressing a determination start button, but is not limited to this. For example, the identity determination unit 46 may start the determination based on an instruction to freeze the ultrasound image input by the user, which is unrelated to the instruction to start the determination, or may start the determination based on an instruction to specify a second region of interest of the breast by enclosing it in a rectangular frame, input by the user, after the ultrasound image is generated based on an instruction to freeze the ultrasound image input by the user.

[0089] Furthermore, when it is determined that a plurality of first regions of interest are included in a radiological image or a plurality of second regions of interest are included in an ultrasound image, i.e., when there is a possibility of multiple lesions, the identity determination unit 46 may automatically start the determination. In this case, the identity determination may be performed on one breast region of interest selected from the plurality of regions of interest based on an instruction from the user to select a breast region of interest.

[0090] In actual clinical practice, when a breast biopsy is performed while viewing an ultrasound image in cases where there is a possibility of multiple lesions, there have been cases where a biopsy was performed on the wrong breast region of interest due to a misidentification of the first breast region of interest identified in the mammography image with the second breast region of interest identified in the ultrasound image. In contrast, if there were a function to determine whether multiple regions of interest are the same as those identified above, it would be possible to reduce the number of misidentifications of regions of interest when a biopsy is performed, thereby shortening the time required for the biopsy.

[0091] Furthermore, the ultrasound diagnostic apparatus 20 may acquire mammography images from a remote computer rather than directly from the mammography apparatus 30 .

[0092] Figure 10 is a block diagram of an embodiment showing a configuration in which an ultrasound diagnostic apparatus acquires mammography images from a remote computer. As shown in Figure 10, a mammography apparatus 30 is connected to each of an ultrasound diagnostic apparatus 20 and a remote computer 70 via a network 40. The ultrasound diagnostic apparatus 20 and the remote computer 70 are also connected to each other via the network 40, which allows data to be exchanged in both directions.

[0093] Here, the remote computer 70 is a workstation or PACS connected to the ultrasound diagnostic apparatus 20 and the mammography apparatus 30 via the network 40 .

[0094] 10 , the remote computer 70 receives, stores, and manages mammography images transmitted from the mammography apparatus 30 via the network 40 and ultrasound images transmitted from the ultrasound diagnostic apparatus 20 via the network 40. The ultrasound diagnostic apparatus 20 can acquire mammography images directly from the mammography apparatus 30 via the network 40, or can acquire mammography images from the remote computer 70 via the network 40.

[0095] Furthermore, the identity determination may be performed in the ultrasound diagnostic apparatus 20, or may be performed in the remote computer 70. When the identity determination is performed in the ultrasound diagnostic apparatus 20, the remote computer 70 is not necessary, and the ultrasound diagnostic apparatus 20 includes all of the components shown in FIG. 1 , such as the ultrasound probe 1, the image generation unit, and the region of interest processing unit 35. On the other hand, when the identity determination is performed in the remote computer 70, the remote computer 70 includes the identity determination unit, and the ultrasound diagnostic apparatus 20 includes components other than the identity determination unit, such as the ultrasound probe 1, the image generation unit, and the second region of interest identification unit 44. Note that the second region of interest identification unit 44 may be included in the ultrasound diagnostic apparatus 20 or the remote computer 70.

[0096] Fig. 11 is a block diagram showing the configuration of a remote computer according to one embodiment. The remote computer 70 shown in Fig. 11 includes an identity determination unit 73, an image storage unit 72, and a remote computer control unit 75. The identity determination unit 73 is bidirectionally connected to the image storage unit 72. The identity determination unit 73 and the image storage unit 72 are connected to the remote computer control unit 75. Note that remote computer 70 may include not only identity determination unit 73 but also a region of interest processing unit similar to region of interest processing unit 35 included in ultrasound diagnostic apparatus 20. In other words, remote computer 70 may include a second region of interest identification unit similar to second region of interest identification unit 44.

[0097] The identity determination unit 73 is the same as the identity determination unit 46 provided in the main body 3 of the ultrasonic diagnostic apparatus 20 , but operates under the control of the remote computer control unit 75 .

[0098] The image storage unit 72 stores medical images such as mammography images and ultrasound images. The image storage unit 72 may be, for example, a storage device that stores medical images in the remote computer 70. Alternatively, instead of the image storage unit 72, an image storage unit on a PACS connected to the network 40 may be used.

[0099] The remote computer control unit 75 controls each unit of the remote computer 70 based on pre-stored programs, etc. More specifically, the remote computer control unit 75 controls the identity determination unit 73 so that identity determination is performed. The remote computer control unit 75 also controls the image storage unit 72 so that medical images such as mammography images and ultrasound images are stored.

[0100] When the identity determination is performed in the remote computer 70, the identity determination unit 73 acquires the mammography image and ultrasound image to be determined from, for example, medical images stored in the image storage unit 72. The operation of the identity determination unit 73 is the same as the operation when the identity determination unit 44 performs identity determination in the ultrasound diagnostic device 20. After the identity determination is performed in the remote computer 70, the result of the identity determination is transmitted from the remote computer 70 to the ultrasound diagnostic device 20. Then, in the ultrasonic diagnostic apparatus 20, the display control unit 33 displays the result of the identity determination on the monitor .

[0101] Although the remote computer 70 performs the identity determination, transmits the identity determination result to the ultrasound diagnostic apparatus 20, and causes the display control unit of the ultrasound diagnostic apparatus 20 to display the determination result on the monitor 34 of the ultrasound diagnostic apparatus 20, the present invention is not limited to this. For example, if the remote computer 70 is a workstation, the identity determination may be performed in the workstation, and a display control unit included in the workstation may display the identity determination result on a monitor included in the workstation. Furthermore, if the workstation includes a client connected via the network 40, the identity determination may be performed in the workstation, transmits the identity determination result from the workstation to the client, and causes a display control unit included in the client to display the identity determination result on a monitor included in the client. Furthermore, if the remote computer 70 is a PACS, the identity determination may be performed in the PACS, and a display control unit included in the PACS may display the identity determination result on a monitor included in the PACS. Alternatively, the ultrasonic diagnostic device 20 may perform the same determination, and the display control unit of the ultrasonic diagnostic device 20 may display the result of the determination on the monitor 34 of the ultrasonic diagnostic device 20, or the result of the same determination may be transmitted to the remote computer 70, and the display control unit of the remote computer 70 may display the result of the determination on the monitor 34 of the remote computer 70.

[0102] If the remote computer 70 is a workstation, a display control unit included in the workstation may cause a radiographic image and an ultrasound image of the same subject to be displayed side by side on a monitor included in the workstation. If the workstation includes a client connected via the network 40, radiographic images and ultrasound images of the same subject may be transmitted from the workstation to the client, and a display control unit included in the client may cause a radiographic image and an ultrasound image of the same subject to be displayed side by side on a monitor included in the client. If the remote computer 70 is a PACS, a display control unit included in the PACS may cause a radiographic image and an ultrasound image of the same subject to be displayed side by side on a monitor included in the PACS. In this way, regardless of whether the identity determination is performed by the ultrasound diagnostic apparatus 20 or the remote computer 70, a display control unit in at least one of the ultrasound diagnostic apparatus 20 and the remote computer 70 can cause a radiographic image and an ultrasound image of the same subject to be displayed side by side on the monitor 34.

[0103] The present invention is not limited to mammography images generated by the mammography device 30, but can utilize radiation images generated by various types of radiation diagnostic devices, including CT devices and MRI devices.

[0104] Furthermore, the present invention is not limited to stationary ultrasound systems, but can also be applied to portable ultrasound systems in which the device main body is realized by a laptop-type terminal device, and handheld ultrasound systems in which the device main body is realized by a handheld terminal device such as a smartphone or a tablet PC (Personal Computer).

[0105] In the device of the present invention, the hardware configuration of the processing units that perform various processes, such as the transmitter / receiver circuit 14, signal processing unit 16, image generation unit 31, display control unit 33, region of interest processing unit 35, and device control unit 36, may be dedicated hardware or various processors or computers that execute programs. Furthermore, the hardware configuration of the image memory 32 and image storage unit 72 may be dedicated hardware or may be a memory such as a semiconductor memory and a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive).

[0106] Various types of processors include CPUs (Central Processing Units), which are general-purpose processors that execute software (programs) and function as various processing units, programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as FPGAs (Field Programmable Gate Arrays), and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations designed specifically for performing specific processes.

[0107] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types, for example, a combination of multiple FPGAs, or a combination of an FPGA and a CPU, etc. Also, multiple processing units may be configured with one of the various processors, or two or more of the multiple processing units may be combined into one processor.

[0108] For example, as typified by server and client computers, one processor is configured by combining one or more CPUs and software, and this processor functions as multiple processing units. Another form is the use of a processor that realizes the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by system-on-chip (SoC).

[0109] Furthermore, the hardware configuration of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0110] The method of the present invention can be implemented, for example, by a program that causes a computer to execute each step. Also, a computer-readable recording medium on which this program is recorded can be provided.

[0111] Although the present invention has been described in detail above, the present invention is not limited to the above-described embodiments, and various improvements and modifications may be made without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0112] 1 Ultrasound probe, 3 Device main body, 11 Transducer array, 14 Transmitting and receiving circuit, 16 Signal processing unit, 17 Image processing unit, 18 DSC, 20 Ultrasound diagnostic device, 30 Mammography device, 32 Image memory, 33 Display control unit, 34 Monitor, 35 Region of interest processing unit, 36 Device control unit, 37 Input device, 39 Processor, 40 Network, 44 Second region of interest identification unit, 46, 73 Identity determination unit, 51 Pulser, 52 Amplification unit, 53 AD conversion unit, 54 Beamformer, 60 X-ray source, 62 X-ray detector, 70 Remote computer, 72 Image storage unit, 75 Remote computer control unit.

Claims

1. an ultrasound probe; an image generating unit that generates an ultrasound image including a second region of interest of the subject's breast corresponding to a first region of interest of the subject's breast included in the radiation image, from a received signal obtained by transmitting and receiving an ultrasound beam to and from the subject's breast using the ultrasound probe; a second region of interest identifying unit that identifies the second region of interest included in the ultrasound image; an identity determination unit that determines whether the first region of interest and the second region of interest included in the radiation image are identical to each other; an input device for receiving instructions input by a user; Equipped with The ultrasound system, wherein the identity determination unit starts determination based on an instruction to freeze the ultrasound image input by the user.

2. Equipped with an ultrasound diagnostic device, The ultrasound system according to claim 1 , wherein the ultrasound diagnostic apparatus comprises the ultrasound probe, the image generating unit, the second region of interest specifying unit, and the identity determining unit.

3. an ultrasound diagnostic device; a remote computer connected to the ultrasound diagnostic device via a network, the ultrasonic diagnostic apparatus includes the ultrasonic probe and the image generating unit, The ultrasound system of claim 1 , wherein the remote computer includes the identity determination unit.

4. the ultrasound diagnostic apparatus includes a monitor and a display control unit; The ultrasound system according to claim 2 , wherein the display control unit causes the monitor to display the radiographic image and the ultrasound image side by side.

5. the remote computer is a workstation; A monitor and a display control unit are provided, The ultrasound system according to claim 3 , wherein the display control unit causes the monitor to display the radiographic image and the ultrasound image side by side.

6. the remote computer is a workstation; a client connected to the workstation via the network; the client includes a monitor and a display control unit; The ultrasound system according to claim 3 , wherein the display control unit causes the monitor to display the radiographic image and the ultrasound image side by side.

7. the remote computer is a medical image management system; A medical image management system includes a monitor and a display control unit, The ultrasound system according to claim 3 , wherein the display control unit causes the monitor to display the radiographic image and the ultrasound image side by side.

8. The ultrasound system according to claim 4 , wherein the display control unit enlarges and displays the first region of interest included in the radiation image and the second region of interest included in the ultrasound image on the monitor.

9. The ultrasound system according to claim 4 , wherein the display control unit causes the monitor to display the result of the determination made by the identity determination unit.

10. The ultrasound system according to claim 1 , wherein the identity determination unit makes the determination based on a radiographic image from one direction including the first region of interest, or based on radiographic images from two different directions including the first region of interest.

11. 11. The ultrasound system according to claim 1, wherein the identity determination unit makes the determination based on an ultrasound image of one cross section including the second region of interest, or ultrasound images of two cross sections that are orthogonal to each other and that include the second region of interest.

12. 11. The ultrasound system according to claim 1, wherein the identity determination unit makes the determination based on volume data consisting of ultrasound images of a plurality of different cross sections including the second region of interest, or a three-dimensional image reconstructed using the volume data.

13. The ultrasound system according to claim 1 , wherein the identity determination unit identifies the second region of interest included in the ultrasound image based on an instruction input by the user to designate the second region of interest.

14. the second region of interest identification unit has a first determination model that uses a training ultrasound image of a subject's breast as first training data and learns a relationship between the training ultrasound image and a region of interest included in the training ultrasound image for a plurality of the first training data; The ultrasound system of claim 1 , wherein the first determination model uses the ultrasound image as an input to identify the second region of interest included in the ultrasound image.

15. the ultrasound image is a moving image, The ultrasound system of claim 14 , wherein the first determination model uses the video as an input to identify the second region of interest included in the video.

16. the identity determination unit has a second determination model that uses a training radiographic image including a region of interest of a subject's breast and a training ultrasound image including a region of interest of the subject's breast that is identical to the region of interest included in the training radiographic image as a set of second teacher data, and that has learned a relationship between the training radiographic image and the training ultrasound image, and whether or not the region of interest included in the training radiographic image and the region of interest included in the training ultrasound image are identical for a plurality of the second teacher data; 16. The ultrasound system of claim 14, wherein the second determination model receives the radiographic image and the ultrasound image as a set of inputs and outputs a determination result of whether the first region of interest included in the radiographic image and the second region of interest included in the ultrasound image are identical.

17. 2. The ultrasound system according to claim 1, wherein the identity determination unit starts determination based on an instruction to specify the second region of interest input by the user after the ultrasound image is generated based on an instruction to freeze the ultrasound image input by the user.

18. The ultrasound system according to claim 1 , wherein the identity determination unit identifies the second region of interest included in the radiographic image based on information of the first region of interest of the radiographic image acquired from an external device.

19. generating an ultrasound image including a second region of interest of the subject's breast corresponding to a first region of interest of the subject's breast included in the radiation image from a received signal obtained by transmitting and receiving an ultrasound beam to and from the subject's breast using an ultrasound probe; identifying the second region of interest included in the ultrasound image; A control method for an ultrasound system, which starts determining whether the first region of interest and the second region of interest included in the radiation image are identical to each other based on an instruction to freeze the ultrasound image input by a user.

Images