Ultrasound diagnostic equipment

Abstract

This aims to eliminate the need to re-select the desired image when using optical images as a substitute for body markers. [Solution] In the ultrasound diagnostic apparatus of the present invention, an ultrasound image generation unit generates an ultrasound image including the area to be examined, and an optical camera generates an optical image including the subject in contact with the ultrasound probe. A display control unit displays the ultrasound image and optical image on a monitor, and a first image memory stores ultrasound images and optical images generated over a certain period of time prior to a timing specified by the user. An image selection unit selects one frame of ultrasound image and one frame of optical image from the ultrasound images and optical images stored in the first image memory, and a second image memory stores the one frame of ultrasound image and the one frame of optical image in association.

Description

Technical Field

[0001] The present invention relates to an ultrasonic diagnostic apparatus that generates an ultrasonic image including an examination site of a subject.

Background Art

[0002] When performing an examination of a subject using an ultrasonic diagnostic apparatus, for example, for later diagnosis, it is important to record in association an ultrasonic image including the examination site and a body mark for specifying the position and orientation of the ultrasonic probe at the time of generating the ultrasonic image. Conventionally, when applying a body mark to an ultrasonic image, for each ultrasonic image, it is necessary to manually apply a probe mark representing the position and orientation of the ultrasonic probe at the time of generating the ultrasonic image in the body mark to be applied to the ultrasonic image, which is very laborious. In contrast, Patent Documents 1, 2, etc. propose to store in association an ultrasonic image and an optical image (moving image) of the subject at the time of generating this ultrasonic image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Documents 1 and 2, an optical image can be referred to instead of a body mark. However, simply storing an ultrasonic image and an optical image in association as in Patent Documents 1 and 2 has a problem that, for example, at the time of later diagnosis, every time the image at the time of examination is viewed again, it takes time to reselect a desired one-frame ultrasonic image and one-frame optical image from the stored ultrasonic image and optical image in association.

[0005] The objective of the present invention is to provide an ultrasound diagnostic device that eliminates the need to re-select a desired image when using optical images instead of body marks. [Means for solving the problem]

[0006] To achieve the above objective, the present invention comprises an ultrasonic probe and a device body connected to the ultrasonic probe. The main body of the device is, An ultrasonic image generation unit generates an ultrasonic image including the examination area from the received signal obtained by transmitting and receiving an ultrasonic beam to the examination area of ​​the subject using an ultrasonic probe, An optical camera that generates an optical image including the subject with the ultrasound probe in contact with it, Monitor and, A display control unit that displays ultrasound images and optical images on a monitor, A first image memory that stores ultrasound and optical images generated over a certain period of time prior to a time specified by the user, An image selection unit selects one ultrasound image and one optical image from among the ultrasound images and optical images stored in the first image memory. The present invention provides an ultrasound diagnostic apparatus having a second image memory that stores one frame of ultrasound image and one frame of optical image in association with each other.

[0007] In this case, it is preferable that the display control unit, in response to instructions from the user, scrolls back from the ultrasound images stored in the first image memory to display a past ultrasound image specified by the user on the monitor, and scrolls back from the optical images stored in the first image memory to display a past optical image specified by the user on the monitor.

[0008] Furthermore, it is preferable that the image selection unit selects one frame of ultrasound image and one frame of optical image based on user specifications.

[0009] Furthermore, it is preferable that the image selection unit automatically selects at least one of one frame of ultrasound image and one frame of optical image.

[0010] Furthermore, it is preferable that the image selection unit automatically selects both a single-frame ultrasound image and a single-frame optical image.

[0011] Furthermore, it is preferable that the image selection unit automatically selects one of the images from a single-frame ultrasound image and one of the images from a single-frame optical image, and then automatically selects the other image from the ultrasound images and optical images stored in the first image memory, from among the ultrasound images and optical images generated within a certain period before and after the generation time of the other image.

[0012] Furthermore, it is preferable that the image selection unit selects one of the two images—a single-frame ultrasound image and a single-frame optical image—based on user specification, and automatically selects the other image from the two.

[0013] Furthermore, it is preferable that the image selection unit automatically selects the other image from among the ultrasound images and optical images stored in the first image memory, from among the ultrasound images and optical images generated within a certain period before and after the generation time of the other image.

[0014] Furthermore, it is preferable that the second image memory stores the processed ultrasound image and optical image in association with each other if the user has processed at least one of the two frames of ultrasound images and optical images.

[0015] Furthermore, the present invention relates to a control method for an ultrasound diagnostic apparatus comprising an ultrasound probe and a device body connected to the ultrasound probe, A step of generating an ultrasonic image including an inspection location from a received signal obtained by transmitting and receiving an ultrasonic beam to an inspection location of a subject using an ultrasonic probe by an ultrasonic image generation unit included in the apparatus main body; A step of generating an optical image including the subject in a state where the ultrasonic probe is in contact by an optical camera included in the apparatus main body; A step of causing a display control unit included in the apparatus main body to display the ultrasonic image and the optical image on a monitor included in the apparatus main body; A step of storing ultrasonic images and optical images generated during a certain period in the past from a timing specified by a user in a first image memory included in the apparatus main body; A step of selecting one frame of ultrasonic image and one frame of optical image from the ultrasonic images and optical images stored in the first image memory by an image selection unit included in the apparatus main body; A step of storing one frame of ultrasonic image and one frame of optical image in association with each other in a second image memory included in the apparatus main body, and providing a control method for an ultrasonic diagnostic apparatus.

Effect of the Invention

[0016] In the present invention, one frame of ultrasonic image and one frame of optical image are stored in association with each other. Thereby, according to the present invention, when the user, for example, views the images at the time of inspection again during a later diagnosis, the user can save the trouble of reselecting a desired one frame of ultrasonic image and one frame of optical image from the ultrasonic images and optical images stored in the first image memory. Further, by referring to one frame of optical image, the user can easily grasp the position and orientation of the ultrasonic probe 1 at the time of generating one frame of ultrasonic image associated with this one frame of optical image.

Brief Description of the Drawings

[0017] [Figure 1] It is a block diagram of an embodiment showing the configuration of an ultrasonic diagnostic apparatus. [Figure 2] It is a block diagram of an embodiment showing the configuration of a transmission / reception circuit. [Figure 3] It is a block diagram of an embodiment showing the configuration of an ultrasonic image generation unit. [Figure 4] It is a flowchart of an embodiment showing the operation of an ultrasonic diagnostic apparatus. [Figure 5] It is a conceptual diagram of an embodiment showing the display screen of a monitor during examination of a subject. [Figure 6] It is a conceptual diagram of another embodiment showing the display screen of a monitor after a freeze button is pressed. [Figure 7A] It is a conceptual diagram of an embodiment of an optical image of a subject being examined on the abdomen being imaged. [Figure 7B] It is a conceptual diagram of another embodiment of an optical image of a subject being examined on the abdomen being imaged.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, based on the preferred embodiments shown in the accompanying drawings, an ultrasonic diagnostic apparatus and a control method for the ultrasonic diagnostic apparatus of the present invention will be described in detail.

[0019] FIG. 1 is a block diagram of an embodiment showing the configuration of an ultrasonic diagnostic apparatus of the present invention. The ultrasonic diagnostic apparatus shown in FIG. 1 is a hand-held ultrasonic diagnostic apparatus, and includes an ultrasonic probe 1 and a device main body 3 connected to the ultrasonic probe 1. The ultrasonic diagnostic apparatus of the present embodiment is realized by the ultrasonic probe 1, the device main body 3, and an ultrasonic diagnostic application program operating on the device main body 3.

[0020] The ultrasonic probe 1 scans an examination site of a subject with an ultrasonic beam and outputs a beam signal corresponding to an ultrasonic image of the examination site. As shown in FIG. 1, the ultrasonic probe 1 includes a vibrator array 11, a transmission / reception circuit 13, and a battery 15. The vibrator array 11 and the transmission / reception circuit 13 are connected bidirectionally, and a device control unit 47 of the device main body 3 described later is connected to the transmission / reception circuit 13. Further, the ultrasonic probe 1 incorporates a battery 15.

[0021] The transducer array 11 has a plurality of ultrasonic transducers arranged in one or two dimensions. Each of these transducers transmits ultrasound according to a drive signal supplied from the transmit / receive circuit 13 and receives reflected waves from the subject to output an analog received signal. Each oscillator is constructed using an element in which electrodes are formed at both ends of a piezoelectric body, such as a piezoelectric ceramic represented by PZT (Lead Zirconate Titanate), a polymer piezoelectric element represented by PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal represented by PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0022] The transmitting / receiving circuit 13, under the control of the device control unit 47, causes the transducer array 11 to transmit an ultrasonic beam and generates an ultrasonic signal by performing a reception focus process on the received signal output from the transducer array 11 that has received the ultrasonic echo. As shown in Figure 2, the transmitting / receiving circuit 13 includes a pulser 51 connected to the transducer array 11, and an amplifier 53, an AD (Analog Digital) converter 55, and a beamformer 57 that are sequentially connected in series from the transducer array 11.

[0023] The pulser 51 includes, for example, multiple pulse generators and performs a transmit focus process to supply drive signals to multiple transducers of the transducer array 11, adjusting the delay amount, so that the ultrasonic waves transmitted from the transducers form an ultrasonic beam, based on a transmit delay pattern selected by the device control unit 47. Through this transmit focus process, when a pulsed or continuous wave voltage is applied to the electrodes of the transducers of the transducer array 11, the piezoelectric material expands and contracts, generating pulsed or continuous wave ultrasonic waves from each transducer, and an ultrasonic beam is formed from the combined wave of these ultrasonic waves.

[0024] The transmitted ultrasonic beam is reflected by an object, such as a part of the subject, and propagates toward the transducer array 11 of the ultrasonic probe 1. Each transducer constituting the transducer array 11 expands and contracts upon receiving the ultrasonic echo propagating toward the transducer array 11, generating a received signal which is an electrical signal, and outputs these received signals to the amplification unit 53.

[0025] The amplification unit 53 amplifies the signals input from each oscillator constituting the oscillator array 11 and transmits the amplified signals to the AD conversion unit 55. The AD conversion unit 55 converts the analog signals transmitted from the amplification unit 53 into digital received data and outputs this received data to the beamformer 57.

[0026] The beamformer 57 performs a reception focus process by adding each received data converted by the AD converter 55 with a corresponding delay, according to the sound velocity or sound velocity distribution set based on the reception delay pattern selected by the device control unit 47. This reception focus process generates a sound ray signal in which each received data converted by the AD converter 55 is added in phase and the focus of the ultrasonic echo is narrowed.

[0027] The battery 15 is built into the ultrasonic probe 1 and supplies power to each circuit of the ultrasonic probe 1.

[0028] Next, the main unit 3 generates an ultrasound image including the area to be examined on the subject based on the sound signal generated by the ultrasound probe 1, and displays this ultrasound image including the area to be examined on the subject. The main unit 3 is a handheld terminal device such as a smartphone or tablet PC (Personal Computer), and as shown in Figure 1, it comprises an ultrasound image generation unit 31, an optical camera 33, a first image memory 35, an image selection unit 37, a second image memory 39, a monitor 41, a display control unit 43, an input device 45, and a device control unit 47.

[0029] The ultrasound image generation unit 31 is connected to the transmitting / receiving circuit 13 of the ultrasound probe 1, and the display control unit 43 and monitor 41 are sequentially connected to the ultrasound image generation unit 31. The first image memory 35 is connected to both the ultrasound image generation unit 31 and the display control unit 43. Furthermore, the first image memory 35 and image selection unit 37 are sequentially connected to the optical camera 33, and the second image memory 39 and display control unit 43 are sequentially connected to the image selection unit 37. The device control unit 47 is connected to the transmitting / receiving circuit 13, ultrasound image generation unit 31, optical camera 33, display control unit 43, first image memory 35, image selection unit 37, and second image memory 39, and the device control unit 47 is connected to the input device 45.

[0030] The ultrasound probe 1 and the main unit 3 are connected wirelessly via Wi-Fi (Wireless Fidelity) or via a wired connection using a USB (Universal Serial Bus) cable.

[0031] The ultrasonic image generation unit 31 generates an ultrasonic image (ultrasonic image signal) including the area being examined of the subject from the received signal obtained by transmitting and receiving an ultrasonic beam to the area being examined of the subject using the ultrasonic probe 1 (more precisely, the transducer array 11) under the control of the device control unit 47, or more precisely, from the sound line signal generated from the received signal by the transmitting and receiving circuit 13. As shown in Figure 3, the ultrasonic image generation unit 31 has a configuration in which the signal processing unit 21, DSC 23 and image processing unit 25 are connected in series in sequence.

[0032] The signal processing unit 21 generates image information data corresponding to an ultrasound image based on the sound line signal generated by the transmitting / receiving circuit 13. More specifically, the signal processing unit 21 performs signal processing on the sound line signal generated by the beamformer 57 of the transmitting / receiving circuit 13, for example, correcting for attenuation due to propagation distance according to the depth of the position where the ultrasound is reflected, and then performs envelope detection processing to generate image information data representing tomographic image information about the tissue within the subject.

[0033] The DSC (Digital Scan Converter) 23 converts the image information data generated by the signal processing unit 21 into a raster image signal that follows the scanning method of a normal television signal.

[0034] The image processing unit 25 generates an ultrasound image by applying various image processing functions to the image signal input from the DSC 23, such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction, in accordance with the display format of the monitor 41, and outputs the processed ultrasound image to the first image memory 35 and the display control unit 43.

[0035] The optical camera 33, under the control of the device control unit 47, uses the ultrasonic probe 1 to image the subject being examined at the examination site, thereby generating an optical image (camera image) that includes the subject in contact with the ultrasonic probe 1.

[0036] The first image memory 35, under the control of the device control unit 47, stores a series of multiple frames of ultrasound images (moving images) generated by the ultrasound image generation unit 31 and a series of multiple frames of optical images (moving images) generated by the optical camera 33 for each examination. In the first image memory 35, ultrasound images (still images) and optical images (still images) are stored independently and sequentially until the memory capacity of the first image memory 35 is full. It is not necessary to match the generation time (timestamp) when saving the ultrasound images and optical images. In other words, it is not necessary to save ultrasound images and optical images from frames with the same generation time as a set. Subsequently, the ultrasound images and optical images from the most recent frames are sequentially overwritten in the first image memory 35, respectively. As a result, the first image memory 35 stores ultrasound images and optical images generated over a certain period prior to the most recent frame.

[0037] The image selection unit 37, under the control of the device control unit 47, selects one ultrasound image and one optical image from the ultrasound images and optical images stored in the first image memory 35. Details on how to select one frame of ultrasound image and one frame of optical image will be described later.

[0038] The second image memory 39 stores, under the control of the device control unit 47, one frame of ultrasound image and one frame of optical image selected by the image selection unit 37 in association with each other.

[0039] The display control unit 43, under the control of the device control unit 47, causes various information to be displayed on the monitor (display unit) 41. The monitor 41 displays, for example, ultrasound images and optical images, as well as an operation screen for selecting one frame of ultrasound image and one frame of optical image. The monitor 41 is not particularly limited, but examples include LCD (Liquid Crystal Display) and organic EL (Electro-Luminescence) displays. The monitor 41 is located on one side of the main body 3 of the device. In contrast, the optical camera 33 is located on the other side of the main body 3, that is, the side opposite to the side on which the monitor 41 is located.

[0040] The input device 45 receives various instructions from the user (examiner) of the ultrasound diagnostic device. The input device 45 is not particularly limited, but may include, for example, various buttons and a touch panel on which the user inputs various instructions by touch operation.

[0041] The device control unit 47 controls the ultrasonic probe 1 and the main body of the device 3 based on a pre-stored program and user instructions input from the input device 45.

[0042] The ultrasonic image generation unit 31, image selection unit 37, display control unit 43, and device control unit 47 are all composed of a processor 49.

[0043] Next, we will explain the operation of the ultrasound diagnostic device while referring to the flowchart in Figure 4.

[0044] When performing an examination on a subject, the user first brings the ultrasound probe 1 into contact with the area to be examined on the subject to begin transmitting and receiving ultrasound waves, and uses the optical camera 33 to image the examination area and the subject, including the ultrasound probe 1, during the examination.

[0045] In this case, the transmitting and receiving circuit 13, under the control of the device control unit 47, transmits and receives ultrasound while the ultrasound probe 1 is in contact with the examination site of the subject, and an acoustic signal is generated (step S1).

[0046] In other words, according to the drive signal from the pulser 51, an ultrasonic beam is transmitted from multiple transducers of the transducer array 11 to the area of ​​the subject being examined. The ultrasonic echo from the inspection site, based on the ultrasonic beam transmitted from the pulsar 51, is received by each transducer of the transducer array 11, and an analog received signal is output from each transducer of the transducer array 11 that has received the ultrasonic echo. The received signals output from each oscillator of the oscillator array 11 are amplified by the amplification unit 53 and converted to AD by the AD conversion unit 55 to obtain the received data. The beamformer 57 performs reception focus processing on this received data, thereby generating a sound ray signal.

[0047] Next, the ultrasound image generation unit 31, under the control of the device control unit 47, generates an ultrasound image including the area to be examined of the subject based on the sound line signal generated by the beamformer 57 of the transmitting / receiving circuit 13 (step S2).

[0048] In other words, the sound line signals generated by the beamformer 57 are subjected to various signal processing by the signal processing unit 21 to generate image information data representing tomographic image information about the tissue within the subject. The image information data generated by the signal processing unit 21 is converted to a raster by the DSC 23, and then various image processing is applied by the image processing unit 25 to sequentially generate ultrasound images (moving images).

[0049] Meanwhile, the optical camera 33, under the control of the device control unit 47, captures images of the subject being examined using the ultrasonic probe 1, and optical images (moving images) including the subject in contact with the ultrasonic probe 1 are sequentially generated (step S3). In other words, an ultrasound image of the area being examined on the subject and an optical image of the subject, including the ultrasound probe 1 being examined on the area being examined, are generated simultaneously.

[0050] The ultrasonic images generated by the ultrasonic image generation unit 31 and the optical images generated by the optical camera 33 are sequentially stored in the first image memory 35 under the control of the device control unit 47.

[0051] Furthermore, the display control unit 43, under the control of the device control unit 47, sequentially displays the ultrasonic image generated by the ultrasonic image generation unit 31 and the optical image generated by the optical camera 33 on the monitor 41, as shown in Figure 5 (step S4). Therefore, the user can generate an ultrasound image including the area being examined of the subject using the ultrasound probe 1 while viewing the ultrasound image and optical image displayed on the display screen of the monitor 41 located on one side of the main body of the device 3, and at the same time generate an optical image including the subject in contact with the area being examined using the optical camera 33 located on the other side of the main body of the device 3.

[0052] Figure 5 is a conceptual diagram of one embodiment showing the display screen of a monitor during examination of a subject. In the upper area of ​​the display screen of the monitor 41 shown in Figure 5, an ultrasound image 61 is displayed, and in the area from the lower center to the right edge below the ultrasound image 61, an optical image 63 is displayed. In the area below the optical image 63, a first operation screen 65 is displayed. A freeze button 67 is located in the center of this first operation screen.

[0053] Here, the user, while viewing the ultrasound and optical images sequentially displayed on the monitor 41 of the device body 3, presses the freeze button 67 at the moment when the user believes that a desired ultrasound image including the area being examined of the subject has been generated (step S5).

[0054] When the freeze button 67 is pressed, under the control of the device control unit 47, the ultrasound and optical images generated during a certain period prior to the time the freeze was specified by the user are stored in the first image memory 35 (step S6). In other words, the ultrasound and optical images generated during a certain period prior to the time the freeze was specified by the user, which are already stored in the first image memory 35, are retained in the first image memory 35.

[0055] Furthermore, when the freeze button 67 is pressed, the display control unit 43 displays the second operation screen 71 on the monitor 41, as shown in Figure 6, instead of the first operation screen 65 shown in Figure 5.

[0056] Figure 6 is a conceptual diagram of another embodiment showing the monitor display screen after the freeze button has been pressed. Figure 6 shows the display screen of the monitor 41 shown in Figure 5, but with the second operation screen 71 displayed instead of the first operation screen 65. In the second operation screen 71, a slider bar 73 for ultrasound images is located at the top, and a slider bar 75 for optical images (camera images) is located at the bottom, from the left end to the center. A save button 77 is located on the right side of the second operation screen 71.

[0057] Next, the user uses the slide bar 73 for ultrasound images and the slide bar 75 for optical images to select a desired ultrasound image and an optical image from the ultrasound images and optical images stored in the first image memory 35.

[0058] The left and right directions of the ultrasound image slide bar 73 represent the time axis, with the position of its right end corresponding to the generation time of the most recent frame of ultrasound image stored in the first image memory 35, and the position of its left end corresponding to the generation time of the oldest frame of ultrasound image.

[0059] The user can scroll back through the ultrasound images on the monitor 41, displaying the most recent frame and then earlier frames, by moving a triangular knob from the rightmost position to the left of the ultrasound image slide bar 73. Furthermore, by stopping the movement of the knob, the user can select an ultrasound image frame from those stored in the first image memory 35 that corresponds to the generation time at the position where the knob stopped moving. The operation of the slide bar 75 for optical imaging is similar.

[0060] In other words, when the user moves the knob of the slide bar 73 for ultrasound images left or right, the display control unit 43 scrolls back from the ultrasound images stored in the first image memory 35 to the previous ultrasound image specified by the user's movement of the knob, and displays it on the monitor 41. Furthermore, when the user stops moving the knob at a desired position, an ultrasound image frame corresponding to the generation time of the position where the knob stopped is specified (step S7).

[0061] Furthermore, when the user moves the knob of the optical image slider bar 75 left or right, the display control unit 43 scrolls back from the optical images stored in the first image memory 35 to the previous optical image specified by the user's movement of the knob, and displays it on the monitor 41. Also, when the user stops moving the knob at a desired position, one frame of optical image corresponding to the generation time of the position where the knob stopped is specified (step S8).

[0062] The user specifies the desired one-frame ultrasound image and one-frame optical image, and then presses the save button 77.

[0063] Accordingly, the image selection unit 37, under the control of the device control unit 47, selects one frame of ultrasound image and one frame of optical image from among the ultrasound images and optical images stored in the first image memory 35, based on the user's specification (step S9).

[0064] Next, under the control of the device control unit 47, the ultrasound image frame and optical image frame selected by the image selection unit 37 are associated and stored in the second image memory 39 (step S10).

[0065] In the ultrasound diagnostic apparatus of this embodiment, one frame of ultrasound image and one frame of optical image are stored in association with each other. This eliminates the need for the user to re-select the desired ultrasound image and optical image from those stored in the first image memory 35 when reviewing the images from the examination during a later diagnosis. Furthermore, by referring to the optical image, the user can easily determine the position and orientation of the ultrasound probe 1 at the time of generation of the ultrasound image associated with that optical image.

[0066] Furthermore, the user can perform various processing operations on at least one of the ultrasound image and optical image frames specified by the user.

[0067] In this case, the user can enter the ultrasound image processing mode and perform various processing operations on the ultrasound image 61 by, for example, pressing the ultrasound image 61 button on the display screen shown in Figure 6. In processing mode, the user can, for example, add annotations to the ultrasound image 61, or perform various image processing operations, including image sharpening. The same applies to optical images; by pressing the optical image 63 button, the user can enter the optical image processing mode and perform various operations on the optical image.

[0068] In this manner, when the user processes at least one of one of the ultrasound image frame and one of the optical image frame, the processed ultrasound image frame and optical image frame are associated and stored in the second image memory 39.

[0069] Furthermore, the image selection unit 37 can not only select one ultrasound image frame and one optical image frame specified by the user, but can also automatically select at least one of the ultrasound image frame and one optical image frame.

[0070] For example, when the freeze button 67 is pressed, the image selection unit 37 may automatically select both one frame of ultrasound image and one frame of optical image.

[0071] The method for automatically selecting ultrasound images is not particularly limited, as long as it can automatically select an ultrasound image that shows the area being examined on the subject, or more precisely, an ultrasound image that shows the area being examined most clearly. For example, if the area being examined has been specified in advance by the user, the image selection unit 37 can automatically select a single ultrasound image containing the area being examined by identifying the area included in each ultrasound image stored in the first image memory 35 through image recognition processing.

[0072] The image selection unit 37 can use, for example, a machine learning-based judgment model as an image recognition process to identify the inspection area included in the ultrasound image. The judgment model is a pre-trained model that has learned the relationship between training ultrasound images and the examination areas included in these training ultrasound images, using training ultrasound images containing the same examination area of ​​any given subject as training data, for multiple training data sets. Based on its learning results, the judgment model takes an ultrasound image to be judged as input and outputs a judgment result (prediction result) of the examination area contained in the ultrasound image. The image selection unit 37 selects a single frame of ultrasound image containing the area to be examined from the ultrasound images based on the result of the judgment by the judgment model.

[0073] Furthermore, the method for automatically selecting optical images is not particularly limited, as long as it is possible to select an optical image in which the entire subject is visible, so that the position and orientation of the ultrasound probe 1 at the time of ultrasound image generation can be determined. For example, the image selection unit 37 can automatically select one frame of optical image that includes the examination site and the ultrasound probe 1 in contact with the subject by identifying the ultrasound probe 1 included in the optical image through image recognition processing for each optical image stored in the first image memory 35.

[0074] For example, Figures 7A and 7B are conceptual diagrams of optical images taken of a subject during an abdominal examination. In the optical image shown in Figure 7A, the subject's abdomen, including the ultrasound probe, is visible more prominently than in the optical image shown in Figure 7B. However, because the entire subject is not visible, it is difficult to determine that the ultrasound probe is making contact with the subject's abdomen. In contrast, in the optical image shown in Figure 7B, the entire subject is visible, making it clear that the ultrasound probe is making contact with the subject's abdomen. In this case, the image selection unit 37 automatically selects the optical image shown in Figure 7B, which captures the entire subject. Furthermore, the image selection unit 37 can similarly utilize a machine learning-based judgment model as an image recognition process to identify the ultrasonic probe 1 included in the optical image.

[0075] By automatically selecting both one frame of ultrasound image and one frame of optical image, the user can avoid the time and effort required to manually select images. However, if the user is not satisfied with the automatically selected images, they may manually select them again.

[0076] Furthermore, the image selection unit 37 may limit the range within which it selects one ultrasound image and one optical image from all ultrasound images and optical images stored in the first image memory 35. When ultrasound images and optical images are stored in the first image memory 35, their generation time (timestamp) is stored. This generation time information can be used to limit the range within which images can be selected.

[0077] In this case, the image selection unit 37 automatically selects one of the two images, an ultrasound image and an optical image, and then automatically selects the other image from among the ultrasound images and optical images stored in the first image memory 35 that were generated within a certain period before and after the generation time of the other image. Furthermore, it is not necessary for the creation times of one image to perfectly match those of the other image; in other words, it is not necessary to select the other image with the same timestamp as the first image. It is sufficient to select the other image from within a certain period before and after the creation time of the first image.

[0078] By limiting the range of images that can be selected, images can be selected more efficiently and in less time than selecting one ultrasound image and one optical image from all ultrasound images and optical images stored in the first image memory 35.

[0079] Furthermore, it is desirable that the period before and after the generation time be, for example, at least 1 second and no more than 2 seconds. If this period is shorter than 1 second, there is a higher possibility that the image will be affected by hand tremors caused by the user pressing the freeze button 67. On the other hand, if the period is longer than 2 seconds, there is a higher possibility that, for example, the position of the ultrasound probe 1 has moved, and the image will include images of a location other than the examination site.

[0080] Alternatively, the image selection unit 37 may, when a user specifies one of the ultrasound images and optical images stored in the first image memory 35, select that one image based on the user's specification, and automatically select the other image from the ultrasound images and optical images.

[0081] In this case, the user specifies a desired single frame of ultrasound image by, for example, moving the knob of the slide bar 73 for ultrasound images left or right, and then stopping the movement of the knob at the desired position. Accordingly, the image selection unit 37 selects the single frame of ultrasound image manually specified by the user from among the ultrasound images stored in the first image memory 35. When a user specifies a single frame of ultrasound image, the image selection unit 37 automatically selects a single frame of optical image from the optical images stored in the first image memory 35. When automatically selecting a single frame of optical image, the knob of the optical image slider bar 75 shown in Figure 6 may be automatically moved to a position corresponding to the generation time of the optical image to be automatically selected.

[0082] By automatically selecting one image, the user can avoid the time and effort required to manually select the other image. However, if the user is not satisfied with the automatically selected image, they may manually select the other image again.

[0083] The image manually specified by the user is not limited to an ultrasound image; the user may specify an optical image as the other image. However, since ultrasound images are used for diagnosis, and optical images are used as body marks representing the position and orientation of the ultrasound probe 1 when the ultrasound image was generated, it is preferable that the image manually specified by the user be an ultrasound image.

[0084] Similarly, the image selection unit 37 may automatically select the other image from among the ultrasound images and optical images stored in the first image memory that were generated within a certain period before and after the generation time of the other image.

[0085] Furthermore, in any of the above cases, if either one of the ultrasound image frames or the optical image frames is selected, the one image may be stored in the second image memory 39 first, and then, if the other image is selected, the other image may be stored in the second image memory 39. Alternatively, both the ultrasound image frames and the optical image frames may be selected, and then both images may be stored in the second image memory 39.

[0086] As shown in Figure 1, the main body of the device 3 may include an ultrasonic image generation unit 31, but is not limited to this; the entire ultrasonic image generation unit 31 or only the signal processing unit 21 may be provided on the ultrasonic probe 1 side.

[0087] In the apparatus of the present invention, the hardware configuration of the processing unit (Processing Unit) that performs various processes such as the transmitting / receiving circuit 13, the ultrasonic image generation unit 31, the image selection unit 37, the display control unit 43, and the device control unit 47 may be dedicated hardware, or it may be various processors or computers that execute programs. Furthermore, the first image memory 35 and the second image memory 39 may be recording media such as flash memory, SD card (Secure Digital card), USB memory (Universal Serial Bus memory), or an HDD (Hard Disk Drive), SSD (Solid State Drive), or an external server.

[0088] Various types of processors include CPUs (Central Processing Units), which are general-purpose processors that execute software (programs) and function as various processing units; Programmable Logic Devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing; and Dedicated Electrical Circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed for performing particular processing.

[0089] A single processing unit may be composed of one of these various processors, or it may be composed of a combination of two or more processors of the same or different types, such as a combination of multiple FPGAs, or a combination of an FPGA and a CPU. Furthermore, multiple processing units may be composed of one of these various processors, or two or more of the multiple processing units may be combined and composed of a single processor.

[0090] For example, as exemplified by servers and client computers, one processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Alternatively, as exemplified by System on Chip (SoC), a processor is used that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip.

[0091] Furthermore, the hardware configuration of these various processors is, more specifically, an electrical circuit (Circuitry) made up of circuit elements such as semiconductor devices.

[0092] Furthermore, the method of the present invention can be implemented, for example, by a program that causes a computer to execute each of its steps. A computer-readable recording medium on which this program is recorded can also be provided.

[0093] Although the present invention has been described in detail above, the present invention is not limited to the embodiments described above, and various improvements and modifications may be made without departing from the spirit of the present invention. [Explanation of symbols]

[0094] 1 Ultrasound probe, 3 Main unit, 11 Transducer array, 13 Transceiver circuit, 15 Battery, 21 Signal processing unit, 23 DSC, 25 Image processing unit, 31 Ultrasound image generation unit, 33 Optical camera, 35 First image memory, 37 Image selection unit, 39 Second image memory, 41 Monitor, 43 Display control unit, 45 Input device, 47 Device control unit, 49 Processor, 51 Pulsar, 53 Amplifier unit, 55 AD conversion unit, 57 Beamformer, 61 Ultrasound image, 63 Optical image, 65 First operation screen, 67 Freeze button, 71 Second operation screen, 73 Slide bar for ultrasound image, 75 Slide bar for optical image, 77 Save button

Claims

1. The device comprises an ultrasonic probe and a main body connected to the ultrasonic probe, The main body of the aforementioned device is An ultrasonic image generation unit generates an ultrasonic image including the inspection area from a received signal obtained by transmitting and receiving an ultrasonic beam to the inspection area of ​​the subject using the ultrasonic probe, An optical camera that generates an optical image including the subject in contact with the ultrasonic probe, A display control unit that displays the ultrasound image and the optical image on a monitor, A first image memory that stores multiple frames of ultrasound images and multiple frames of optical images generated in the past, An image selection unit selects from the ultrasound images and optical images stored in the first image memory to be stored in the second image memory, An ultrasound diagnostic apparatus comprising: a second image memory that stores the selected ultrasound image and the optical image in association.

2. The first image memory stores multiple frames of the ultrasound image and multiple frames of the optical image that were generated during a certain period of time prior to a timing specified by the user. The ultrasound diagnostic apparatus according to claim 1, wherein the user specifies the timing by pressing a freeze button displayed on the monitor.

3. The ultrasound diagnostic apparatus according to claim 1 or 2, wherein the display control unit, in response to instructions from the user, scrolls back from the ultrasound images stored in the first image memory to display a past ultrasound image specified by the user on the monitor, and scrolls back from the optical images stored in the first image memory to display a past optical image specified by the user on the monitor.

4. The ultrasound diagnostic apparatus according to any one of claims 1 to 3, wherein the image selection unit selects the ultrasound image and the optical image to be stored in the second image memory based on the user's specification.

5. The ultrasound diagnostic apparatus according to any one of claims 1 to 3, wherein the image selection unit automatically selects at least one of the ultrasound image and the optical image to be stored in the second image memory.

6. The ultrasound diagnostic apparatus according to claim 5, wherein the image selection unit automatically selects both the ultrasound image and the optical image to be stored in the second image memory.

7. The ultrasound diagnostic apparatus according to claim 6, wherein the image selection unit automatically selects one of the ultrasound images and optical images to be stored in the second image memory, and from among the ultrasound images and optical images stored in the first image memory, it automatically selects the other ultrasound image and optical image to be stored in the second image memory from among the ultrasound images and optical images generated within a certain period before and after the generation time of the one image.

8. The ultrasound diagnostic apparatus according to claim 5, wherein the image selection unit selects one of the ultrasound images and optical images stored in the second image memory based on a user specification, and automatically selects the other of the ultrasound images and optical images stored in the second image memory.

9. The ultrasound diagnostic apparatus according to claim 8, wherein the image selection unit automatically selects the other image from among the ultrasound images and optical images stored in the first image memory, from among the ultrasound images and optical images generated within a certain period before and after the generation time of the other image.

10. The ultrasound diagnostic apparatus according to any one of claims 1 to 9, wherein the second image memory stores the processed ultrasound image and the optical image in association with each other when the user has processed at least one of the ultrasound image and the optical image stored in the second image memory.

Images