ULTRASONIC SYSTEM AND METHOD FOR CONTROLLING AN ULTRASONIC SYSTEM - Patent application

Abstract

To provide an ultrasonic system capable of obtaining an appropriate ultrasonic image and capable of improving accuracy of ultrasonic diagnosis even in a case where the ultrasonic image is taken at a remote area.SOLUTION: An ultrasonic system (1C) includes an ultrasonic probe (2C), a head-mounted display (3C), and an external device (4C). The ultrasonic probe (2C) includes an ultrasonic image generation part for generating an ultrasonic image, and a probe side radio communication part (24). The head-mounted display (3C) includes a camera part (33) for obtaining a visual field image, an image synchronization part (32) for synchronizing the ultrasonic image and the visual field image with each other by referring to a time stamp, and a head-mounted display side radio communication part (31). The external device (4C) includes an external radio communication part (41), an external monitor (45), and a display control part (44) for displaying the ultrasonic image and the visual field image on the external monitor (45).SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to an ultrasound system and a control method for an ultrasound system, and more particularly to an ultrasound system that displays an ultrasound image on a head mounted display and a control method for an ultrasound system. [Background technology]

[0002] Conventionally, ultrasonic diagnostic devices using ultrasonic images have been put to practical use in the medical field. In general, this type of ultrasonic diagnostic device has an ultrasonic probe with a built-in transducer array and a device main body connected to the ultrasonic probe, and transmits ultrasonic waves from the ultrasonic probe to a subject, receives ultrasonic echoes from the subject by the ultrasonic probe, and electrically processes the received signals in the device main body to generate an ultrasonic image.

[0003] In such an ultrasound diagnostic device, the monitor on which the ultrasound image is displayed is usually placed at a position away from the ultrasound probe, such as at the bedside, and the operator must alternately move his / her line of sight between the ultrasound probe at hand and the monitor. Therefore, in order to reduce the movement of the operator's line of sight, an ultrasound diagnostic device equipped with a so-called head-mounted display, such as that disclosed in Patent Document 1, has been developed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-183056 A Summary of the Invention [Problem to be solved by the invention]

[0005] It is generally known that in ultrasound diagnosis using an ultrasound diagnostic device such as that disclosed in Patent Document 1, a certain level of skill is required to accurately grasp an area in a subject depicted in an ultrasound image by checking the ultrasound image. It is also known that the image quality of a generated ultrasound image is greatly affected by the technique of the operator.

[0006] Here, for example, when ultrasound images are taken in a remote location outside a hospital, such as in home nursing, the operator who operates the ultrasound probe to take the ultrasound image may be different from the observer, such as a doctor, who observes the taken ultrasound image and makes a diagnosis. In this case, since the operator usually needs to operate the ultrasound probe while checking the obtained ultrasound image by himself, and take an ultrasound image of the part inside the subject of interest, it may be difficult for the operator to judge whether or not the part of the subject of interest is accurately observed, particularly when the operator is less skilled. Also, an operator with a low level of skill may not be able to operate the ultrasound probe using an appropriate technique, and may obtain an ultrasound image of low image quality. Also, the observer checks the ultrasound image taken by the operator of the ultrasound diagnostic device to make a diagnosis, but since the observer cannot grasp how the operator takes the ultrasound image, it may be difficult to accurately judge whether or not the taken ultrasound image is taken using an appropriate technique, particularly when the ultrasound image is taken by an operator with a low level of skill.

[0007] The present invention has been made to solve such conventional problems, and has an object to provide an ultrasound system and a control method for an ultrasound system that can obtain appropriate ultrasound images and improve the accuracy of ultrasound diagnosis even when ultrasound images are taken in a remote location.

[0008] In order to achieve the above object, a first ultrasound system according to the present invention is an ultrasound system including an ultrasound probe, a head-mounted display, and an external device, wherein the ultrasound probe includes a transducer array, a transmission / reception circuit that transmits ultrasound from the transducer array and generates a sound ray signal based on a reception signal acquired by the transducer array, an ultrasound image generation unit that generates an ultrasound image based on the sound ray signal generated by the transmission / reception circuit, and a probe-side wireless communication unit that wirelessly transmits the ultrasound image, and the head-mounted display includes a camera unit that acquires a field of view image of a scanning location of the ultrasound probe in a subject, an image synchronization unit that synchronizes the ultrasound image and the field of view image captured at the same timing by referring to the timestamp of the ultrasound image wirelessly transmitted from the ultrasound probe and the timestamp of the field of view image acquired by the camera unit, and a head-mounted display-side wireless communication unit that wirelessly transmits the ultrasound image and the field of view image synchronized with each other by the image synchronization unit, and the external device includes at least an external wireless communication unit wirelessly connected to the head-mounted display-side wireless communication unit, an external monitor, and a display control unit that displays on the external monitor the ultrasound image and the field of view image synchronized with each other by the image synchronization unit and wirelessly transmitted from the head-mounted display.

[0009] The head-mounted display may include a head-mounted display monitor, and the ultrasound image may be displayed on the head-mounted display monitor.

[0010] The external device may include an input device, and the external wireless communication unit may wirelessly transmit external input information input via the input device to the head-mounted display side wireless communication unit, and the external input information may be displayed on a monitor on the head-mounted display side.

[0011] The first ultrasound system is capable of bidirectional wireless communication of audio data between the head mounted display side wireless communication unit and the external wireless communication unit.

[0012] A first ultrasonic system control method according to the present invention is a control method for an ultrasonic system including an ultrasonic probe, a head-mounted display, and an external device, characterized in that, in the ultrasonic probe, ultrasonic waves are transmitted from a transducer array of the ultrasonic probe, and a sound ray signal is generated based on a received signal acquired by the transducer array, an ultrasonic image is generated based on the generated sound ray signal, and the ultrasonic image is wirelessly transmitted; in the head-mounted display, a field of view image capturing a scanning location of the ultrasonic probe on the subject is acquired, and by referring to a timestamp of the ultrasonic image wirelessly transmitted from the ultrasonic probe and a timestamp of the acquired field of view image, the ultrasonic image and the field of view image captured at the same timing are synchronized with each other by referring to a timestamp of the ultrasonic image wirelessly transmitted from the ultrasonic probe and a timestamp of the acquired field of view image, and the acquired field of view image is wirelessly transmitted; and, in the external device, the ultrasonic image and the field of view image that are synchronized with each other and wirelessly transmitted from the head-mounted display are displayed on an external monitor.

[0013] A second ultrasound system according to the present invention is an ultrasound system including an ultrasound probe, a head-mounted display, and an external device, wherein the ultrasound probe includes a transducer array, a transmission / reception circuit that transmits ultrasound from the transducer array and generates a sound ray signal based on a reception signal acquired by the transducer array, a reception data generation unit that generates reception data before imaging by performing signal processing on the sound ray signal generated by the transmission / reception circuit, and a probe-side wireless communication unit that wirelessly transmits the reception data, and the head-mounted display includes a camera unit that acquires a field of view image of a scanning location of the ultrasound probe in a subject, an image synchronization unit that synchronizes the ultrasound image and the field of view image captured at the same timing by referring to the timestamp of the ultrasound image generated based on the reception data wirelessly transmitted from the ultrasound probe and the timestamp of the field of view image acquired by the camera unit, and a head-mounted display-side wireless communication unit that wirelessly transmits the field of view image acquired by the camera unit, and the external device includes at least an external wireless communication unit wirelessly connected to the head-mounted display-side wireless communication unit, an external monitor, and a display control unit that displays on the external monitor the ultrasound image and the field of view image synchronized with each other by the image synchronization unit and wirelessly transmitted from the head-mounted display.

[0014] The head mounted display can include an image processing unit that generates an ultrasound image based on received data wirelessly transmitted from the probe side wireless communication unit.

[0015] The head-mounted display may include a head-mounted display monitor, and the ultrasound image may be displayed on the head-mounted display monitor.

[0016] The external device may include an input device, and the external wireless communication unit may wirelessly transmit external input information input via the input device to the head-mounted display side wireless communication unit, and the external input information may be displayed on a monitor on the head-mounted display side.

[0017] The second ultrasound system is capable of bidirectional wireless communication of audio data between the head mounted display side wireless communication unit and the external wireless communication unit.

[0018] A second ultrasonic system control method according to the present invention is a control method for an ultrasonic system including an ultrasonic probe, a head-mounted display, and an external device, characterized in that, in the ultrasonic probe, ultrasonic waves are transmitted from a transducer array of the ultrasonic probe, and a sound ray signal is generated based on a received signal acquired by the transducer array, and received data before imaging is generated by performing signal processing on the generated sound ray signal, and the received data is wirelessly transmitted; in the head-mounted display, a field of view image capturing a scanning location of the ultrasonic probe on the subject is acquired; by referring to a timestamp of the ultrasonic image generated based on the received data wirelessly transmitted from the ultrasonic probe and a timestamp of the acquired field of view image, the ultrasonic image and the field of view image captured at the same timing are synchronized with each other, and the acquired field of view image is wirelessly transmitted; and, in the external device, the ultrasonic image and the field of view image synchronized with each other and wirelessly transmitted from the head-mounted display are displayed on an external monitor. Effect of the Invention

[0019] According to the present invention, the ultrasonic probe includes a transducer array, a transmission / reception circuit that transmits ultrasonic waves from the transducer array and generates a sound ray signal based on a received signal acquired by the transducer array, an ultrasonic image generating unit that generates an ultrasonic image based on the sound ray signal generated by the transmission / reception circuit, and a probe-side wireless communication unit that wirelessly transmits the ultrasonic image, and the head-mounted display includes a camera unit that acquires a field of view image of a scanning location of the ultrasonic probe in the subject, an image synchronization unit that synchronizes the ultrasonic image and the field of view image captured at the same timing by referring to the timestamp of the ultrasonic image wirelessly transmitted from the ultrasonic probe and the timestamp of the field of view image acquired by the camera unit, and a head-mounted display-side wireless communication unit that wirelessly transmits the ultrasonic image and the field of view image synchronized with each other by the image synchronization unit, and the external device includes at least an external wireless communication unit wirelessly connected to the head-mounted display-side wireless communication unit, an external monitor, and a display control unit that displays on the external monitor the ultrasonic image and the field of view image synchronized with each other by the image synchronization unit and wirelessly transmitted from the head-mounted display. Therefore, even when an ultrasonic image is captured in a remote location, an appropriate ultrasonic image can be obtained and the accuracy of ultrasonic diagnosis can be improved. [Brief description of the drawings]

[0020] [Figure 1] 1 is a block diagram showing a configuration of an ultrasound system according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an internal configuration of a transmission / reception circuit according to a first embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram illustrating an example of an HMD according to the first embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing a schematic diagram of an ultrasound image displayed on an HMD-side monitor of the HMD in the first embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram illustrating an example of an external device according to the first embodiment of the present invention. [Figure 6]FIG. 13 is a diagram showing a schematic example of a cursor displayed on a visual field image on an external monitor in a modified example of the first embodiment of the present invention. [Figure 7] FIG. 13 is a diagram illustrating an example of a cursor displayed on an HMD monitor in a modified example of the first embodiment of the present invention. [Figure 8] FIG. 13 is a block diagram showing a configuration of an HMD according to another modified example of the first embodiment of the present invention. [Figure 9] FIG. 13 is a block diagram showing a configuration of an external device according to another modified example of the first embodiment of the present invention. [Figure 10] FIG. 11 is a block diagram showing a configuration of an ultrasound system according to a second embodiment of the present invention. [Figure 11] FIG. 11 is a block diagram showing a configuration of an ultrasound system according to a third embodiment of the present invention. [Figure 12] FIG. 11 is a block diagram showing the configuration of an ultrasound system according to a fourth embodiment of the present invention. [Figure 13] FIG. 11 is a block diagram showing the configuration of an ultrasound system according to a fifth embodiment of the present invention. [Figure 14] 13A to 13C are diagrams showing schematic examples of an ultrasound image and a field of view image displayed on an external device in accordance with a fifth embodiment of the present invention. [Figure 15] FIG. 13 is a block diagram showing the configuration of an ultrasound system according to a sixth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The following description of the components will be given based on a representative embodiment of the present invention, but the present invention is not limited to such an embodiment. In this specification, a numerical range expressed using "to" means a range that includes the numerical values ​​before and after "to" as the lower and upper limits. In this specification, the terms "same" and "identical" include a generally acceptable margin of error in the technical field.

[0022] First embodiment 1 shows the configuration of an ultrasound system 1 according to a first embodiment of the present invention. The ultrasound system 1 includes an ultrasound probe 2, an HMD (Head Mounted Display) 3, and an external device 4. The HMD 3 and the external device 4 are connected to the ultrasound probe 2 via wireless communication, and the HMD 3 and the external device 4 are connected to each other via wireless communication.

[0023] The ultrasonic probe 2 includes a transducer array 21, to which a transmission / reception circuit 22, a signal processing unit 23, and a probe-side wireless communication unit 24 are sequentially connected. The probe-side wireless communication unit 24 is connected by wireless communication to the HMD 3 and the external device 4. Although not shown, the signal processing unit 23 constitutes a received data generating unit. A probe control unit 26 is connected to the transmission / reception circuit 22, the signal processing unit 23, and the probe side wireless communication unit 24. A probe side processor 27 is configured by the signal processing unit 23, the probe side wireless communication unit 24, and the probe control unit 26.

[0024] The HMD 3 includes an HMD-side wireless communication unit 31 that is connected to the ultrasonic probe 2 and the external device 4 by wireless communication, and an image processing unit 32, a display control unit 35, and an HMD-side monitor (head mounted display-side monitor) 36 are sequentially connected to the HMD-side wireless communication unit (head mounted display-side wireless communication unit) 31. The HMD 3 also includes a camera unit 33 that is connected to the HMD-side wireless communication unit 31. An HMD control unit (head mounted display control unit) 37 is connected to the HMD side wireless communication unit 31, the image processing unit 32, the camera unit 33, and the display control unit 35. An HMD side processor (head mounted display side processor) 39 is configured by the HMD side wireless communication unit 31, the image processing unit 32, the display control unit 35, and the HMD control unit 37.

[0025] The external device 4 includes an external wireless communication unit 41 that is connected to the ultrasound probe 2 and the HMD 3 by wireless communication, and an image processing unit 42 and an image synchronization unit 43 are connected to the external wireless communication unit 41. The image processing unit 42 is also connected to the image synchronization unit 43. A display control unit 44 and an external monitor 45 are also connected to the image synchronization unit 43 in this order. An external control unit 46 is connected to the external wireless communication unit 41, the image processing unit 42, the image synchronization unit 43, and the display control unit 44. An input device 47 is connected to the external control unit 46. An external device side processor 48 is configured by the external wireless communication unit 41, the image processing unit 42, the image synchronization unit 43, the display control unit 44, and the external control unit 46.

[0026] The transducer array 21 of the ultrasonic probe 2 has a plurality of transducers arranged one-dimensionally or two-dimensionally. Each of these transducers transmits ultrasonic waves in accordance with a drive signal supplied from the transmission / reception circuit 22, receives ultrasonic echoes from the subject, and outputs a reception signal based on the ultrasonic echoes. Each transducer is configured by forming electrodes on both ends of a piezoelectric body made of, for example, a piezoelectric ceramic represented by PZT (Lead Zirconate Titanate), a polymeric piezoelectric element represented by PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal represented by PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0027] The transmission / reception circuit 22, under the control of the probe control unit 26, transmits ultrasonic waves from the transducer array 21 and generates sound ray signals based on reception signals acquired by the transducer array 21. As shown in Fig. 2, the transmission / reception circuit 22 has a pulser 51 connected to the transducer array 21, and an amplifier unit 52, an AD (Analog-to-Digital) converter unit 53, and a beamformer 54, which are connected in series from the transducer array 21 in this order.

[0028] The pulser 51 includes, for example, a plurality of pulse generators, and adjusts the delay amount of each drive signal and supplies it to the plurality of transducers so that the ultrasonic waves transmitted from the plurality of transducers of the transducer array 21 form an ultrasonic beam based on the transmission delay pattern selected in response to the control signal from the probe control unit 26. In this way, when a pulsed or continuous wave voltage is applied to the electrodes of the transducers of the transducer array 21, the piezoelectric body expands and contracts, and each transducer generates a pulsed or continuous wave ultrasonic wave, and an ultrasonic beam is formed from the composite wave of those ultrasonic waves.

[0029] The transmitted ultrasonic beam is reflected by an object such as a part of the subject, and propagates toward the transducer array 21 of the ultrasonic probe 2. When the ultrasonic echo propagates toward the transducer array 21 in this manner, each transducer constituting the transducer array 21 receives the propagating ultrasonic echo and expands and contracts to generate received signals, which are electrical signals.

[0030] The amplifier 52 amplifies signals input from each transducer constituting the transducer array 21 and transmits the amplified signals to the AD converter 53. The AD converter 53 converts the signals transmitted from the amplifier 52 into digital reception data and transmits the reception data to the beamformer 54. The beamformer 54 performs so-called reception focusing processing by adding each of the reception data converted by the AD converter 53 with a delay according to the sound speed or sound speed distribution set based on the reception delay pattern selected in response to a control signal from the probe control unit 26. This reception focusing processing causes each of the reception data converted by the AD converter 53 to be phased and added, and a sound ray signal with a narrowed focus of the ultrasonic echo is acquired.

[0031] The signal processing unit 23 generates reception data before imaging by performing signal processing on the sound ray signals generated by the beamformer 54 of the transmission / reception circuit 22. More specifically, the signal processing unit 23 performs an envelope detection process on the sound ray signals generated by the beamformer 54 of the transmission / reception circuit 22 to correct attenuation caused by the propagation distance in accordance with the depth of the position where the ultrasonic waves are reflected, and then generates a signal representing tomographic image information on tissues in the subject as reception data before imaging.

[0032] The probe-side wireless communication unit 24 includes an antenna for transmitting and receiving radio waves, and generates a transmission signal representing the pre-imaging reception data by modulating a carrier based on the pre-imaging reception data generated by the signal processing unit 23. The probe-side wireless communication unit 24 wirelessly transmits the pre-imaging reception data sequentially to the HMD-side wireless communication unit 31 of the HMD 3 and the external wireless communication unit 41 of the external device 4 by supplying the thus generated transmission signal to an antenna and transmitting radio waves from the antenna. As a modulation method of the carrier, ASK (Amplitude Shift Keying), PSK (Phase Shift Keying), QPSK (Quadrature Phase Shift Keying), 16QAM (16 Quadrature Amplitude Modulation), etc. are used.

[0033] In addition, wireless communication between the probe side wireless communication unit 24 of the ultrasonic probe 2, the HMD side wireless communication unit 31 of the HMD 3, and the external wireless communication unit 41 of the external device 4 can be performed in accordance with communication standards for mobile communications such as 5G (5th Generation: fifth generation mobile communication system) and 4G (4th Generation: fourth generation mobile communication system), and communication standards for short-range wireless communication such as WiFi (registered trademark), Bluetooth (registered trademark), and UWB (Ultra Wide Band: ultra-wideband wireless system).

[0034] Since the ultrasonic probe 2 and the HMD 3 are assumed to be located in close proximity to each other, either of the wireless communication methods, mobile communication or short-range wireless communication, can be adopted for wireless communication between the ultrasonic probe 2 and the HMD 3. Furthermore, since it is assumed that the external device 4 is located in a remote location relative to the ultrasonic probe 2 and the HMD 3, it is preferable that mobile communication is performed as the wireless communication between the external device 4 and the ultrasonic probe 2 and the wireless communication between the external device 4 and the HMD 3. In particular, from the viewpoint of reducing the time lag in data transmission between the external device 4 and the ultrasonic probe 2 and the HMD 3, it is preferable that mobile communication conforming to 5G is performed as the wireless communication between the external device 4 and the ultrasonic probe 2 and the wireless communication between the external device 4 and the HMD 3.

[0035] The probe control unit 26 controls each part of the ultrasonic probe 2 based on a control program stored in advance. Although not shown, a probe side storage unit is connected to the probe control unit 26. The probe side storage unit stores a control program for the ultrasonic probe 2 and the like. As the probe side storage unit, for example, a flash memory, a RAM (Random Access Memory), an SD card (Secure Digital card), an SSD (Solid State Drive), or the like can be used. Although not shown, the ultrasonic probe 2 includes a built-in battery, and power is supplied to each circuit of the ultrasonic probe 2 from the battery.

[0036] The probe side processor 27 having the signal processing unit 23, the probe side wireless communication unit 24 and the probe control unit 26 is composed of a CPU (Central Processing Unit) and a control program for causing the CPU to perform various processes, but may also be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit) or other ICs (Integrated Circuits), or may be composed of a combination of these.

[0037] Furthermore, the signal processing unit 23, the probe wireless communication unit 24, and the probe control unit 26 of the probe-side processor 27 may be configured to be partially or entirely integrated into a single CPU or the like.

[0038] The HMD 3 is a display device that is worn on the head of an operator and is visually recognized by the operator wearing it, and has a so-called eyeglass-like shape as shown in Fig. 3. The HMD 3 includes two monitors 36A and 36B as HMD-side monitors 36. The two monitors 36A and 36B are connected to each other by a bridge B, and a tendril T is connected to each end of the two monitors 36A and 36B. For example, the bridge B is hooked on the operator's nose and the two tendrils T are hooked on both ears of the operator, thereby fixing the HMD 3 to the operator's head. At this time, the two monitors 36A and 36B face the right and left eyes of the operator.

[0039] In addition, when the operator wears the HMD 3, a camera unit 33 with a photographing lens F disposed in front is attached to the connecting portion between the monitor 36B on the left side as seen from the operator and the tendon T. In addition, a storage unit D that stores various circuits and the like required for the operation of the HMD 3 is disposed in the tendon T connected to the monitor 36A on the right side.

[0040] 1 includes an antenna for transmitting and receiving radio waves, and receives a transmission signal representing reception data before imaging transmitted by the probe-side wireless communication unit 24 of the ultrasound probe 2 via the antenna under the control of the HMD control unit 37, and outputs the reception data before imaging by demodulating the received transmission signal. Furthermore, the HMD-side wireless communication unit 31 sends the reception data before imaging to the image processing unit 32.

[0041] The image processing unit 32 raster-converts the received data before imaging sent from the HMD wireless communication unit 31 into an image signal conforming to a normal television signal scanning method, and generates a B-mode (Brightness mode) image signal by performing various necessary image processing such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction on the converted image signal conforming to a display format for the HMD monitor 36. The B-mode image signal generated in this manner is simply referred to as an ultrasound image U. The image processing unit 32 also sends the generated ultrasound image U to the display control unit 35.

[0042] The HMD-side monitor 36 displays the ultrasound image U and the like under the control of the display control unit 35, and further has translucency to ensure the operator's field of vision when the operator is wearing the HMD 3. Therefore, the operator can check the ultrasound image U and the like displayed on the HMD-side monitor 36 while checking the forward field of vision through the HMD-side monitor 36.

[0043] The display control unit 35, under the control of the HMD control unit 37, performs predetermined processing on the ultrasound image U sent from the image processing unit 32, and displays the ultrasound image U on the HMD-side monitor 36 of the HMD 3, as shown in Fig. 4. Fig. 4 shows an example in which the operator checks the subject P located in front of him through the HMD-side monitor 36, and the ultrasound image U is displayed on the HMD-side monitor 36 so as to be superimposed on a part of the operator's field of vision in front of him.

[0044] The camera unit 33 acquires a visual field image C by capturing an image of a scanning location of the ultrasonic probe 2 on the subject P. The camera unit 33 includes a photographing lens F, an image sensor (not shown) that captures an image of the scanning location of the ultrasonic probe 2 through the photographing lens F and acquires a visual field image signal, which is an analog signal, an analog signal processing circuit (not shown) that amplifies the visual field image signal acquired by the image sensor and converts it into a digital signal, and a digital signal processing circuit (not shown) that performs various corrections such as gain on the converted digital signal to generate the visual field image C. The analog signal processing circuit and the digital signal processing circuit can also be built into the HMD processor 39. The camera unit 33 transmits the generated visual field image C to the HMD wireless communication unit 31. The visual field image C transmitted to the HMD wireless communication unit 31 is wirelessly transmitted to the external device 4 by the HMD wireless communication unit 31.

[0045] The HMD control unit 37 controls each part of the HMD 3 based on a control program stored in advance. Although not shown, an HMD side storage unit is connected to the HMD control unit 37. The HMD side storage unit stores a control program and the like for the HMD 3. As the HMD side storage unit, for example, a flash memory, a RAM, an SD card, an SSD, etc. can be used. Although not shown, the HMD 3 has a built-in battery, and power is supplied to each circuit of the HMD 3 from this battery.

[0046] The HMD side processor 39 having the HMD side wireless communication unit 31, image processing unit 32, display control unit 35 and HMD control unit 37 is composed of a CPU and a control program for causing the CPU to perform various processes, but may also be composed of an FPGA, DSP, ASIC, GPU or other ICs, or a combination of these. Furthermore, the HMD wireless communication unit 31, image processing unit 32, display control unit 35 and HMD control unit 37 of the HMD processor 39 can be configured as being partially or entirely integrated into a single CPU or the like.

[0047] The external wireless communication unit 41 of the external device 4 includes an antenna for transmitting and receiving radio waves, and receives, under the control of the external control unit 46, a transmission signal representing the received data before imaging transmitted by the probe side wireless communication unit 24 of the ultrasound probe 2 and a transmission signal representing the visual field image C transmitted by the HMD side wireless communication unit 31 of the HMD 3 via the antenna, and demodulates the received transmission signal to output the received data before imaging and the visual field image C. Furthermore, the external wireless communication unit 41 sends the received data before imaging to the image processing unit 42, and sends the visual field image C to the image synchronization unit 43.

[0048] The image processing unit 42 raster-converts the received data before imaging sent from the external wireless communication unit 41 into an image signal conforming to a normal television signal scanning method, and performs various necessary image processing on the converted image signal, such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction, conforming to a display format for the external monitor 45, to generate an ultrasound image U. The image processing unit 42 sends the generated ultrasound image U to the image synchronization unit 43.

[0049] The image synchronization unit 43 of the external device 4 synchronizes the ultrasonic image U sent from the image processing unit 42 with the visual field image C sent from the external wireless communication unit 41, and generates a composite image M based on the ultrasonic image U and visual field image C synchronized with each other. Here, synchronizing the ultrasonic image U and visual field image C with each other means associating the ultrasonic image U and visual field image C captured at the same timing with each other. For example, when the image processing unit 42 of the external device 4 assigns a time stamp representing the time when the ultrasonic image U was generated to the ultrasonic image U, and the camera unit 33 of the HMD 3 assigns a time stamp representing the time when the visual field image C was generated to the visual field image C, the image synchronization unit 43 regards the time stamp of the ultrasonic image U as representing the time when the ultrasonic image U was captured, and regards the time stamp of the visual field image C as representing the time when the visual field image C was captured, and refers to the time stamps of the ultrasonic image U and the visual field image C, thereby synchronizing the ultrasonic image U and the visual field image C captured at the same timing with each other.

[0050] In addition, when associating the ultrasonic image U with the field of view image C, the image synchronization unit 43 can, for example, refer to the time stamp of the ultrasonic image U and the time stamp of the field of view image C, and if the difference between the time when the ultrasonic image U was captured and the time when the field of view image C was captured is within a certain range, for example, within 0.1 seconds, regard the ultrasonic image U and the field of view image C as having been captured at the same timing, and perform the association. In addition, the image synchronization unit 34 can, for example, refer to the time stamp of the ultrasonic image U and the time stamp of the field of view image C, and select the field of view image C captured at the time closest to the time when the ultrasonic image U to be associated was captured, and associate the selected ultrasonic image U with the field of view image C. In addition, the image synchronization unit 34 can, for example, select the ultrasonic image U captured at the time closest to the time when the field of view image C to be associated was captured, and associate the selected ultrasonic image U with the field of view image C. The image synchronization unit 43 sends the ultrasound image U and the field of view image C synchronized in this manner to the display control unit 44.

[0051] The display control unit 44, under the control of the external control unit 46, performs a predetermined processing on the composite image M sent from the image synchronization unit 43, and displays the synchronized ultrasound image U and field of view image C together on the external monitor 45 of the external device 4, as shown in Figure 5. The external monitor 45 displays the ultrasound image U and the visual field image C under the control of the display control unit 44, and includes, for example, a display device such as a Liquid Crystal Display or an Organic Electroluminescence Display. The input device 47 of the external device 4 is used by the operator to perform an input operation, and includes a touch sensor disposed on top of the external monitor 45 .

[0052] The external control unit 46 controls each part of the external device 4 based on a control program stored in advance. Although not shown, an external device side storage unit is connected to the external device 4. The external device side storage unit stores a control program and the like for the external device 4. As the external device side storage unit, for example, a flash memory, a RAM, an SD card, an SSD, etc. can be used. Although not shown, the external device 4 includes a built-in battery, and power is supplied to each circuit of the external device 4 from the battery.

[0053] The external device side processor 48 having the external wireless communication unit 41, image processing unit 42, image synchronization unit 43, display control unit 44 and external control unit 46 is composed of a CPU and a control program for causing the CPU to perform various processes, but may also be composed of an FPGA, DSP, ASIC, GPU or other ICs, or a combination of these. In addition, the external wireless communication unit 41, image processing unit 42, image synchronization unit 43, display control unit 44 and external control unit 46 of the external device side processor 48 can be configured as being partially or entirely integrated into a single CPU or the like.

[0054] Next, the operation of the ultrasound system 1 according to the first embodiment of the present invention will be described. First, the operator brings the ultrasonic probe 2 into contact with the body surface of the subject P, and under the control of the probe control unit 26, ultrasonic beams are transmitted from the multiple transducers of the transducer array 21 into the subject P in accordance with a drive signal from a pulsar 51 of the transmission / reception circuit 22. Ultrasonic echoes based on the transmitted ultrasonic beams are received by each transducer, and the received signals, which are analog signals, are output to the amplifier unit 52 and amplified, and are then AD converted by the AD conversion unit 53 to obtain received data. A beamformer 54 performs reception focusing processing on this received data, generating sound ray signals.

[0055] The generated sound ray signal is converted by the signal processing unit 23 into reception data before imaging, which is a signal representing tomographic image information related to tissue in the subject P. At this time, the signal processing unit 23 performs envelope detection processing on the sound ray signal after correcting attenuation caused by the propagation distance according to the depth of the position where the ultrasonic wave is reflected. The probe side wireless communication unit 24 wirelessly transmits the generated sound ray signal to the HMD 3 and the external device 4.

[0056] The HMD-side wireless communication unit 31 of the HMD 3 receives the pre-imaging reception data wirelessly transmitted from the ultrasound probe 2, and sends the received pre-imaging reception data to the image processing unit 32. The image processing unit 32 raster-converts the pre-imaging reception data sent from the HMD-side wireless communication unit 31 into an image signal conforming to a normal television signal scanning method, and performs various necessary image processes on the converted image signal, such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction, conforming to a display format for the HMD-side monitor 36, to generate an ultrasound image U. The ultrasound image U generated in this manner is sent to the display control unit 35.

[0057] The display control unit 35 performs a predetermined process on the ultrasound image U, and then transmits the ultrasound image U to the HMD monitor 36, and displays the ultrasound image U on the HMD monitor 36 as shown in Fig. 4. The ultrasound image U is displayed on the HMD monitor 36 so as to be superimposed on a part of the operator's forward visual field, so that the operator can simultaneously check the forward visual field and the ultrasound image U, and easily associate and grasp the scanning location of the ultrasound probe 2 on the subject P with the tissue in the subject P observed thereby. In the example of Fig. 4, the operator's forward visual field shows the ultrasound probe 2 being in contact with the abdomen of the subject P, and the ultrasound image U depicts the internal abdominal tissue of the subject P.

[0058] Further, under the control of the HMD control unit 37, the camera unit 33 of the HMD 3 acquires a field of view image C obtained by capturing an image of a location on the subject P scanned by the ultrasonic probe 2. For example, as shown in Fig. 4, when the operator directs the forward field of view toward the subject P, the camera unit 33 acquires a field of view image C obtained by capturing an image of a location on the subject P scanned by the ultrasonic probe 2. The field of view image C acquired in this manner is sent to the HMD side wireless communication unit 31.

[0059] The HMD side wireless communication unit 31 wirelessly transmits the visual field image C sent from the camera unit 33 to the external device 4. The external wireless communication unit 41 of the external device 4 receives the pre-imaging received data wirelessly transmitted from the ultrasound probe 2 and the field of view image C wirelessly transmitted from the HMD 3, sends the received pre-imaging received data to the image processing unit 42, and sends the received field of view image C to the image synchronization unit 43.

[0060] The image processing unit 42 raster-converts the received data before imaging sent from the external wireless communication unit 41 into an image signal conforming to a normal television signal scanning method, and performs various necessary image processing on the converted image signal, such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction, conforming to a display format for the external monitor 45, to generate an ultrasound image U. The image processing unit 42 sends the generated ultrasound image U to the image synchronization unit 43.

[0061] The image synchronization unit 43 synchronizes the ultrasonic image U sent from the image processing unit 42 with the visual field image C sent from the external wireless communication unit 41, and generates a composite image M in which the ultrasonic image U and the visual field image C synchronized with each other are combined into one. For example, when the image processing unit 42 of the external device 4 assigns a time stamp representing the time when the ultrasonic image U was generated to the ultrasonic image U, and the camera unit 33 of the HMD 3 assigns a time stamp representing the time when the visual field image C was generated to the visual field image C, the image synchronization unit 43 regards the time stamp of the ultrasonic image U as representing the time when the ultrasonic image U was captured, and regards the time stamp of the visual field image C as representing the time when the visual field image C was captured, and refers to the time stamps of the ultrasonic image U and the visual field image C, thereby synchronizing the ultrasonic image U and the visual field image C captured at the same timing with each other. At this time, the time in the HMD 3 and the time in the external device 4 can be shared with each other.

[0062] The time in the HMD 3 and the time in the external device 4 can be shared with each other, specifically, for example, the time can be shared based on either the HMD 3 or the external device 4. Also, for example, when either the HMD 3 or the external device 4 is connected to the Internet, the time of the built-in clock may be set using a communication protocol such as NTP (Network Time Protocol) or NITZ (Network Identity and Time Zone).

[0063] The ultrasound image U and the field of view image C synchronized with each other by the image synchronization unit 43 are sent to the display control unit 44 as a composite image M. The display control unit 44 performs a predetermined process on the composite image M, and then sends the composite image M to the external monitor 45, and as shown in FIG. 5, the ultrasound image U and the field of view image C are displayed together on the external monitor 45. In the example of FIG. 5, the external device 4 is shown to be a portable thin computer called a smartphone or tablet, and the field of view image C and the ultrasound image U are displayed on the top and bottom of the HMD side monitor 36 of the external device 4. In this example, the field of view image C depicts the ultrasound probe 2 being in contact with the abdomen of the subject P, and the ultrasound image U depicts the internal tissue of the abdomen of the subject P.

[0064] Here, an ultrasound image U is displayed on the HMD-side monitor 36 of the HMD 3, and a visual field image C and an ultrasound image U that are synchronized with each other are displayed on the external monitor 45 of the external device 4, but the same ultrasound image U is displayed on the HMD-side monitor 36 and the external monitor 45 almost simultaneously. Also, a visual field image C corresponding to the forward visual field observed by the operator through the HMD-side monitor 36 is displayed almost in real time on the external monitor 45. Therefore, for example, even if the external device 4 is located at a remote location from the HMD 3, the observer observing the external monitor 45 can observe the ultrasound image U captured at the examination site where the subject P and the operator are located, and the visual field image C corresponding to the operator's forward visual field, almost in real time.

[0065] Incidentally, it is generally known that a certain level of skill is required to accurately grasp a part of the subject P depicted in an ultrasound image by checking the ultrasound image. It is also known that the image quality of an ultrasound image generated by an ultrasound diagnostic device is greatly influenced by the technique of the operator.

[0066] For example, when an ultrasound image is taken in a remote location outside a hospital, such as in home nursing, the operator who operates the ultrasound probe to take the ultrasound image may be different from the observer, such as a doctor, who observes the taken ultrasound image and makes a diagnosis. In this case, the operator usually needs to operate the ultrasound probe while checking the obtained ultrasound image himself, and take an ultrasound image of a target part in the subject P. In particular, when the operator is less skilled, it may be difficult for the operator to judge whether the target part of the subject P is accurately observed. In addition, an operator with less skill may not be able to operate the ultrasound probe using appropriate techniques, resulting in an ultrasound image of low image quality.

[0067] In addition, an observer located remotely from the subject P and the operator makes a diagnosis by checking the ultrasound images captured by the operator of the ultrasound diagnostic device. However, since the observer cannot grasp how the operator captures the ultrasound images, it can be difficult to accurately determine whether the ultrasound images were captured using appropriate techniques, particularly when the ultrasound images are captured by an operator with low skill levels.

[0068] According to the ultrasound system 1 according to the first embodiment of the present invention, the HMD monitor 36 and the external monitor 45 display the same ultrasound image U almost simultaneously, and furthermore, the external monitor 45 displays the field of view image C corresponding to the field of view in front of the operator in almost real time, so that, for example, even if the external device 4 is located at a remote location from the HMD 3, the observer observing the external monitor 45 can observe the field of view image C and the ultrasound image U captured at the examination site where the subject P and the operator are located in almost real time. As a result, for example, a highly skilled observer can give advice to the operator in real time, so that even if the operator located at a remote location from the observer has a low level of skill, an appropriate ultrasound image U can be obtained and the accuracy of ultrasound diagnosis can be improved.

[0069] Furthermore, according to the ultrasound system 1 according to the first embodiment of the present invention, for example, it is possible to allow an observer who is located remotely from the operator and has a low level of skill to confirm a field of view image C showing how a highly skilled operator operates the ultrasound probe 2, and an appropriate ultrasound image U corresponding to the field of view image C. In this way, the ultrasound system 1 according to the first embodiment of the present invention is also very useful from the viewpoint of education.

[0070] In addition, in the ultrasound system 1 according to the first embodiment of the present invention, the ultrasound image U is displayed on the HMD-side monitor 36 of the HMD 3, so the operator can check the ultrasound image U while checking the forward field of view. Furthermore, the field of view image C is captured by the camera unit 33 of the HMD 3, so the operator does not need to use his or her hands to capture the field of view image C. Therefore, the operator can perform a variety of examinations, such as operating the ultrasound probe 2 with one hand while performing the procedure of inserting a so-called puncture needle into the subject P with the other hand, without taking his or her eyes off the subject P.

[0071] Although it has been described that the image processing unit 32 of the HMD 3 and the image processing unit 42 of the external device 4 each provide a time stamp to the generated ultrasound image U, the signal processing unit 23 of the ultrasound probe 2 can provide a time stamp to the signal subjected to the envelope detection process instead of the image processing unit 32 of the HMD 3 and the image processing unit 42 of the external device 4 providing a time stamp to the ultrasound image U. In this case, for example, by sharing the time in the ultrasound probe 2 and the time in the HMD 3 with each other, the ultrasound image U generated by the image processing unit 32 of the HMD 3 and the field of view image C generated by the camera unit 33 based on the signal provided with the time stamp can be synchronized with each other, and the ultrasound image U generated by the image processing unit 42 of the external device 4 and the field of view image C generated by the camera unit 33 can be synchronized with each other.

[0072] Here, the time in the ultrasonic probe 2 and the time in the HMD 3 can be shared, for example, based on either the ultrasonic probe 2 or the HMD 3. Also, for example, when either the ultrasonic probe 2 or the HMD 3 is connected to the Internet, the time of the built-in clock may be set using a communication protocol such as NTP or NITZ.

[0073] In addition, the method of synchronizing the ultrasonic image U and the field of view image C with each other is not limited to the method using the time stamp described above. For example, as disclosed in Japanese Patent Application Laid-Open No. 2011-183056, when the timing of capturing the ultrasonic image U by the ultrasonic probe 2 and the timing of capturing the field of view image C by the camera unit 33 of the HMD 3 are synchronized, and further, when the time difference between the time when the ultrasonic image U is captured and the time when the field of view image C is captured is within a certain range, for example, within 0.1 seconds, the image synchronization unit 43 of the external device 4 regards the ultrasonic image U and the field of view image C as having been captured at the same time, and can synchronize the ultrasonic image U and the field of view image C with each other.

[0074] In addition, the image synchronization unit 43 of the external device 4 generates a composite image M in which the ultrasound image U and the field of view image C synchronized with each other are combined into one, and sends the generated composite image M to the display control unit 44. However, instead of generating the composite image M, the display control unit 44 can also send the ultrasound image U and the field of view image C synchronized with each other to the display control unit 44. In this case, the display control unit 44 performs a predetermined process on the ultrasound image U and the field of view image C sent from the image synchronization unit 43, and displays the ultrasound image U and the field of view image C synchronized with each other together on the external monitor 45, for example, as shown in FIG. 5. Therefore, the observer watching the external monitor 45 can observe the field of view image C and the ultrasound image U taken at the examination site where the subject P and the operator are located in almost real time.

[0075] Although not shown, for example, the HMD 3 may be provided with an eye camera unit for photographing the operator's eyes, and an eye movement detection unit for analyzing the eye image representing the operator's eyes photographed by the eye camera unit to detect the blinking and gaze of the operator. The detected blinking and gaze movement of the operator may be used, for example, as an input operation of the operator. For example, the number of blinks of the operator, the position of the gaze, etc. detected by the eye movement detection unit may be used as a trigger for starting and stopping the photographing of the visual field image C by the camera unit 33. In this way, by using the blinking and gaze movement of the operator as an input operation, the operator can perform input operations without using his / her hands even if both hands are occupied during the examination of the subject P.

[0076] Furthermore, the ultrasonic probe 2 and the HMD 3 are connected to each other by wireless communication, but for example, instead of being connected to each other by wireless communication, they may also be connected to each other by wired communication. 4, the external device 4 is illustrated as a so-called smartphone or a portable thin computer called a tablet, but the external device 4 is not limited to this. For example, the external device 4 may be a so-called notebook personal computer or a desktop personal computer.

[0077] 2, the transmission / reception circuit 22 has a beamformer 54 together with the amplifier 52 and the AD conversion unit 53, but the beamformer 54 may be disposed between the transmission / reception circuit 22 and the signal processing unit 23, rather than inside the transmission / reception circuit 22. In this case, the beamformer 54 can also be configured by the probe-side processor 27.

[0078] 6 and 7, external input information such as a cursor A that can be moved by an observer's input operation via the input device 47 of the external device 4 can be simultaneously displayed on the external monitor 45 of the external device 4 and the HMD-side monitor 36 of the HMD 3. In this case, for example, when the cursor A displayed on the external monitor 45 of the external device 4 is moved by an observer's input operation via the input device 47 of the external device 4, the cursor A displayed on the HMD-side monitor 36 of the HMD 3 is similarly moved. This allows, for example, more detailed information to be shared between the operator of the ultrasonic probe 2 and the HMD 3A and the observer located near the external device 4A. For example, a highly skilled observer who is observing the ultrasonic image U and the field of view image C on the external monitor 45 of the external device 4A can easily support the examination performed by the operator by indicating, for example, the position to be scanned by the ultrasonic probe 2 using the cursor A to an operator with a low level of skill of the ultrasonic probe 2 and the HMD 3A.

[0079] Furthermore, for example, when an input operation can be performed via the HMD 3 by the movement of the operator's line of sight or the like, a cursor A that can be moved by the operator's input operation can be simultaneously displayed on the HMD-side monitor 36 of the HMD 3 and the external monitor 45 of the external device 4. In this case, for example, a highly skilled operator can provide more easily and in detail education on ultrasound diagnosis to a less skilled observer positioned near the external device 4A. The shape of the cursor A is not limited to an arrow shape, but may be any shape such as a circle or a polygon.

[0080] Also, for example, audio data can be wirelessly communicated bidirectionally between the HMD 3 and the external device 4. Fig. 8 shows the configuration of an HMD 3A in a modification of the first embodiment of the present invention. The HMD 3A is similar to the HMD 3 shown in Fig. 1 except that a microphone 61 and a speaker 62 are added, an HMD control unit 37A is provided instead of the HMD control unit 37, and an HMD-side processor 39A is provided instead of the HMD-side processor 39. In the HMD 3A, the microphone 61 and the speaker 62 are connected to the HMD-side wireless communication unit 31.

[0081] 9 shows a configuration of an external device 4A in a modified example of the first embodiment of the present invention. The external device 4A is configured by adding a microphone 63 and a speaker 64 to the external device 4 shown in FIG. 1, providing an external control unit 46A instead of the external control unit 46, and providing an external device-side processor 48A instead of the external device-side processor 48. In the external device 4A, the microphone 63 and the speaker 64 are connected to the external wireless communication unit 41.

[0082] In a modification of the first embodiment of the present invention, audio data is transmitted and received bidirectionally between the HMD 3A and the external device 4A. For example, when an operator of the ultrasonic probe 2 and the HMD 3A speaks toward the HMD 3A, the spoken voice is input to a microphone 61 of the HMD 3A, and audio data is generated by the microphone 61. The generated audio data is wirelessly transmitted from the HMD side wireless communication unit 31 to the external device 4A. The external wireless communication unit 41 of the external device 4A receives the audio data wirelessly transmitted from the HMD 3A, and transmits the received audio data to a speaker 64. The speaker 64 reproduces the audio spoken by the operator of the ultrasonic probe 2 and the HMD 3A based on the audio data received from the external wireless communication unit 41.

[0083] Also, for example, when an observer observing an ultrasound image U and a visual field image C on the external monitor 45 of the external device 4A speaks toward the external device 4A, the spoken voice is input to a microphone 63 of the external device 4A, and voice data is generated by the microphone 63. The generated voice data is wirelessly transmitted from the external wireless communication unit 41 to the HMD 3A. The HMD-side wireless communication unit 31 of the HMD 3A receives the voice data wirelessly transmitted from the external device 4A, and sends the received voice data to a speaker 62. The speaker 62 reproduces the voice spoken by the observer located near the external device 4A based on the voice data received from the HMD-side wireless communication unit 31.

[0084] In this way, by transmitting and receiving audio data bidirectionally between the HMD 3A and the external device 4A, more detailed information can be shared between the operator of the ultrasound probe 2 and HMD 3A and the observer located near the external device 4A. For example, a highly skilled observer observing the ultrasound image U and the visual field image C on the external monitor 45 of the external device 4A can give more easy and detailed advice to the less skilled operator of the ultrasound probe 2 and HMD 3A. Also, for example, a highly skilled operator can provide more easy and detailed education on ultrasound diagnosis to a less skilled observer located near the external device 4A.

[0085] Also, the voice of the operator inputted to the microphone 61 of the HMD 3A can be used as an input operation of the operator. For example, the HMD control unit 37A can obtain instruction information by analyzing voice data generated based on the voice of the operator by the microphone 61, and control each part of the HMD 3A, such as starting and stopping the shooting of the field of view image C by the camera unit 33, according to the obtained instruction information. Furthermore, the ultrasonic probe 2 can be controlled based on the voice data analyzed by the HMD control unit 37A. In this case, for example, the voice data analyzed by the HMD control unit 37A is wirelessly transmitted from the HMD side wireless communication unit 31 to the ultrasonic probe 2 as input information from the operator, and inputted to the probe control unit 26 via the probe side wireless communication unit 24. The probe control unit 26 can control each part of the ultrasonic probe 2, such as starting and stopping the transmission of ultrasonic waves by the transducer array 21, and starting and stopping the shooting of the field of view image C by the camera unit 33, based on the input information.

[0086] Also, the observer's voice input to the microphone 63 of the external device 4A can be used as an input operation by the observer. For example, the external control unit 46A can acquire instruction information by analyzing voice data generated by the microphone 63 based on the observer's voice, and control each part of the head mounted display 3A, such as starting and stopping the capture of the visual field image C by the camera unit 33 of the head mounted display 3A, and control each part of the ultrasonic probe 2, such as starting and stopping the transmission of ultrasonic waves by the transducer array 21 of the ultrasonic probe 2, according to the acquired instruction information.

[0087] Embodiment 2 In embodiment 1, in the ultrasonic probe 2, pre-imaging received data is generated by performing envelope detection processing on the sound ray signal, and the generated pre-imaging received data is wirelessly transmitted to the HMD 3 and the external device 4. However, an ultrasonic image U can also be generated in the ultrasonic probe 2, and the generated ultrasonic image U can also be wirelessly transmitted to the HMD 3 and the external device 4.

[0088] Fig. 10 shows the configuration of an ultrasound system 1B according to embodiment 2 of the present invention. The ultrasound system 1 is the same as the ultrasound system 1 of embodiment 1 shown in Fig. 1 except that an ultrasound probe 2A is provided instead of the ultrasound probe 2, an HMD 3B is provided instead of the HMD 3, and an external device 4B is provided instead of the external device 4.

[0089] The ultrasonic probe 2B is obtained by adding an image processing unit 71 to the ultrasonic probe 2 in the first embodiment, providing a probe control unit 26B instead of the probe control unit 26, and providing a probe-side processor 27B instead of the probe-side processor 27. In the ultrasonic probe 2B, the image processing unit 71 is connected to the signal processing unit 23. In addition, the probe-side wireless communication unit 24 and the probe control unit 26B are connected to the image processing unit 71. Although not shown, the signal processing unit 23 and the image processing unit 71 configure an ultrasonic image generating unit.

[0090] The HMD 3B is obtained by removing the image processing unit 32 from the HMD 3 in the first embodiment, providing an HMD control unit 37B instead of the HMD control unit 37, and providing an HMD-side processor 39B instead of the HMD-side processor 39. In the HMD 3B, a display control unit 35 and a camera unit 33 are connected to the HMD-side wireless communication unit 31. The external device 4B is similar to the external device 4 in embodiment 1 in that the image processing unit 42 has been removed, the external control unit 46 has been replaced with an external control unit 46B, and the external device side processor 48 has been replaced with an external device side processor 48B.

[0091] The image processing unit 71 of the ultrasonic probe 2B raster-converts the signal that has been subjected to the envelope detection process by the signal processing unit 23 into an image signal conforming to the scanning method of a normal television signal, and performs various necessary image processes such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction on the converted image signal to generate an ultrasonic image U conforming to the display format for the HMD-side monitor 36 of the HMD 3B and an ultrasonic image U conforming to the format for the external monitor 45 of the external device 4B. Furthermore, the image processing unit 71 wirelessly transmits the ultrasonic image U conforming to the display format for the HMD-side monitor 36 of the HMD 3B from the probe-side wireless communication unit 24 to the HMD 3B, and wirelessly transmits the ultrasonic image U conforming to the format for the external monitor 45 of the external device 4B from the probe-side wireless communication unit 24 to the external device 4B.

[0092] The HMD wireless communication unit 31 of the HMD 3B receives the ultrasound image U wirelessly transmitted from the ultrasound probe 2B, and sends the received ultrasound image U to the display control unit . The display control unit 35 performs a predetermined processing on the ultrasound image U sent from the HMD side wireless communication unit 31, and then sends the ultrasound image U to the HMD side monitor 36, and displays the ultrasound image U on the HMD side monitor 36, as shown in Figure 4.

[0093] The external wireless communication unit 41 of the external device 4B receives the ultrasound image U wirelessly transmitted from the ultrasound probe 2B and the field of view image C wirelessly transmitted from the HMD 3B, and sends the received ultrasound image U and field of view image C to the image synchronization unit 43. The image synchronization unit 43 synchronizes the ultrasound image U and the field of view image C sent from the external wireless communication unit 41 with each other, and generates a composite image M based on the ultrasound image U and the field of view image C that are synchronized with each other. The display control unit 44 performs a predetermined processing on the composite image M generated by the image synchronization unit 43, and then transmits the composite image M to the external monitor 45, and displays the ultrasound image U and the field of view image C, which are synchronized with each other, together on the external monitor 45, as shown in FIG. 5.

[0094] As described above, according to the ultrasound system 1B of the second embodiment of the present invention, even if the ultrasound probe 2B includes the image processor 71, the same ultrasound image U is displayed on the HMD side monitor 36 and the external monitor 45 almost simultaneously, as in the ultrasound system 1 of the first embodiment in which the HMD 3 includes the image processor 32 and the external device 4 includes the image processor 42, and further, the field of view image C corresponding to the field of view in front of the operator of the ultrasound probe 2B and the HMD 3B is displayed on the external monitor 45 in almost real time. Therefore, for example, an observer who observes the field of view image C and the ultrasound image U by the external device 4B located at a remote location can give advice to the operator of the ultrasound probe 2B and the HMD 3B, so that an appropriate ultrasound image U can be obtained and the accuracy of ultrasound diagnosis can be improved.

[0095] 1, the HMD 3 is provided with an image processor 32 and the external device 4 is provided with an image processor 42, but in the ultrasound system 1B according to the second embodiment, the ultrasound probe 2B is provided with an image processor 71, so that the HMD 3B and the external device 4B do not need to have the image processors 32 and 42, respectively, and the internal configurations of the HMD 3B and the external device 4B are simplified compared to the internal configurations of the HMD 3 and the external device 4 in the ultrasound system 1 according to the first embodiment. Therefore, according to the ultrasound system 1B according to the second embodiment, it is possible to reduce the power consumption and calculation load of the HMD 3B and the external device 4B compared to the ultrasound system 1 according to the first embodiment.

[0096] Third embodiment In the first embodiment, the ultrasound image U and the field of view image C are synchronized in the external device 4. However, for example, the ultrasound image U and the field of view image C can also be synchronized in the HMD 3.

[0097] Fig. 11 shows the configuration of an ultrasound system 1C according to the third embodiment of the present invention. The ultrasound system 1 is the ultrasound system 1 of the first embodiment shown in Fig. 1, except that it is provided with an ultrasound probe 2C having the same internal configuration as the ultrasound probe 2, an HMD 3C instead of the HMD 3, and an external device 4C instead of the external device 4. The ultrasound probe 2C is connected only to the HMD 3C via wireless communication, and the external device 4C is connected only to the HMD 3C via wireless communication.

[0098] The HMD 3C is obtained by adding an image synchronization unit 34 to the HMD 3 in the first embodiment, providing an HMD control unit 37C instead of the HMD control unit 37, and providing an HMD-side processor 39C instead of the HMD-side processor 39. In the HMD 3C, the image synchronization unit 34 is connected to the HMD-side wireless communication unit 31, the image processing unit 32, and the camera unit 33. In addition, a display control unit 35 is connected to the image synchronization unit 34.

[0099] The external device 4C is different from the external device 4 in the first embodiment in that the image processing unit 42 and the image synchronization unit 43 are removed, an external control unit 46C is provided instead of the external control unit 46, and an external device-side processor 48C is provided instead of the external device-side processor 48. In the external device 4C, a display control unit 44 is connected to the external wireless communication unit 41.

[0100] The probe side wireless communication unit 24 of the ultrasonic probe 2C wirelessly transmits the received data, which has been subjected to envelope detection processing by the signal processing unit 23 and has not yet been imaged, only to the HMD 3C. The HMD side wireless communication unit 31 of the HMD 3 C receives the pre-imaging reception data wirelessly transmitted from the ultrasound probe 2 C, and sends the received pre-imaging reception data to the image processing unit 32 .

[0101] The image processing unit 32 raster-converts the received data before imaging sent from the HMD-side wireless communication unit 31 into an image signal conforming to a normal television signal scanning method, and performs various necessary image processing such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction on the converted image signal to generate an ultrasound image U conforming to a display format for the HMD-side monitor 36 of the HMD 3B and an ultrasound image U conforming to a format for the external monitor 45 of the external device 4B. Furthermore, the image processing unit 32 sends the ultrasound image U conforming to the display format for the HMD-side monitor 36 of the HMD 3B to the display control unit 35 via the image synchronization unit 34, and sends the ultrasound image U conforming to the format for the external monitor 45 of the external device 4B to the image synchronization unit 34.

[0102] The display control unit 35 performs a predetermined processing on the ultrasound image U conforming to the display format for the HMD side monitor 36, which is sent from the image processing unit 32 via the image synchronization unit 34, and then displays the ultrasound image U on the HMD side monitor 36, as shown in FIG. 4.

[0103] Furthermore, the camera unit 33 acquires a visual field image C obtained by imaging a location on the subject P scanned by the ultrasonic probe 2C, and sends the acquired visual field image C to the image synchronization unit . The image synchronization unit 34 synchronizes the ultrasound image U sent from the image processing unit 32 and the field of view image C sent from the camera unit 33 with each other, and generates a composite image M based on the ultrasound image U and the field of view image C that are synchronized with each other. Furthermore, the image synchronization unit 34 sends to the HMD side wireless communication unit 31 a composite image M generated based on the ultrasound image U and the field of view image C that conforms to a display format for the external monitor 45 of the external device 4C.

[0104] The HMD side wireless communication section 31 wirelessly transmits the composite image M sent from the image synchronization section 34 to the external device 4C. The external wireless communication unit 41 of the external device 4C receives the composite image M wirelessly transmitted from the HMD 3C, and sends the received composite image M to the display control unit 44. The display control unit 44 performs a predetermined process on the composite image M sent from the external wireless communication unit 41, and then sends the composite image M to the external monitor 45, and causes the external monitor 45 to display the ultrasound image U and the field of view image C, which are synchronized with each other, together, as shown in FIG.

[0105] From the above, according to the ultrasound system 1C according to the third embodiment of the present invention, even if the image processor 32 and the image synchronizer 34 are provided only in the HMD 3C, the same ultrasound image U is displayed almost simultaneously on the HMD side monitor 36 and the external monitor 45, similar to the ultrasound system 1 according to the first embodiment in which the HMD 3 includes the image processor 32 and the external device 4 includes the image processor 42 and the image synchronizer 43, and further, the field of view image C corresponding to the field of view in front of the operator of the ultrasound probe 2C and the HMD 3C is displayed almost in real time on the external monitor 45. Therefore, for example, an observer who observes the field of view image C and the ultrasound image U by the external device 4C arranged at a remote location can give advice to the operator of the ultrasound probe 2C and the HMD 3C, so that an appropriate ultrasound image U can be obtained and the accuracy of ultrasound diagnosis can be improved.

[0106] 1, the external device 4 includes an image processor 42 and an image synchronizer 43, but in the ultrasound system 1C according to the third embodiment, the HMD 3C transmits a composite image M generated based on the ultrasound image U and the field of view image C to the external device 4C wirelessly, so that the external device 4C does not need to include the image processor 42 and the image synchronizer 43, and the internal configuration of the external device 4C is simpler than that of the external device 4 according to the first embodiment. Therefore, the ultrasound system 1C according to the third embodiment can reduce the power consumption and calculation load of the external device 4C.

[0107] Fourth embodiment In embodiment 3, the received data before imaging, which has been subjected to envelope detection processing by the signal processing unit 23 of the ultrasonic probe 2, is wirelessly transmitted to the HMD 3 and the external device 4, but the ultrasonic image U can also be generated in the ultrasonic probe 2.

[0108] Fig. 12 shows the configuration of an ultrasound system 1D according to the fourth embodiment of the present invention. The ultrasound system 1D is the same as the ultrasound system 1C according to the third embodiment shown in Fig. 11 except that an ultrasound probe 2D is provided instead of the ultrasound probe 2C, an HMD3D is provided instead of the HMD3C, and an external device 4D is provided instead of the external device 4C. The ultrasound probe 2D is connected only to the HMD3D via wireless communication, and the external device 4D is connected only to the HMD3D via wireless communication.

[0109] The ultrasonic probe 2D is obtained by adding an image processing unit 81 to the ultrasonic probe 2C in the third embodiment, providing a probe control unit 26D instead of the probe control unit 26, and providing a probe-side processor 27D instead of the probe-side processor 27. In the ultrasonic probe 2D, the image processing unit 81 is connected to the signal processing unit 23, and the probe-side wireless communication unit 24 and the probe control unit 26D are connected to the image processing unit 81. Although not shown, the signal processing unit 23 and the image processing unit 81 configure an ultrasonic image generating unit.

[0110] The HMD3D is obtained by removing the image processing unit 32 from the HMD3C in the third embodiment, providing an HMD control unit 37D instead of the HMD control unit 37C, and providing an HMD-side processor 39D instead of the HMD-side processor 39C. In the HMD3D, an image synchronization unit 34 is connected to the HMD-side wireless communication unit 31. Also, the camera unit 33 is connected to the image synchronization unit 34.

[0111] The external device 4D is configured such that an external control unit 46D is provided instead of the external control unit 46C of the external device 4C in the third embodiment, and an external device-side processor 48D is provided instead of the external device-side processor 48C.

[0112] The image processing unit 81 of the ultrasonic probe 2D raster-converts the signal that has been subjected to the envelope detection processing by the signal processing unit 23 into an image signal conforming to the scanning method of a normal television signal, and performs various necessary image processing such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction on the converted image signal to generate an ultrasonic image U conforming to the display format for the HMD-side monitor 36 of the HMD 3D and an ultrasonic image U conforming to the format for the external monitor 45 of the external device 4D. In addition, the image processing unit 81 transmits these generated ultrasonic images U to the probe-side wireless communication unit 24.

[0113] The probe side wireless communication unit 24 wirelessly transmits the ultrasound image U sent from the image processing unit 81 to the HMD 3D. The HMD-side wireless communication unit 31 receives the ultrasound image U wirelessly transmitted from the ultrasound probe 2D. In addition, the HMD-side wireless communication unit 31 transmits the ultrasound image U conforming to the display format for the HMD-side monitor 36 to the display control unit 35 via the image synchronization unit 34, and transmits the ultrasound image U conforming to the format for the external monitor 45 to the image synchronization unit 34.

[0114] The display control unit 35 performs a predetermined processing on the ultrasound image U sent from the HMD side wireless communication unit 31 via the image synchronization unit 34, and then displays the ultrasound image U on the HMD side monitor 36, as shown in Figure 4.

[0115] The camera unit 33 acquires a field of view image C obtained by imaging a location on the subject P scanned by the ultrasonic probe 2D, and sends the acquired field of view image C to an image synchronization unit . The image synchronization unit 34 synchronizes the ultrasound image U sent from the HMD side wireless communication unit 31 and the field of view image C sent from the camera unit 33 with each other, and generates a composite image M based on the ultrasound image U and the field of view image C that are synchronized with each other.

[0116] The image synchronization unit 34 sends to the HMD side wireless communication unit 31 a composite image M generated based on the ultrasound image U and the field of view image C that comply with a format for the external monitor 45 that are synchronized with each other. The HMD side wireless communication section 31 wirelessly transmits the composite image M sent from the image synchronization section 34 to the external device 4D.

[0117] The external wireless communication unit 41 of the external device 4D receives the composite image M wirelessly transmitted from the HMD 3D, and sends the received composite image M to the display control unit . The display control unit 44 performs a predetermined processing on the composite image M sent from the external wireless communication unit 41, and then sends the composite image M to the external monitor 45, and displays the ultrasound image U and the field of view image C, which are synchronized with each other, together on the external monitor 45, as shown in Figure 5.

[0118] From the above, according to the ultrasound system 1D of the fourth embodiment, even if the image processor 81 is provided only in the ultrasound probe 2D and the image synchronization unit 34 is provided only in the HMD3D, the same ultrasound image U is displayed almost simultaneously on the HMD side monitor 36 and the external monitor 45, similar to the ultrasound system 1 of the first embodiment in which the HMD3C has the image processor 32 and the external device 4 has the image processor 42 and the image synchronization unit 43, and further, the field of view image C corresponding to the field of view in front of the operator of the ultrasound probe 2C and the HMD3C is displayed almost in real time on the external monitor 45. Therefore, for example, an observer who observes the field of view image C and the ultrasound image U by the external device 4D located at a remote location can give advice to the operator of the ultrasound probe 2D and the HMD3D, so that an appropriate ultrasound image U can be obtained and the accuracy of ultrasound diagnosis can be improved.

[0119] Fifth embodiment In embodiment 3, the external device 4C receives a composite image M from the HMD 3C and displays the received composite image M on the external monitor 45. Therefore, the position and size of the ultrasound image U and the field of view image C displayed on the external monitor 45 could not be freely changed on the external device 4C side. However, according to the ultrasound diagnostic device 1E of embodiment 5 shown in FIG. 13, the position and size of the ultrasound image U and the field of view image C displayed on the external monitor 45 can be arbitrarily changed on the external device 4E side.

[0120] 13 shows the configuration of an ultrasonic system 1E according to the fifth embodiment of the present invention. The ultrasonic system 1E is the ultrasonic system 1C according to the third embodiment, except that it is provided with an ultrasonic probe 2E having the same internal configuration as the ultrasonic probe 2C, an HMD3E instead of the HMD3C, and an external device 4E instead of the external device 4C. The ultrasonic probe 2E is connected only to the HMD3E by wireless communication, and the external device 4E is connected only to the HMD3E by wireless communication.

[0121] The HMD 3E is the HMD 3C in the third embodiment, except that an HMD control unit 37E is provided instead of the HMD control unit 37, and an HMD-side processor 39E is provided instead of the HMD-side processor 39. In the HMD 3E, an image synchronization unit 34 is connected to the HMD-side wireless communication unit 31, the image processing unit 32, and the camera unit 33, and a display control unit 35 is connected to the image synchronization unit 34.

[0122] The external device 4E is configured such that an external control unit 46E is provided instead of the external control unit 46 of the external device 4C in the third embodiment, and an external device-side processor 48E is provided instead of the external device-side processor 48.

[0123] The probe side wireless communication unit 24 of the ultrasonic probe 2E wirelessly transmits the received data, which has been subjected to envelope detection processing by the signal processing unit 23 and has not yet been imaged, to the HMD 3E. The HMD side wireless communication unit 31 of the HMD 3E receives the pre-imaging reception data wirelessly transmitted from the ultrasound probe 2E, and sends the received pre-imaging reception data to the image processing unit 32.

[0124] The image processing unit 32 generates an ultrasound image U conforming to the format for the HMD-side monitor 36 of the HMD 3E and an ultrasound image U conforming to the display format for the external monitor 45 of the external device 4E, based on the received data before imaging sent from the HMD-side wireless communication unit 31. In addition, the image processing unit 32 sends the ultrasound image U conforming to the format for the HMD-side monitor 36 to the display control unit 35 via the image synchronization unit 34, and sends the ultrasound image U conforming to the format for the HMD-side monitor 36 to the image synchronization unit 34.

[0125] The display control unit 35 performs a predetermined process on the ultrasound image U sent from the image processing unit 32 via the image synchronization unit 34, and then displays the ultrasound image U on the HMD side monitor 36, as shown in FIG.

[0126] Furthermore, the camera unit 33 acquires a visual field image C obtained by capturing an image of a spot on the subject P scanned by the ultrasonic probe 2E, and sends the acquired visual field image C to the image synchronization unit . The image synchronization unit 34 synchronizes the ultrasound image U sent from the HMD side wireless communication unit 31 and the visual field image C sent from the camera unit 33 with each other.

[0127] In addition, the image synchronization unit 34 transmits the ultrasound image U and the field of view image C to the HMD side wireless communication unit 31, respectively, without generating a single composite image M based on the ultrasound image U and the field of view image C that conform to the format for the external monitor 45 that are synchronized with each other. The HMD side wireless communication unit 31 wirelessly transmits the ultrasound image U and the visual field image C sent from the image synchronization unit 34 to the external device 4E. The external wireless communication unit 41 of the external device 4E receives the ultrasound image U and the field of view image C wirelessly transmitted from the HMD 3E, and sends the received ultrasound image U and field of view image C to the display control unit 44, respectively.

[0128] The display control unit 44 performs predetermined processing on the ultrasound image U and the field of view image C sent from the external wireless communication unit 41, and displays the ultrasound image U and the field of view image C, which are synchronized with each other, together on the external monitor 45. Here, the arrangement and size of the ultrasound image U and the field of view image C displayed on the external monitor 45 can be adjusted by the observer's input operation via the input device 47. For example, when the observer inputs instruction information to adjust the arrangement and size of the ultrasound image U and the field of view image C on the external monitor 45 via the input device 47, the input instruction information is input to the display control unit 44 via the external control unit 46E. Based on the input instruction information, the display control unit 44 displays the ultrasound image U and the field of view image C synchronized with each other, for example, with the arrangement and size as shown in FIG. 14. In the example shown in FIG. 14, the external device 4E, the ultrasound image U, and the field of view image C are rotated 90 degrees compared to the example shown in FIG. 5, and the ultrasound image U and the field of view image C are displayed on the external monitor 45 so that the field of view image C is superimposed on a part of the ultrasound image U.

[0129] As described above, according to the ultrasound system 1E of embodiment 5 of the present invention, the position and size of the ultrasound image U and field of view image C displayed on the external monitor 45 of the external device 4E can be adjusted, so that an observer observing the ultrasound image U and field of view image C displayed on the external monitor 45 can view the ultrasound image U and field of view image C more clearly according to his or her preferences.

[0130] Sixth embodiment In embodiment 5, the received data before imaging, which has been subjected to envelope detection processing by the signal processing unit 23, is wirelessly transmitted to the HMD 3E by the probe side wireless communication unit 24, but an ultrasound image U can also be generated in the ultrasound probe 2.

[0131] Fig. 15 shows the configuration of an ultrasound system 1F according to the sixth embodiment of the present invention. The ultrasound system 1F is a configuration in which an ultrasound probe 2F is provided instead of the ultrasound probe 2E, an HMD 3F is provided instead of the HMD 3E, and an external device 4F is provided instead of the external device 4E in the ultrasound system 1E according to the fifth embodiment shown in Fig. 13. The ultrasound probe 2F is connected only to the HMD 3F by wireless communication, and the external device 4F is connected only to the HMD 3F by wireless communication.

[0132] The ultrasonic probe 2F is obtained by adding an image processing unit 91 to the ultrasonic probe 2E in the fifth embodiment, providing a probe control unit 26E instead of the probe control unit 26, and providing a probe-side processor 27E instead of the probe-side processor 27. In the ultrasonic probe 2F, the image processing unit 91 is connected to the signal processing unit 23, and the probe-side wireless communication unit 24 and the probe control unit 26F are connected to the image processing unit 91. Although not shown, the signal processing unit 23 and the image processing unit 91 configure an ultrasonic image generating unit.

[0133] The HMD 3F is obtained by removing the image processing unit 32 from the HMD 3E in the fifth embodiment, providing an HMD control unit 37F instead of the HMD control unit 37E, and providing an HMD-side processor 39F instead of the HMD-side processor 39E. In the HMD 3F, an image synchronization unit 34 is connected to the HMD-side wireless communication unit 31. Also, the camera unit 33 is connected to the image synchronization unit 34.

[0134] The external device 4F is configured such that an external control unit 46F is provided instead of the external control unit 46E of the external device 4E in the fifth embodiment, and an external device-side processor 48F is provided instead of the external device-side processor 48E.

[0135] The image processing unit 91 of the ultrasonic probe 2F raster-converts the signal that has been subjected to the envelope detection process by the signal processing unit 23 into an image signal conforming to the scanning method of a normal television signal, and performs various necessary image processes such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction on the converted image signal to generate an ultrasonic image U conforming to the display format for the HMD-side monitor 36 of the HMD 3F and an ultrasonic image U conforming to the format for the external monitor 45 of the external device 4F. In addition, the image processing unit 91 transmits these generated ultrasonic images U to the probe-side wireless communication unit 24.

[0136] The HMD-side wireless communication unit 31 receives the ultrasound image U wirelessly transmitted from the ultrasound probe 2F. In addition, the HMD-side wireless communication unit 31 transmits the ultrasound image U conforming to the display format for the HMD-side monitor 36 to the display control unit 35 via the image synchronization unit 34, and transmits the ultrasound image U conforming to the format for the external monitor 45 to the image synchronization unit 34. The display control unit 35 performs a predetermined processing on the ultrasound image U sent from the HMD side wireless communication unit 31 via the image synchronization unit 34, and then displays the ultrasound image U on the HMD side monitor 36, as shown in Figure 4.

[0137] Furthermore, the camera unit 33 acquires a field of view image C obtained by capturing an image of a portion of the subject P scanned by the ultrasonic probe 2F, and transmits the acquired field of view image C to the image synchronization unit . The image synchronization unit 34 synchronizes the ultrasound image U sent from the HMD side wireless communication unit 31 and the visual field image C sent from the camera unit 33 with each other.

[0138] In addition, the image synchronization unit 34 transmits the ultrasound image U and the field of view image C to the HMD side wireless communication unit 31, respectively, without generating a single composite image M based on the ultrasound image U and the field of view image C that conform to the format for the external monitor 45 that are synchronized with each other. The HMD side wireless communication unit 31 wirelessly transmits the ultrasound image U and the field of view image C sent from the image synchronization unit 34 to the external device 4F.

[0139] The external wireless communication unit 41 of the external device 4F receives the ultrasound image U and the field of view image C wirelessly transmitted from the HMD 3F, and sends the received ultrasound image U and field of view image C to the display control unit 44, respectively. The display control unit 44 performs predetermined processing on the ultrasound image U and the field of view image C sent from the external wireless communication unit 41, and displays the ultrasound image U and the field of view image C, which are synchronized with each other, together on the external monitor 45.

[0140] At this time, the display control unit 44 can adjust the arrangement and size of the ultrasound image U and the field of view image C displayed on the external monitor 45 according to an input operation by the observer via the input device 47. As a result, for example, as shown in FIG. 14, the ultrasound image U and the field of view image C that are synchronized with each other are displayed together on the external monitor 45.

[0141] As described above, according to the ultrasound system 1F of embodiment 6 of the present invention, even if the ultrasound probe 2F is equipped with an image processing unit 91, the position and size of the ultrasound image U and field of view image C displayed on the external monitor 45 of the external device 4F can be adjusted, so that an observer observing the ultrasound image U and field of view image C displayed on the external monitor 45 can view the ultrasound image U and field of view image C more clearly according to his or her preferences. [Explanation of symbols]

[0142] 1, 1B, 1C, 1D, 1E, 1F Ultrasound system, 2, 2B, 2C, 2D, 2E, 2F Ultrasound probe, 3, 3A, 3B, 3C, 3D, 3E, 3F HMD, 4, 4A, 4B, 4C, 4D, 4E, 4F External device, 21 Transducer array, 22 Transmitter / receiver circuit, 23 Signal processing unit, 24 Probe side wireless communication unit, 26, 26B, 26D, 26F Probe control unit, 27, 27B, 27D, 27F Probe side processor, 31 HMD side wireless communication unit, 32, 42, 71, 81, 91 Image processing unit, 33 Camera unit, 34, 43 Image synchronization unit, 35, 44 Display control unit, 36 HMD side monitor, 36A, 36B Monitor, 37, 37A, 37B, 37C, 37D, 37E, 37F HMD control unit, 39, 39A, 39B, 39C, 39D, 39E, 39F HMD side processor, 41 external wireless communication unit, 45 external monitor, 46, 46A, 46B, 46C, 46D, 4E6, 46F external control unit, 47 input device, 48, 48A, 48B, 48C, 48D, 48E, 48F external device side processor, 51 pulsar, 52 amplifier, 53 AD conversion unit, 54 beamformer, 61, 63 microphone, 62, 64 speaker, A cursor, B bridge unit, C field of view image, D storage unit, F imaging lens, P subject, T tendon unit, U ultrasound image.

Claims

1. An ultrasound system comprising an ultrasound probe, a head mounted display, and an external device, The ultrasonic probe includes: A transducer array; a transmission / reception circuit that transmits ultrasonic waves from the transducer array and generates a sound ray signal based on a reception signal acquired by the transducer array; an ultrasonic image generating unit that generates an ultrasonic image based on the sound ray signal generated by the transmission and reception circuit; a probe-side wireless communication unit that wirelessly transmits the ultrasound image; Including, The head mounted display includes: a camera unit for acquiring a visual field image obtained by imaging a portion of a subject to be scanned by the ultrasonic probe; an image synchronization unit that synchronizes the ultrasound image and the field of view image captured at the same timing by referring to a time stamp of the ultrasound image wirelessly transmitted from the ultrasound probe and a time stamp of the field of view image acquired by the camera unit; a head mounted display side wireless communication unit that wirelessly transmits the ultrasound image and the field of view image that are synchronized with each other by the image synchronization unit; Including, The external device is An external wireless communication unit wirelessly connected to at least the head mounted display side wireless communication unit; An external monitor and a display control unit that displays, on the external monitor, the ultrasound image and the field of view image that are synchronized with each other by the image synchronization unit and wirelessly transmitted from the head mounted display; 4. An ultrasound system comprising:

2. the head mounted display includes a head mounted display side monitor, The ultrasound system of claim 1 , wherein the ultrasound image is displayed on a monitor on the head-mounted display side.

3. The external device includes an input device, the external wireless communication unit wirelessly transmits external input information input via the input device to the head mounted display side wireless communication unit; The ultrasound system according to claim 2 , wherein the external input information is displayed on a monitor on the head mounted display side.

4. 4. The ultrasound system according to claim 1, wherein audio data is wirelessly transmitted in a bidirectional manner between the head-mounted display wireless communication unit and the external wireless communication unit.

5. A method for controlling an ultrasound system including an ultrasound probe, a head mounted display, and an external device, comprising: In the ultrasonic probe, Transmitting ultrasonic waves from a transducer array of the ultrasonic probe and generating a sound ray signal based on a reception signal acquired by the transducer array; generating an ultrasound image based on the generated sound ray signals; wirelessly transmitting the ultrasound image; In the head mounted display, acquiring a visual field image of a portion of a subject scanned by the ultrasonic probe; By referring to a time stamp of the ultrasound image wirelessly transmitted from the ultrasound probe and a time stamp of the acquired field of view image, the ultrasound image and the field of view image, which are taken at the same timing, are synchronized with each other; wirelessly transmitting the acquired field of view image; In the external device, The ultrasound image and the field of view image, which are synchronized with each other and wirelessly transmitted from the head mounted display, are displayed on an external monitor. A method for controlling an ultrasound system.

6. An ultrasound system comprising an ultrasound probe, a head mounted display, and an external device, The ultrasonic probe includes: A transducer array; a transmission / reception circuit that transmits ultrasonic waves from the transducer array and generates a sound ray signal based on a reception signal acquired by the transducer array; a reception data generating unit that generates reception data before imaging by performing signal processing on the sound ray signal generated by the transmission and reception circuit; a probe-side wireless communication unit that wirelessly transmits the received data; Including, The head mounted display includes: a camera unit for acquiring a visual field image obtained by imaging a portion of a subject to be scanned by the ultrasonic probe; an image synchronization unit that synchronizes the ultrasound image and the field of view image captured at the same timing by referring to a time stamp of the ultrasound image generated based on the reception data wirelessly transmitted from the ultrasound probe and a time stamp of the field of view image acquired by the camera unit; a head mounted display side wireless communication unit that wirelessly transmits the field of view image acquired by the camera unit; Including, The external device is An external wireless communication unit wirelessly connected to at least the head mounted display side wireless communication unit; An external monitor and a display control unit that displays, on the external monitor, the ultrasound image and the field of view image that are synchronized with each other by the image synchronization unit and wirelessly transmitted from the head mounted display; 4. An ultrasound system comprising:

7. The head mounted display includes an image processing unit that generates an ultrasound image based on the reception data wirelessly transmitted from the probe side wireless communication unit. The ultrasound system of claim 6.

8. the head mounted display includes a head mounted display side monitor, The ultrasound system of claim 7 , wherein the ultrasound image is displayed on a monitor on the head-mounted display side.

9. The external device includes an input device, the external wireless communication unit wirelessly transmits external input information input via the input device to the head mounted display side wireless communication unit; The ultrasound system according to claim 8 , wherein the external input information is displayed on a monitor on the head mounted display side.

10. 10. The ultrasound system according to claim 6, wherein audio data is wirelessly transmitted in a bidirectional manner between the head-mounted display wireless communication unit and the external wireless communication unit.

11. A method for controlling an ultrasound system including an ultrasound probe, a head mounted display, and an external device, comprising: In the ultrasonic probe, Transmitting ultrasonic waves from a transducer array of the ultrasonic probe and generating a sound ray signal based on a reception signal acquired by the transducer array; generating reception data before imaging by performing signal processing on the generated sound ray signal; wirelessly transmitting the received data; In the head mounted display, acquiring a visual field image of a portion of a subject scanned by the ultrasonic probe; By referring to a time stamp of an ultrasound image generated based on the reception data wirelessly transmitted from the ultrasound probe and a time stamp of the acquired field of view image, the ultrasound image and the field of view image, which are taken at the same timing, are synchronized with each other; wirelessly transmitting the acquired field of view image; In the external device, The ultrasound image and the field of view image, which are synchronized with each other and wirelessly transmitted from the head mounted display, are displayed on an external monitor. A method for controlling an ultrasound system.

Images