Ultrasonic diagnostic apparatus

Abstract

The present disclosure relates to an ultrasonic diagnostic apparatus. A camera (24) is provided at a display (16). A representative coordinate of an examinee is calculated by analysis of a camera image (S12). The orientation of the display (16) is controlled so that the representative coordinate enters a target region in the camera image (S18). Based on the camera image, the position and the posture of the display (16) can also be controlled so that an illumination shot is not taken (S20).

Description

Technical Field

[0001] This disclosure relates to an ultrasound diagnostic apparatus, and in particular to the control of the orientation of the display. Background Technology

[0002] An ultrasound diagnostic device is a medical device that forms an ultrasound image by transmitting ultrasound waves to a subject (biological organism) and receiving the reflected waves from the subject's interior. Various ultrasound diagnostic devices are commercially available. Here, a trolley-type ultrasound diagnostic device will be described. The main body of the ultrasound diagnostic device supports the operation panel via a lifting mechanism. A support mechanism is provided on a base located inside the operation panel, holding the display screen. The support mechanism is a multi-joint mechanism. The examiner (user) can change the position and orientation of the display screen to their preferred position and orientation as needed.

[0003] Document 1 (Japanese Patent Application Publication No. 2007-21088) discloses an ultrasound diagnostic device that has the function of automatically positioning a display at a preset position.

[0004] The examiner's head position changes constantly during the ultrasound examination, specifically during probe manipulation. To easily observe the ultrasound images displayed on the monitor, it is sometimes desirable to adjust the monitor's orientation; however, changing the monitor's orientation during probe manipulation is typically difficult for the examiner. Reasons for this include the inability to reach the monitor and the desire to minimize the examination time occupied by both hands due to the need to operate the probe and control panel. Summary of the Invention

[0005] The purpose of this disclosure is to assist examiners in observing ultrasound images. Alternatively, the purpose of this disclosure is to enable easy observation of ultrasound images regardless of the examiner's head position.

[0006] The ultrasound diagnostic apparatus disclosed herein is characterized by comprising: a display for displaying ultrasound images; a camera for capturing images of the space in front of the examinee from the display or its vicinity, generating camera images; a support mechanism for supporting the display and having a drive source for changing at least one of the position and orientation of the display; and a controller for controlling the drive source based on the image of the examinee included in the camera images. Attached Figure Description

[0007] Figure 1 This is a block diagram illustrating the ultrasonic diagnostic apparatus involved in the implementation.

[0008] Figure 2This is a side view showing the ultrasound diagnostic apparatus according to the embodiment.

[0009] Figure 3 This is a diagram illustrating the control algorithm involved in the implementation method.

[0010] Figure 4 This is a diagram illustrating an example of follow-up control.

[0011] Figure 5 This is a diagram representing the follower control method.

[0012] Figure 6 It is a diagram showing the image captured by the illumination.

[0013] Figure 7 This is a diagram indicating that the image was captured by avoiding the illumination.

[0014] Figure 8 This is a diagram illustrating methods for avoiding being shot.

[0015] Figure 9 It is a diagram representing following control and avoiding control. Detailed Implementation

[0016] The embodiments will now be described with reference to the accompanying drawings.

[0017] (1) Overview of the implementation method

[0018] The ultrasound diagnostic apparatus according to the embodiment includes a display, a camera, a support mechanism, and a control unit. The display shows an ultrasound image. The camera captures an image of the space in front of the examinee from the display or its vicinity, generating a camera image. The support mechanism supports the display and has a drive source for changing at least one of the display's position and orientation. The control unit controls the drive source based on the image of the examinee included in the camera image. The control unit may also be referred to as a controller.

[0019] Based on the above structure, at least one of the position and orientation of the display is automatically changed according to the position of the examiner in the space in front. For example, the orientation of the display is adaptively changed so that it faces the examiner's head (or face). This facilitates the observation of ultrasound images.

[0020] The above structure functions effectively in situations where the examiner's hands are occupied and they cannot manually change the orientation of the monitor, when the hand without the probe cannot reach the monitor, or when there is no extra time to adjust the orientation of the monitor in order to shorten the examination time.

[0021] In this implementation, the camera is fixed to the display. The camera's field of view coincides with the display's field of view. With this structure, the display and camera are integrated, thus the spatial relationship between the display and the examinee is captured by the position of the examinee's image within the camera's view. Alternatively, the camera can be embedded in the display or externally fixed to it. Regardless, the spatial relationship between the camera and the display is fixed, capturing the space in front of the display. The field of view corresponds to the area capable of image processing, while the field of view corresponds to the area capable of observation. In practice, the field of view suitable for image diagnosis is not that large.

[0022] In this implementation, the control unit calculates a representative position of the inspector's image in the camera image and changes at least one of the position and posture of the display based on the representative position. Examples of representative positions include specific positions within the head image, specific positions within the face image, the positions of the two eyes, and the position between the two eyes.

[0023] In this implementation, the representative position is calculated based on the head image in the examiner's photograph. The control unit changes at least one of the position and orientation of the display so that the representative position matches or approaches the target position in the camera image. According to this structure, the display screen can be continuously oriented towards the examiner's head or face. The target position can be a point, a line, or a region.

[0024] In this implementation, the control unit determines whether the examinee is facing the display based on the camera image, and controls the drive source when the examinee is facing the display. This structure avoids unnecessary changes in the position or orientation of the display. Furthermore, the control of the drive source can be automatically stopped when the body is frozen or when the probe has detached from the surface of the organism.

[0025] In this implementation, the control unit determines the movement speed when at least one of the display's position and posture changes, based on specified response conditions. Whether the change in the display's position or posture is too rapid or too slow, stress will be applied to the inspector. According to the above structure, this stress can be eliminated or mitigated.

[0026] In this embodiment, the control unit determines illumination interference on the display screen based on the illumination image included in the camera image. If illumination interference is determined, it mitigates or eliminates the interference by changing at least one of the position and orientation of the display. According to this structure, problems caused by illumination interference (difficulty in image viewing) can be eliminated or mitigated.

[0027] In one implementation, the control unit determines that illumination has occurred when the outline or representative coordinates of the illumination image enters a determination area within the camera image. The control unit then changes at least one of the position and orientation of the display to cause the outline or representative coordinates of the illumination image to move away from the determination area. In this case, for example, the height and tilt angle of the display can also be changed simultaneously.

[0028] The ultrasound diagnostic apparatus described in this embodiment includes tracking control that keeps the display continuously facing the examiner, and avoidance control that automatically avoids or reduces illumination exposure. These are essentially independent of each other, but they can also be combined to perform the same function. Both are techniques that assist the examiner based on camera images.

[0029] (2) Detailed implementation method

[0030] exist Figure 1 The diagram illustrates a structural example of an ultrasound diagnostic apparatus according to an embodiment. An ultrasound diagnostic apparatus is a medical device installed in medical institutions such as hospitals. In an ultrasound diagnostic apparatus, ultrasound waves are transmitted to the subject (a living organism), reflected waves from within the subject are received, and an ultrasound image representing the tissues within the subject is formed based on the information obtained. The ultrasound diagnostic apparatus according to this embodiment is a trolley-type ultrasound diagnostic apparatus.

[0031] The ultrasound diagnostic apparatus has a main body (ultrasound diagnostic apparatus main body) 10. A probe (ultrasound probe) 12 is detachably connected to the main body 10. An operation panel 14 is supported by the main body 10 via a lifting mechanism. A pedestal is provided on the inner side of the operation panel 14, and a support mechanism 18 is mounted on this pedestal. The support mechanism 18 is a multi-joint mechanism, as described later. The support mechanism 18 holds a display 16. The display 16 is a flat panel display, specifically, it is composed of an LCD, an organic EL display device, etc.

[0032] The support mechanism 18, serving as the drive source 20, includes multiple motors (or multiple actuators). Each movable part in the support mechanism 18 is driven by a specific motor. Multiple drive signals are supplied in parallel to the multiple motors from the driver assembly 22. The driver assembly 22 can be located outside or inside the support mechanism 18. Alternatively, the driver assembly 22 can be located within the main body 10.

[0033] The control panel 14 is an input device with multiple switches, multiple buttons, a trackball, a keyboard, etc. During ultrasound examination, the examiner typically holds the probe 12 with one hand and operates the control panel 14 with the other hand.

[0034] A camera 24 is fixedly mounted on the upper part of the display 16, and is embedded in this embodiment. As explained later, the camera 24 captures images of the space in front of the display 16 (including the space of the examiner) as a moving image. The viewing field of the display 16 overlaps with the photographic field of view of the camera 24. The viewing field of view is the spatial range within which the display 16 can be observed, and the photographic field of view is the spatial range within which images can be taken. The viewing field of view is actually the range within which image diagnosis can be performed, and it is generally not a very wide range. The camera 24 is a black and white camera or a color camera. Multiple cameras may also be provided. The camera 24 may also be fixedly mounted on the outside of the display 16.

[0035] The probe 12 consists of a probe head, a cable, and a connector. The probe head contains an array of transducer elements arranged in a straight line or arc. Ultrasonic waves are transmitted from the transducer array into the subject, and reflected waves from the subject are received by the transducer array. More specifically, the transducer array forms an ultrasonic beam (transmitting beam and receiving beam), and this beam is electronically scanned to form a scanning surface (beam scanning surface). Known electronic scanning methods include electronic sector scanning and electronic linear scanning. The probe head is the main component and is held by the examiner. A two-dimensional transducer array can also be used instead of a one-dimensional transducer array.

[0036] The transmission / reception unit 26 is an electronic circuit that functions as both a transmit beamformer and a receive beamformer. During transmission, multiple transmit signals are supplied in parallel to the vibrating element array from the transmission / reception unit 26, thus forming a transmit beam. During reception, if reflected waves from a biological body are received in the vibrating element array, multiple receive signals are output from the vibrating element array to the transmission / reception unit 26. In the transmission / reception unit 26, receive beam data is generated by phase alignment and summing (delay and summing) of the multiple receive signals. Typically, one receive frame data is constituted by one scan of the ultrasonic beam. One receive frame data consists of multiple receive beam data arranged in the electronic scanning direction. One receive beam data consists of multiple echo data arranged in the depth direction. Repeated electronic scanning of the ultrasonic beam repeatedly generates multiple receive frame data. These receive frame data constitute a receive frame data string.

[0037] The image forming unit 32 generates a tomographic image data string based on the received frame data string. Specifically, the image forming unit 32 has a digital scan converter (DSC). The DSC is a dedicated processor with coordinate transformation function, pixel interpolation function, frame rate transformation function, etc. In the illustrated structural example, the tomographic image data string is sent from the image forming unit 32 to the control unit 30.

[0038] The control unit 30 is composed of a processor that executes programs. Specifically, it is composed of a CPU. In addition to controlling the operation of the various structural elements constituting the ultrasound diagnostic device, the control unit 30 also has image processing and display processing functions. The control unit 30 according to the embodiment simultaneously or selectively performs follow control and avoidance control during the ultrasound examination. Follow control is the control that keeps the screen of the display 16 continuously facing the examiner's face. Avoidance control is the control that prevents illumination from entering the screen of the display 16 when viewed from the examiner's perspective. In fact, the position and posture of the display are optimized by controlling the operation of the drive source 20.

[0039] Camera image data is transmitted from camera 24 to control unit 30. Ultrasonic image data is transmitted from control unit 30 to display 16. The ultrasonic image is displayed on display 16. If necessary, camera images acquired by camera 24 can also be displayed on display 16.

[0040] exist Figure 2 The image shows the appearance of the ultrasound diagnostic device. The main body 10 has a box-like shape and four casters are provided at its bottom. A lifting mechanism 33 is provided between the main body 10 and the operation panel 14. The operation panel 14 is supported by the main body 10 via the lifting mechanism 33. A base 34 is provided on the inside of the operation panel 14, and a support mechanism 18 is provided on the base 34.

[0041] The support mechanism 18 is an arm mechanism that is a multi-joint mechanism, and it has multiple motors as drive sources. The display 16 is held in place by the support mechanism 18. By changing the shape, etc., of the support mechanism 18, the position and orientation of the display 16 can be changed.

[0042] In the illustrated structural example, the support mechanism 18 includes a first arm 36, a second arm 38, a third arm 40, a fourth arm 42, and a tilting mechanism 44. The first arm 36 rotates relative to the base 34. The second arm 38 has a parallel link and tilts relative to the first arm 36. The upper end of the second arm 38 is connected to the base end of the third arm 40. The third arm 40 rotates relative to the second arm 38. The third arm 40 also has a parallel link and tilts relative to the second arm 38. The fourth arm 42 rotates relative to the third arm. The tilting mechanism 44 causes the display 16 to rotate about a horizontal rotation axis. The illustrated support mechanism 18 is just one example. Various mechanisms that move automatically can be used as the support mechanism 18.

[0043] The front of the display 16 displays the screen. A camera 24 is embedded in the upper center of the display 16. The viewing field 46 of the display 16 overlaps with the photographic field 48 of the camera 24. The spatial relationship between the display 16 and the examiner can be determined through the camera image. Furthermore, by analyzing the camera image, the examiner's position, particularly the position of the examiner's head, can be determined. Additionally, the viewing field 46 and the photographic field 48 overlap even when viewed from above, especially since their central axes are aligned.

[0044] Figure 3 The control algorithm is shown. In S10, it is determined whether to execute follow control and / or avoidance control. The user can also select the control mode to execute. Follow control can also be executed as long as ultrasonic waves are transmitted and received.

[0045] For example, as shown in S10A, the orientation of the inspector's face can be determined based on the inspector's image in the camera image. Specifically, determining whether the inspector is facing the monitor is only permissible if the inspector is facing the monitor, allowing the execution of multiple steps from S12 to S14. Conversely, if the inspector is not facing the monitor, the execution of multiple steps from S12 to S14 can be prohibited. The control of the support mechanism can also be temporarily stopped during freezing or when the probe is suspended in mid-air.

[0046] When the execution of follow-up control is instructed, steps S12 and S18 are executed. In image analysis in S12, the head image (or face image) of the examinee is determined based on the camera image IM, and representative coordinates are determined based on the head image. In S18, the orientation of the display screen is controlled based on the examinee's representative coordinates according to specified follow-up conditions. For example, if it is determined that the representative coordinates deviate from the target area within the camera image, the orientation of the display screen (and the camera orientation) is adaptively controlled so that the representative coordinates fall within the target area. This allows the screen to continuously face the examinee. During ultrasound examination, the position of the examinee's face changes, but the orientation of the screen is automatically optimized according to this change.

[0047] The following conditions include a time constant τ1, which serves as a response condition. Decreasing the time constant τ1 results in high-speed following, while increasing it results in low-speed following. These following conditions can also be used to determine the size and location of the target region.

[0048] When avoidance control is instructed to be implemented, steps S14 and S20 are executed. In image processing in S14, illumination ingress is determined based on the camera image; if illumination ingress is determined, the outline of the illumination image is extracted. In S20, according to avoidance conditions, the position and orientation of the screen are changed so that the outline of the illumination image is removed from the determination area in the camera image. For example, the height of the monitor is increased while the orientation of the monitor is lowered. Thus, from the inspector's perspective, the illumination image is removed from the screen. Alternatively, the representative coordinates of the illumination image can be used instead of the illumination image's outline.

[0049] The avoidance condition includes a time constant τ2, which serves as the response condition. Decreasing the time constant τ2 results in a higher avoidance speed, while increasing it results in a lower avoidance speed. The size and location of the decision area can also be determined as an avoidance condition.

[0050] In S16, both follow control (S18) and avoidance control (S20) can be executed. In this case, one control can be prioritized, and the other control can be executed as long as it is possible to control the other party.

[0051] The following sections will provide specific examples to illustrate both follow-control and avoidance control.

[0052] exist Figure 4An example of follow-up control is shown. A bed 52 is arranged adjacent to the ultrasound diagnostic device 50. The patient 54 lies on the bed 52. Symbol 16A indicates the initial position of the monitor. In this case, the examiner's head 58A is located in front of (facing) the monitor 16A. Symbol 60A indicates the direction of gaze. For example, the position of the examiner's head 58A changes constantly as the probe 56 contacts the patient 54 and the contact position changes. Symbol 58B indicates the changed head position. The direction of gaze at this time is indicated by symbol 60B. Along with the change in head position, the monitor's position changes. The changed monitor is indicated by symbol 16B. During continuous observation of the ultrasound image, the screen of monitor 16B can be automatically and continuously oriented towards the examiner. This facilitates the interpretation of the ultrasound image.

[0053] exist Figure 5 The image shown is an example of an image parsing method for tracking control. Camera image 62 includes an inspector image 64. Inspector image 64 includes a head image (or facial image) 64A. To extract it, a region of interest 66 is automatically defined. For example, image recognition techniques can be used to determine the head image or the region it occupies.

[0054] Based on head image 64A, the representative coordinate 70 is determined. In this case, the positions of the two eyes 68R and 68L can also be determined, and the representative coordinate 70 can be set as their midpoint. Other positions can also be set as representative coordinates. For example, the centroid, midpoint, etc. of head image 64A can also be used as representative coordinates.

[0055] A target area (target position) (not shown) is defined in the center of the camera image 62. If the representative coordinates deviate from the target area, the orientation (pose) of the display is changed (refer to symbol 74) so ​​that the representative coordinates fall within the target area. When changing the orientation, a vector from the representative coordinates toward the center point of the target area can also be calculated. The rotation angle θ and tilt angle can also be determined based on the two components defining this vector. The direction and speed of the display. In addition to the orientation of the display, its position can also be changed.

[0056] exist Figure 5 In the example shown, camera image 72 is acquired after the orientation change. A target region 71 is defined at its center, representing that coordinate 70 belongs to the target region 71. In practice, for each frame, tracking control including image parsing is performed, and the orientation of the display changes continuously with the movement of the head. However, the motion speed at this time is based on the time constant τ1.

[0057] The orientation of the face can be determined based on the position of the representative coordinates in the head image (or face image) 64A. Follow-up control can also be performed as long as the examiner is facing the monitor. Other methods can also be used to determine the orientation of the face. For example, the orientation of the face can be determined by detecting the gaze vector.

[0058] exist Figure 6 The image shows the illumination image being captured. The display 16 is held in place by a support mechanism 18A. The screen 17 of the display 16 faces upwards. The examiner's head is indicated by symbol 76, and their line of sight by symbol 78A. Illumination 80 is installed on the ceiling of the examination room. Light 80A from illumination 80 is reflected in the screen 17, resulting in light 80B entering the examiner's field of vision. That is, an illumination image is captured within the screen 17, which obstructs the observation of the ultrasound image. The examination room is usually a dark room, but since illumination is not completely eliminated, depending on the situation, an illumination image may be captured.

[0059] In the ultrasonic diagnostic apparatus described in the embodiments, such as Figure 7 As shown, when illumination interference is detected, avoidance control is executed. Specifically, the drive source of the support mechanism 18 is controlled, the height of the display 16 is raised (reference symbol 80), and the tilt angle of the display 16 is changed, while the orientation of the screen 17 is lowered (reference symbol 82). Thus, even if light 84 from the illumination 8 is reflected on the screen 17 to generate light 86, this light 86 can be prevented from entering the examiner's field of vision (reference symbol 78A). Even assuming that light 86 enters the examiner's field of vision, the problem of illumination interference is mitigated. After avoidance control, the position and orientation of the display are determined so that the screen is as directly facing the examiner's head as possible.

[0060] exist Figure 8 The image below illustrates an example of an image analysis method for avoidance control. (A) shows the state before avoidance, and (B) shows the state after avoidance. As shown in (A), the inspector's image 90 is included in the camera image 88. For example, the direction the inspector is facing towards the monitor can be determined based on the position of the representative coordinate 94 in the head image. An illumination image 92 is captured within the camera image 88. Based on the repetition relationship between the observation field of view and the photographic field of view, it is possible to roughly determine whether the image was captured based on the position of the illumination image 92 within the camera image 88 (actually based on the spatial relationship between the inspector and the monitor).

[0061] For example, a determination region 96 for capturing an image is defined in camera image 88. In this case, the head image or representative coordinates 94 can also be used as a reference. Capture is determined if the illuminated image or its outline falls within determination region 96. Capture can also be determined based on whether the representative coordinates (e.g., center position) of the illuminated image 92 enters determination region 96.

[0062] In the implementation method, if an image is detected, the height of the display is increased (the field of view in the z-direction is increased), and the tilt angle is... Change direction in the negative (the camera view is changed downwards). Multiple avoidance modes can also be pre-registered, and the best avoidance mode will be automatically selected based on the situation.

[0063] As shown in (B), in the altered camera image 100, the illumination image 92A shifts upward from the decision region 106 (the representative coordinate 98A also shifts upward). In the illustrated example, the decision region 106 is set based on the representative coordinate 104 in the inspector image 102 within the camera image 100. For example, the decision region can also be defined with the representative coordinate 104 as the center.

[0064] exist Figure 9 The diagram illustrates the result of simultaneously executing follow control and avoidance control. By controlling the position and posture of the display, the illumination image 92A is deviated from the judgment area defined for the camera image 110. However, the representative coordinate 114 of the inspector image 112 is significantly offset from the center of the camera image 110. Through follow control, the display is rotated, thereby obtaining the camera image 118. A target area 116 is defined relative to this camera image 118, and the representative coordinate 114 belongs to the target area 116. A new judgment area 120 is defined in the camera image 118, but the illumination image 92A does not enter this area. In other words, the display screen is oriented towards the inspector while avoiding illumination. If two controls cannot be performed simultaneously, the predetermined control can be prioritized. Alternatively, a compromise can be made by implementing both controls.

[0065] According to the above embodiment, since the camera can be fixedly mounted on the display, and tracking control and avoidance control can be achieved through image analysis, it is advantageous that a complex structure is not required to implement these controls. Limiters can also be installed in each movable part of the support mechanism to stop the movement when a certain load is generated.

Claims

1. An ultrasonic diagnostic device, characterized in that, include: Display (16) shows an ultrasound image; The camera (24) captures images of the space in front of the inspector from the display (16) or its vicinity, generating a camera image; The support mechanism (18) is a mechanism that supports the display (16) and has a drive source (20) that can change at least one of the position and orientation of the display (16); and The controller (30) controls the drive source (20) based on the inspector image included in the camera image. The controller (30) determines whether to capture illumination onto the screen of the display (16) if the outline or representative coordinates of the illumination image included in the camera image enter the determination area within the camera image. Change at least one of the position and orientation of the display (16) so that the outline or representative coordinates of the illumination image are removed from the determination area.

2. The ultrasonic diagnostic device according to claim 1, characterized in that, The camera (24) is fixed to the display (16). The field of view of the camera (24) overlaps with the field of view of the display (16).

3. The ultrasonic diagnostic device according to claim 2, characterized in that, The controller (30) Calculate the representative position of the inspector's image in the camera image. The position and orientation of the display are changed based on the representative position.

4. The ultrasonic diagnostic device according to claim 3, characterized in that, The representative position is calculated based on the head image in the inspector's image. The controller (30) controls at least one of the position and orientation of the display (16) so that the representative position is consistent with or close to the target position in the camera image.

5. The ultrasonic diagnostic device according to claim 1, characterized in that, The controller (30) determines whether the inspector is facing the display (16) based on the camera image. The drive source (20) is controlled when the inspector is facing the display (16).

6. The ultrasonic diagnostic device according to claim 1, characterized in that, The controller (30) determines the speed of motion when changing at least one of the position and posture of the display (16) according to the specified response conditions.

7. The ultrasonic diagnostic device according to claim 1, characterized in that, The controller (30) determines the illumination image to be captured on the screen of the display (16) based on the illumination image. If the illumination is detected as imploding, the illumination can be mitigated or eliminated by changing at least one of the position and orientation of the display (16).

Images