Head-mounted display and surgical support system using the same

Abstract

To provide a head mount display in which usability is improved by allowing a position and an angle of a display unit to be finely and easily adjusted by a user with one hand.SOLUTION: This head mount display includes: a head mounted section to be mounted to a head of an observer; a first display unit for a right eye, and a second display unit for a left eye; a frame for holding the first and second display units side by side in a longitudinal direction; a first arm link section which is rotatably attached to the head mounted section with a first shaft as a center; and a second arm link section which is rotatably attached to the first arm link section with a second shaft as a center, and rotatably attached to an intermediate section between a first end and a second end of the frame with a third shaft as a center. Since the first to third shafts are substantially parallel, by operating the frame, the first or second arm link section rotates and the position and angle of the display unit to the head mounted section can be adjusted.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a head-mounted display, particularly to a head-mounted display for medical use, and further to a surgery assistance system that uses such a head-mounted display to assist surgery. [Background technology]

[0002] Head-mounted displays have traditionally been used primarily as entertainment tools, but in recent years, their usefulness has led to increased attention being paid to their use in fields such as medicine.

[0003] Head-mounted displays can be broadly divided into monocular and binocular types, and further into 2D (two-dimensional) and 3D (three-dimensional) types. In the case of 3D head-mounted displays, convergence eye movements and visual adjustment of 3D images cause eye stress, which is a major problem. There is a strong demand to avoid this, especially during long surgeries (surgeries lasting four to five hours).

[0004] As a related technique, Patent Document 1 discloses a head-mounted display with improved usability. This head-mounted display comprises a head-mounted unit including a holding unit 4, a display unit 6 that is placed in front of one eye of the viewer during viewing, an arm unit 8 that connects the head-mounted unit and the display unit 6, and first and second connecting mechanisms 9 and 10.

[0005] The first connecting mechanism 9 connects the holding unit 4 and the base end of the arm unit 8 so as to enable rotation of the arm unit 8 in a rotation direction θx1 about the axis X1 relative to the holding unit 4 and to enable positioning of the arm unit 8 relative to the holding unit 4. The second connecting mechanism 10 is configured so as to enable free positioning of the display unit 6 at successive positions within a predetermined adjustment range for positioning the display unit 6 during observation, with regard to rotation of the display unit 6 in a rotation direction θx2 about the axis X2 relative to the arm unit 8. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2009-33308 A (Abstract) Summary of the Invention [Problem to be solved by the invention]

[0007] As disclosed in Patent Document 1, supporting the display unit 6 with an arm unit 8 and first and second connecting mechanisms 9 and 10 improves ease of use when adjusting the position and angle of the display unit 6. However, the head-mounted display of Patent Document 1 is provided with only one display unit 6, and therefore, it is not possible to observe a 3D stereoscopic image.

[0008] To enable the observation of 3D stereoscopic images using the head-mounted display disclosed in Patent Document 1, it is conceivable to provide two display units, one for the left eye and one for the right eye, and to provide arms and first and second connecting mechanisms on the left and right sides to support them. However, in this case, when adjusting the position and angle of the display units, it is necessary to operate the left and right arms and connecting mechanisms in synchronization, making it difficult to finely adjust the position and angle of the display units with one hand. Furthermore, if the torque required to move the display units is set to be small, the display units will easily move due to external forces such as gravity and centrifugal force, which can be a major problem, especially during surgery.

[0009] Furthermore, in the medical field, it is necessary to clean and sterilize the instruments that surgeons use during surgery. Therefore, if a surgeon wears a head-mounted display and adjusts the position and angle of the display during surgery, the head-mounted display must be cleaned and sterilized before the surgery. In particular, cleaning equipment is often used for cleaning, and while it is easy to clean smooth instruments without any irregularities, cleaning a head-mounted display with a complex shape is difficult.

[0010] In view of the above, a first object of the present invention is to provide a head-mounted display that is easy to use and allows the viewer to finely adjust the position and angle of the display unit with one hand, so that 3D stereoscopic images can be viewed with reduced eye stress regardless of individual differences in eyesight, strabismus, astigmatism, etc. A second object of the present invention is to provide a head-mounted display that can be used safely in surgery. A third object of the present invention is to provide a surgery support system or the like that supports surgery using such a head-mounted display. [Means for solving the problem]

[0011] In order to solve at least part of the above problems, a head-mounted display according to a first aspect of the present invention includes a head-mounted unit to be mounted on the head of a viewer, a first display unit for the right eye, a second display unit for the left eye, and a frame for holding the first and second display units aligned in the longitudinal direction. the frame having a first protrusion and a second protrusion provided at a first end and a second end in the longitudinal direction, respectively; and, two caps detachably attached to the first and second protrusions of the frame; a first arm link portion attached to the head mounting portion so as to be rotatable about a first axis; and a second arm link portion attached to the first arm link portion so as to be rotatable about a second axis and in front of the frame. Record number End of 1 and The aforementioned and a second arm link portion attached to an intermediate portion between the second end portion and the arm link portion so as to be rotatable about a third axis, and the first to third axes are approximately parallel to each other, each of the two caps has an attachment portion provided with a recess that fits with the first or second protrusion of the frame, and an edge portion that protrudes from the attachment portion along a plane that is approximately perpendicular to the first to third axes when each cap is attached to the first or second protrusion of the frame;By manipulating the frame, the first or second arm link unit rotates, making it possible to adjust the positions and angles of the first and second display units relative to the head-mounted unit.

[0012] According to a first aspect of the present invention, in a head-mounted display capable of observing a stereoscopic image, a second arm link portion attached to a head-mounted portion via a first arm link portion supports an intermediate portion between a first end and a second end of a frame, thereby making it easy for a viewer to finely adjust the position and angle of the display portion with one hand. do , improving usability Let It is possible. Furthermore, by cleaning and sterilizing the two caps prior to surgery and then attaching them to the first and second protrusions of the frame, when a surgeon performs surgery wearing the head-mounted display, he or she can adjust the position and angle of the display by operating the frame via the caps, thereby providing a head-mounted display that can be used safely during surgery.

[0013] In addition, a head-mounted display according to a second aspect of the present invention further comprises at least one of a first adjustment mechanism arranged on the first axis and having a handle for manually adjusting the magnitude of the torque that is minimum required to change the angle of the first arm link unit relative to the head-mounted unit, a second adjustment mechanism arranged on the second axis and having a handle for manually adjusting the magnitude of the torque that is minimum required to change the angle of the second arm link unit relative to the first arm link unit, and a third adjustment mechanism arranged on the third axis and having a handle for manually adjusting the magnitude of the torque that is minimum required to change the angle of the frame relative to the second arm link unit, and is capable of locking and releasing the position and angle of the first and second display units relative to the head-mounted unit.

[0014] According to the second aspect of the present invention, at least one adjustment mechanism is provided for manually adjusting the ease of movement of the movable part, so that, for example, when a surgeon wears a head-mounted display and performs surgery for a long period of time, the position and angle of the display part can be adjusted and locked before starting the surgery, thereby providing a head-mounted display that can be used safely during surgery.

[0017] Furthermore, the present invention 3The surgical support system according to the above aspect comprises a plurality of image sensors that capture an image of a subject from within an endoscope and generate a plurality of image signals each having parallax, an encoder that generates a three-dimensional image signal based on the plurality of image signals, and the head-mounted display further comprises a decoder that generates an image signal for the right eye and an image signal for the left eye based on the three-dimensional image signal and supplies the image signal for the right eye and the image signal for the left eye to the first and second display units, respectively, thereby displaying a three-dimensional stereoscopic image on the first and second display units.

[0018] The present invention 3 From this viewpoint, in surgical procedures using an endoscope, a clear three-dimensional stereoscopic image seen from inside the endoscope is displayed on a head-mounted display, thereby making the operation by the surgeon safer. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a perspective view showing a state in which a head-mounted display according to an embodiment of the present invention is worn by a viewer. [Figure 2] FIG. 2 is an enlarged perspective view of a part of the head-mounted display shown in FIG. [Figure 3] 1 is a block diagram showing an example of the configuration of a surgery assistance system according to an embodiment of the present invention; [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of the surgery support server shown in FIG. 3. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the same components are given the same reference numerals and redundant description will be omitted.

[0021] <Head-mounted display> FIG. 1 is a perspective view showing a state in which a head-mounted display according to an embodiment of the present invention is worn by a viewer, and FIG. 2 is a perspective view showing an enlarged view of a part of the head-mounted display shown in FIG. 1.

[0022] As shown in Figures 1 and 2, this head-mounted display includes a headband 10 as a head-mounted part to be worn on the viewer's head, a first display unit 21 for the right eye and a second display unit 22 for the left eye, a frame 30 that holds the first display unit 21 and the second display unit 22 side by side in the longitudinal direction (the x-axis direction shown in Figure 2), and a connection cable 40 that supplies DC power and image signals to the head-mounted display.

[0023] When observing an image, the headband 10 is worn on the observer's head, and the first display unit 21 and the second display unit 22 are positioned in front of the observer's right and left eyes, respectively. The first display unit 21 and the second display unit 22 may be of either a non-transmissive or transmissive type, but Fig. 1 shows a non-transmissive display as an example. The first display unit 21 and the second display unit 22 display a two-dimensional planar image or a three-dimensional stereoscopic image based on a supplied two-dimensional image signal or a supplied three-dimensional image signal.

[0024] To ensure a good wearing condition, the headband 10 may be provided with an adjustment screw 11 used to adjust the inner diameter of the band to fit the size of the viewer's head. Note that in this embodiment, the headband 10 is used as an example of a head-mounted part, but the head-mounted part is not limited to the headband 10 and may be, for example, a helmet.

[0025] The headband 10 and the frame 30 are connected by a head mount 50, a first arm link portion 51, a second arm link portion 52, and a frame mount 53. Materials that can be used for the frame 30 and the head mount 50 to the frame mount 53 include metals such as aluminum or titanium, alloys thereof, or plastics.

[0026] 2, a head mount 50 is fixed to the headband 10, and a frame mount 53 is fixed to the frame 30. The first arm link portion 51 is attached to the headband 10 via the head mount 50 so as to be rotatable about a first axis X1.

[0027] The second arm link portion 52 is attached to the first arm link portion 51 so as to be rotatable about the second axis X2, and is also attached to the frame 30 via a frame mount 53 so as to be rotatable about the third axis X3. Here, the second arm link portion 52 is attached to an intermediate portion (preferably, approximately the center portion) between the first end portion 31 and the second end portion 32 in the longitudinal direction of the frame 30 so as to be rotatable about the third axis X3.

[0028] In such a head-mounted display, the visual size of the screen can be adjusted by changing the distance between the observer's right and left eyes and the first and second display units 21 and 22. For example, it is possible to display a large screen equivalent to a 57-inch desktop display.

[0029] Normally, it is desirable to form an image as far away as possible to prevent eye strain. In particular, when displaying a 3D stereoscopic image, a difference between the convergence angle of the eyes and the focal length can also cause eye strain, so in order to obtain an accurate stereoscopic effect, it is necessary to finely adjust the positions and angles of the first display unit 21 and the second display unit 22 relative to the viewer's right and left eyes.

[0030] For this reason, in the head mounted display according to this embodiment, the viewer can operate the frame 30 to rotate the first arm link unit 51 or the second arm link unit 52, thereby adjusting the positions and angles of the first display unit 21 and the second display unit 22 relative to the headband 10, i.e., adjusting the stereoscopic viewpoint. Specifically, the following adjustments are possible.

[0031] (1) By rotating the first arm link portion 51 and the second arm link portion 52, the distance between the headband 10 and the first display unit 21 and the second display unit 22 can be adjusted. (2) By rotating the first arm link portion 51, the orientation of the first display unit 21 and the second display unit 22 relative to the headband 10 can be adjusted. (3) By rotating the second arm link portion 52, the angles of the first display unit 21 and the second display unit 22 relative to the headband 10 can be adjusted.

[0032] In the link mechanism supporting the frame 30, the first axis X1 to the third axis X3 are substantially parallel to one another, and preferably to the longitudinal direction (x-axis direction) of the frame 30. If the length between the axes of the first arm link section 51 is L1 and the rotation angle from the y-axis direction is θ1, and the length between the axes of the second arm link section 52 is L2 and the rotation angle from the longitudinal direction of the first arm link section 51 is θ2, the coordinates (Dx, Dy, Dz) of the third axis X3 with the first axis X1 as the origin are expressed by the following equations.

[0033] Dx≒0 Dy=L1cosθ1+L2cos(θ1+θ2) Dz=L1sinθ1+L2sin(θ1+θ2) Furthermore, the distance D between the first axis X1 and the third axis X3 is expressed by the following equation. D=(Dx 2 +Dy 2 +Dz 2 ) 1 / 2

[0034] Here, the first arm link unit 51, the second arm link unit 52, and each unit of the frame 30 can only move within the yz plane perpendicular to the x-axis direction. Therefore, even if the viewer operates the frame 30 with one hand to move the first display unit 21 and the second display unit 22, the first display unit 21 and the second display unit 22 will not shift in the x-axis direction unless the headband 10 shifts.

[0035] Thus, according to this embodiment, in a head-mounted display capable of observing stereoscopic images, the second arm link portion 52 attached to the headband 10 via the first arm link portion 51 supports the intermediate portion between the first end portion 31 and the second end portion 32 of the frame 30, making it easy for the observer to finely adjust the position and angle of the first display portion 21 and the second display portion 22 with one hand, thereby providing a head-mounted display with improved usability.

[0036] Regarding applications in the medical field, when a stationary monitor is used in endoscopic surgery, the subject can be difficult to see due to various devices and the surgical field, making it difficult to observe the entire endoscopic image, and the viewpoint needs to move a lot, which can cause stress on the neck and surgical posture.

[0037] In contrast, when using a head-mounted display, there is nothing obstructing the view, making it possible to observe the entire endoscopic image. Also, by shifting the position or angle of the frame 30 or by moving the viewpoint left, right, or downward to look away from the first display unit 21 and the second display unit 22, the user can freely view the surrounding scenery or what is at hand.

[0038] Furthermore, the head-mounted display is easy to carry and can be used anytime within the reach of the connection cable 40. It is desirable that the weight of the head-mounted display be 280 g or less so that fatigue does not accumulate even during long surgeries (four to five hours).

[0039] Furthermore, if the torque required to move the first display unit 21 and the second display unit 22 is set to a small value in an attempt to make it easier to adjust the position and angle of the first display unit 21 and the second display unit 22, the first display unit 21 and the second display unit 22 will easily move due to external forces such as gravity and centrifugal force, which will be a major problem, particularly in the medical field.

[0040] Therefore, the head mounted display according to this embodiment may further include at least one of a first adjustment mechanism 54, a second adjustment mechanism 55, and a third adjustment mechanism 56. This makes it possible to lock or release the positions and angles of the first display unit 21 and the second display unit 22 relative to the headband 10. Here, "releasing" refers to releasing the locked state and setting the first display unit 21 and the second display unit 22 in a state where they can be moved with a slight force relative to the headband 10.

[0041] The first adjustment mechanism 54 is positioned on the first axis X1, rotatably attaches the first arm link portion 51 to the head mount 50, and has a handle for manually adjusting the minimum amount of torque required to change the angle of the first arm link portion 51 relative to the headband 10.

[0042] The second adjustment mechanism 55 is arranged on the second axis X2, rotatably attaches the second arm link portion 52 to the first arm link portion 51, and has a handle for manually adjusting the minimum amount of torque required to change the angle of the second arm link portion 52 relative to the first arm link portion 51.

[0043] The third adjustment mechanism 56 is positioned on the third axis X3, rotatably attaches the frame mount 53 to the second arm link portion 52, and has a handle for manually adjusting the minimum amount of torque required to change the angle of the frame 30 relative to the second arm link portion 52.

[0044] The first to third adjustment mechanisms 54 to 56 are mechanical components that can be operated without using tools such as a screwdriver, wrench, or spanner, and include, for example, butterfly bolts, knob bolts, knurled thumb screws, and the like that can be grasped and turned to tighten or loosen the screws. 1 and 2 show, as an example, a case in which all of the first to third adjustment mechanisms 54 to 56 are provided, but an ordinary screw or nut may be used in place of at least one of them.

[0045] By providing at least one of the first adjustment mechanism 54 to the third adjustment mechanism 56 for manually adjusting the ease of movement of the movable part, for example, when a surgeon wears a head-mounted display and performs a long surgery, the surgeon can start the surgery after adjusting and locking the positions and angles of the first display unit 21 and the second display unit 22. Therefore, it is possible to provide a head-mounted display that can be used in surgery with peace of mind.

[0046] Furthermore, when a surgeon wears a head-mounted display and adjusts the position and angle of the first display unit 21 and the second display unit 22 during surgery, the head-mounted display must be cleaned and sterilized prior to surgery. In particular, a cleaning device is often used for cleaning, and while it is easy to clean smooth instruments without any irregularities, cleaning a head-mounted display with a complex shape is difficult.

[0047] Therefore, in this embodiment, the frame 30 has a first convex portion (first knob) 61 and a second convex portion (second knob) 62 provided on the first end portion 31 and the second end portion 32, respectively, and the head-mounted display may further have two caps 71 and 72 that are detachable from the first convex portion 61 and the second convex portion 62 of the frame 30, respectively.

[0048] The caps 71 and 72 may be made of, for example, synthetic resin. Prior to surgery, the caps 71 and 72 are washed and sterilized, and then attached to the first protrusion 61 and the second protrusion 62 of the frame 30, respectively. The caps 71 and 72 may be replaced after each surgery.

[0049] 2, the cap 72 has an attachment portion 72a provided with a recess that fits onto the second protrusion 62 of the frame 30, and an edge portion 72b that protrudes from the attachment portion 72a along a plane that is approximately perpendicular to the first axis X1 to the third axis X3. The cap 72 covers more than half of one side surface of the frame 30, thereby preventing the fingers of the observer (surgeon) from touching the main body of the frame 30.

[0050] 2, the second protrusion 62 of the frame 30 has a cylindrical shape, the attachment portion 72a of the cap 72 has a hollow cylindrical shape with one side (the upper surface in the x-axis direction) closed, and the edge portion 72b of the cap 72 has a disk shape with a diameter larger than that of the attachment portion 72a. Although not shown in FIG. 2, the same applies to the first protrusion 61 and the cap 71 of the frame 30. Furthermore, when at least one of the first adjustment mechanism 54 to the third adjustment mechanism 56 is provided, the head-mounted display may further have a cap that is detachable from the handle of the adjustment mechanism.

[0051] According to this embodiment, prior to surgery, the caps 71 and 72 are cleaned and sterilized and then attached to the first protrusion 61 and the second protrusion 62 of the frame 30. This allows a surgeon to wear a head-mounted display while performing surgery by operating the frame 30 via the cap 71 or 72, thereby adjusting the positions and angles of the first display unit 21 and the second display unit 22 from a stereoscopic viewpoint unique to the surgeon. Therefore, a head-mounted display that can be used safely during surgery can be provided.

[0052] <Surgery support system> Next, a surgery assistance system according to an embodiment of the present invention will be described. Fig. 3 is a block diagram showing an example of the configuration of a surgery assistance system according to an embodiment of the present invention. As shown in Fig. 3, this surgery assistance system includes a surgery assistance robot 100, an encoder 200, and a head-mounted display 300 according to an embodiment of the present invention, and may further include a surgery assistance server 400.

[0053] The surgical support robot 100 includes, for example, a master-slave endoscope and is used in surgery using a robot manipulator. In this case, the surgical support robot 100 is equipped with a manipulator 110 attached to the endoscope and a plurality of image sensors (in FIG. 3, right image sensor 121 and left image sensor 122 are shown) that capture images of a subject from within the endoscope and generate a plurality of image signals with parallax.

[0054] The manipulator 110 is operated by remote control from the surgery support server 400 to enable minimally invasive surgery on patients in thoracic or abdominal surgery performed using an endoscope. The tip of the arm of the manipulator 110 has a joint equivalent to a human wrist, and the tip can be freely bent.

[0055] The right image sensor 121 and the left image sensor 122 are arranged, for example, at a distance from each other inside the endoscope, and generate a right eye image signal representing an image for the right eye and a left eye image signal representing an image for the left eye, respectively. Note that instead of the surgery support robot 100, a manually operated endoscope provided with the right image sensor 121 and the left image sensor 122 may be used.

[0056] The encoder 200 generates a three-dimensional image signal based on a plurality of two-dimensional image signals output from the right imaging element 121 and the left imaging element 122. For example, the encoder 200 generates a side-by-side high-definition signal (e.g., a three-dimensional image signal of 3840×1080 pixels) by horizontally synthesizing a stereoscopic image made up of an image for the right eye (e.g., 1920×1080 pixels) and an image for the left eye (e.g., 1920×1080 pixels). The three-dimensional image signal generated in this manner is supplied in real time to the head-mounted display 300 together with a DC power supply.

[0057] To display images, the head mounted display 300 includes a first display unit 21 for the right eye, a second display unit 22 for the left eye, and a decoder 80. The head mounted display 300 may further include a communication circuit 90 for communicating with the outside world via a wired or wireless connection, and may be a smart glasses type display having a function for connecting to an external computer, smartphone, or other information terminal via a network.

[0058] Each of the first display unit 21 and the second display unit 22 includes a display panel such as an organic EL (Electro-Luminescence) panel or a liquid crystal panel, an optical lens, a case, and a circuit board. For example, the display panel includes a plurality of active matrix pixel circuits provided on a silicon chip. Each pixel circuit includes a light-emitting element such as an OLED, a plurality of transistors, etc. The silicon chip also includes a drive circuit for driving the pixel circuits, etc.

[0059] The decoder 80 generates an image signal for the right eye and an image signal for the left eye based on the three-dimensional image signal supplied from the encoder 200, and supplies these image signals to the first display unit 21 and the second display unit 22, respectively, to display three-dimensional stereoscopic images on the first display unit 21 and the second display unit 22. This allows the viewer to perceive the displayed image as if it had depth and a three-dimensional effect.

[0060] According to the surgical support system of one embodiment of the present invention, during surgical operations using an endoscope, a clear three-dimensional stereoscopic image seen from inside the endoscope is displayed on the head-mounted display 300, thereby making the operation by the surgeon safer.

[0061] <Surgery support server> Fig. 4 is a block diagram showing an example of the configuration of the surgery assistance server shown in Fig. 3. The surgery assistance server 400 is configured, for example, by a computer on which a surgery assistance program is installed.

[0062] As shown in Fig. 4, the surgery support server 400 includes an operation unit 410, a display unit 420, an audio input / output unit 430, a communication circuit 440, an interface 450, a CPU (Central Processing Unit) 460, and a storage unit 470. The interface 450 to the storage unit 470 are connected to one another via a bus line. Note that some of the components shown in Figs. 3 and 4 may be omitted or modified, or other components may be added to the components shown in Figs. 3 and 4.

[0063] The operation unit 410 includes, for example, a keyboard, a mouse, etc., and is used to input various commands and data to the surgery support server 400. The display unit 420 includes, for example, an organic EL display or a liquid crystal display, etc., and displays an operation screen, etc. The audio input / output unit 430 includes, for example, a microphone, an amplifier, a speaker, etc., and converts audio signals into electrical signals, or converts electrical signals into audio signals.

[0064] The communication circuit 440 performs data communication with the surgery support robot 100 or the head-mounted display 300 (FIG. 3) via a network by performing wired or wireless communication. The interface 450 is connected to the operation unit 410 to the communication circuit 440, and transmits various commands and data between them and the CPU 460. The CPU 460 performs various calculations and data processing in accordance with various software (including a surgery support program) stored in the storage unit 470.

[0065] In addition to the above software, storage unit 470 stores various data used in processing electronic information. The recording medium (storage medium) in storage unit 470 may be an internal hard disk, a flexible disk, a magneto-optical disk, a magnetic tape, a RAM (random access memory), a ROM (read only memory), a CD-ROM, a DVD-ROM, or the like.

[0066] Here, the CPU 460 and the software (surgery assistance program) stored in the storage unit 470 configure a robot remote control unit 461, an image signal management unit 462, and an image recognition processing unit 463 as functional blocks.

[0067] The robot remote control unit 461 remotely operates the manipulator 110 (FIG. 3) of the surgery support robot 100 in accordance with the operation of the surgeon using the operation unit 410. The image signal management unit 462 accumulates the three-dimensional image signals generated by the encoder 200 in the storage unit 470, and, as necessary, reads out the three-dimensional image signals stored in the storage unit 470 and supplies them to the display unit 420 or the head-mounted display 300 (FIG. 3).

[0068] The image recognition processing unit 463 recognizes a predetermined part or region in the image of the patient represented by the three-dimensional image signal generated by the encoder 200, as necessary. The image recognition result may be displayed as an image on the display unit 420, or output as sound from the sound input / output unit 430. An example of the image recognition processing performed by the image recognition processing unit 463 will be described below.

[0069] First, image recognition processing unit 463 roughly detects parts or regions of a human being in an image represented by a three-dimensional image signal (hereinafter also referred to as an "input image"). Next, image recognition processing unit 463 performs part recognition processing on the three-dimensional image signal based on learning data stored in advance in storage unit 470, thereby extracting a plurality of feature points that identify the positions of a plurality of parts or regions of a human being (for example, the right atrium, left atrium, right ventricle, left ventricle) in the input image, and determining the coordinates of these feature points.

[0070] In this part recognition process, the image of the target part or region is divided into shape and texture, and then the dimensions are reduced by principal component analysis, making it possible to represent changes in the shape and texture of the target with fewer parameters. This allows shape and texture information to be represented with low-dimensional parameters. Furthermore, parameters related to global changes, such as where the target is located in the image, its size, and its orientation, are used.

[0071] The image recognition processing unit 463 generates a comparison image by locally and globally changing the model of the part or area in the training data using these parameters, and compares the comparison image with the input image to find parameters that minimize the error. This allows the image recognition processing unit 463 to recognize a part in the input image that corresponds to the predetermined part or area specified based on the training data.

[0072] Furthermore, the image recognition processing unit 463 may add information serving as a guide for surgery to the 3D image signal by performing image processing on the 3D image signal based on the image recognition result. For example, in the image of the patient represented by the 3D image signal, the site or region to be surgically treated may be highlighted by increasing the brightness or changing the color.

[0073] In this way, the surgery assistance program causes the CPU 460 to execute the following steps: (a) recognizing a predetermined site or area in an image of the patient represented by a three-dimensional image signal generated by the encoder 200; and (b) adding information serving as a guide for the surgery to the three-dimensional image signal by performing image processing on the three-dimensional image signal based on the image recognition result. The CPU 460 supplies the three-dimensional image signal that has undergone image processing to the head-mounted display 300.

[0074] The head-mounted display 300 displays a 3D stereoscopic image based on a 3D image signal supplied from the CPU 460 of the surgery support server 400. The head-mounted display 300 may be provided to the user as a surgery support system together with a recording medium on which a surgery support program is recorded. Such a recording medium may be a flexible disk, a magneto-optical disk, a magnetic tape, a non-volatile memory, a CD-ROM, a DVD-ROM, or the like.

[0075] The present invention is not limited to the above-described embodiments, and many modifications can be made within the technical concept of the present invention by those skilled in the art. [Industrial Applicability]

[0076] The present invention can be used in head-mounted displays, particularly head-mounted displays used for medical purposes, and surgery support systems that use such head-mounted displays to support surgery. [Explanation of symbols]

[0077] 10...headband, 11...adjustment screw, 21...first display section, 22...second display section, 30...frame, 31...first end section, 32...second end section, 40...connection cable, 50...head mount, 51...first arm link section, 52...second arm link section, 53...frame mount, 54...first adjustment mechanism, 55...second adjustment mechanism, 56...third adjustment mechanism, 61...first protrusion, 62...second protrusion, 71 and 72...cap, 72a...mounting section, 72b...edge section, 80 ...decoder, 90...communication circuit, 100...surgical support robot, 110...manipulator, 121...right image sensor, 122...left image sensor, 200...encoder, 300...head-mounted display, 400...surgical support server, 410...operation unit, 420...display unit, 430...audio input / output unit, 440...communication circuit, 450...interface, 460...CPU, 461...robot remote control unit, 462...image signal management unit, 463...image recognition processing unit, 470...storage unit

Claims

1. a head-mounted unit to be mounted on the observer's head; a first display unit for the right eye and a second display unit for the left eye; a frame that holds the first and second display units side by side in a longitudinal direction, the frame having a first protrusion and a second protrusion provided at a first end and a second end in the longitudinal direction, respectively; two caps detachably attached to the first and second protrusions of the frame; a first arm link unit attached to the head mounting unit so as to be rotatable about a first axis; a second arm link portion attached to the first arm link portion so as to be rotatable about a second axis and attached to an intermediate portion between the first end and the second end of the frame so as to be rotatable about a third axis; wherein the first to third axes are substantially parallel to one another, and each of the two caps has an attachment portion provided with a recess that fits into the first or second convex portion of the frame, and an edge portion that protrudes from the attachment portion along a plane that is substantially perpendicular to the first to third axes when each cap is attached to the first or second convex portion of the frame, and wherein by operating the frame, the first or second arm link portion rotates, making it possible to adjust the positions and angles of the first and second display units with respect to the head-mounted unit.

2. a first adjustment mechanism disposed on the first axis and having a handle for manually adjusting the magnitude of a minimum torque required to change the angle of the first arm link unit relative to the head mounting unit; a second adjustment mechanism disposed on the second axis and having a handle for manually adjusting the magnitude of a minimum torque required to change the angle of the second arm link portion relative to the first arm link portion; a third adjustment mechanism disposed on the third axis and having a handle for manually adjusting the magnitude of a minimum torque required to change the angle of the frame relative to the second arm link portion; and 2. The head-mounted display according to claim 1, further comprising at least one of: a display unit configured to display a display image of the first and second display units relative to the head-mounted unit; and a display unit configured to display a display image of the first and second display units relative to the head-mounted unit;

3. 3. A head-mounted display as described in claim 1 or 2, wherein the second arm link portion is attached to a substantially central portion between the first end and the second end of the frame so as to be rotatable about the third axis.

4. The head-mounted display according to any one of claims 1 to 3, further comprising a decoder that generates a right-eye image signal and a left-eye image signal based on a three-dimensional image signal, and supplies the right-eye image signal and the left-eye image signal to the first and second display units, respectively, thereby displaying a three-dimensional stereoscopic image on the first and second display units.

5. a plurality of image pickup elements that capture an image of a subject from within the endoscope and generate a plurality of image signals with parallax; an encoder for generating a three-dimensional image signal based on the plurality of image signals; The head-mounted display according to claim 4; A surgical support system comprising:

6. a recording medium having recorded thereon a program for causing a CPU to execute the steps of (a) recognizing a predetermined site or region in an image of a patient represented by a three-dimensional image signal, and (b) adding information serving as a guide for surgery to the three-dimensional image signal by performing image processing on the three-dimensional image signal based on the image recognition result; a head-mounted display according to claim 4, which displays a three-dimensional stereoscopic image based on a three-dimensional image signal supplied from the CPU; A surgical support system comprising:

Images