Head-mounted information output device

The head-mounted information output device allows direct object specification and information output by using a camera and input area to analyze and identify objects within the user's field of view, addressing the complexity of existing selection methods and enhancing usability.

JP7595728B2Active Publication Date: 2024-12-06MAXELL LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
JP2023183635
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-10-26
Publication Date
2024-12-06
Estimated Expiration
2039-12-27

AI Technical Summary

Technical Problem

Existing head-mounted displays require complex image display processing to specify and select objects within the user's field of view, making it difficult for users to easily obtain information about desired objects.

Method used

A head-mounted information output device equipped with a camera to capture the landscape image, an input area for object specification, and components to analyze and identify the specified object, allowing direct selection and output of object information without pre-displaying images.

Benefits of technology

Enables easy and direct specification of objects within the user's field of view, providing real-time information output through text and audio, enhancing user experience and usability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595728000001
    Figure 0007595728000001
  • Figure 0007595728000002
    Figure 0007595728000002
  • Figure 0007595728000003
    Figure 0007595728000003
Patent Text Reader

Abstract

To provide a head-mount type information output device which directly designates a real object within a field of view of a user, without displaying an image including the object, and acquires and outputs information regarding the designated object.SOLUTION: A head-mount type information output device 100 comprises: a camera 111 which photographs a scene image within a field of view of a user; an object designation unit 121 in which the user designates a position of a desired object within the scene image; an object identification unit 114 which identifies the object designated by the user; an object information acquisition unit 115 which acquires and generates object information regarding the object identified by the object identification unit; and an information output unit which outputs the object information acquired by the object information acquisition unit, by text characters from an information display unit 118 or by voice sound from a voice output unit 123. In the object designation unit 121, without displaying the image photographed by the camera 111 to the user, the user designates the position of the object while viewing the scene image within the field of view.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a head-mounted information output device that is worn on a user's head and allows the user to view images in real space or virtual space. [Background technology]

[0002] Head Mounted Displays (HMDs) worn on the user's head include those that display images of real space or virtual space on a display unit provided on the front (so-called video transmission type), and those that display virtual space images on a display unit while allowing the user to view real space images in front of the user (so-called optical transmission type). These methods seamlessly blend the real world and the virtual world in real time, allowing the user to experience as if a virtual object were present in the real world. In addition, such head mounted displays also make it possible to specify a desired object from the displayed image and perform various processes on the specified object.

[0003] Regarding the configuration for a user to specify a desired object, for example, Patent Document 1 describes a configuration that "includes a display device provided on the surface that is on the inside when the head mounted display is worn on the user's head, a touchpad provided on the surface that is on the outside when the head mounted display is worn on the user's head and that is capable of detecting when a part of the user's body has come into contact with or approached, and a display control unit that acquires image data generated based on input information detected by the touchpad that indicates that a part of the user's body has come into contact with or approached, and displays the acquired image data on the display device." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-81476 A Summary of the Invention [Problem to be solved by the invention]

[0005] The technology described in Patent Document 1 involves a touchpad having an input area associated with the display area of ​​a display device, whereby a user touches and inputs a specified position, and acquires image data of an object on the display device that corresponds to the specified position. In order to do this, it is of course necessary that an image including an object that is a candidate for selection by the user is displayed in the display area on the display device that corresponds to the touch position of the touchpad.

[0006] On the other hand, there are cases where a user wants to instantly know detailed information about an object in front of the user's field of view. In such a case, the issue is how to easily select and specify a desired object from the field of view. To achieve this, it is desirable for the user to perform a direct operation such as pointing at an object in the field of view. In the method of Patent Document 1, it is necessary to provide the user with an image including the object in advance, but this makes the processing and device configuration for image display complicated, and the user has to perform an indirect operation while looking at the displayed image, which is not convenient.

[0007] In view of the above problems, an object of the present invention is to provide a head-mounted information output device that directly specifies a real object within a user's field of vision without displaying an image including the object, and obtains and outputs information about the specified object. [Means for solving the problem]

[0008] In order to solve the above problems, one representative head-mounted information output device of the present invention comprises a camera that captures a scenic image within the user's field of view, an object designation unit having an input area corresponding to the image area captured by the camera and allowing the user to specify the position of a desired object within the scenic image, an object identification unit that analyzes the image portion within the image captured by the camera that corresponds to the position specified by the user by the object designation unit to identify the object specified by the user, an object information acquisition unit that acquires and generates object information related to the object identified by the object identification unit, and an information output unit that outputs the object information acquired by the object information acquisition unit as text characters from an information display unit or as audio from an audio output unit, and the object designation unit is configured to allow the user to specify the position of the object while visually recognizing the scenic image within the field of view, without displaying to the user the image captured by the camera. Effect of the Invention

[0009] According to the present invention, a user can directly select and designate a real object within the user's field of vision and output information relating to the designated object, thereby realizing a user-friendly head-mounted information output device. Furthermore, problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a usage form of a head-mounted information output device according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a head-mounted information output device according to a first embodiment. [Diagram 3] FIG. 4 is a diagram illustrating an information output operation according to the first embodiment. [Figure 4A] 11A to 11C are diagrams showing the operation when a desired image is designated and input with one finger. [Figure 4B] 11A to 11C are diagrams showing the operation when a desired image is designated and input with one finger. [Figure 4C] 11A to 11C are diagrams showing the operation when a desired image is designated and input with one finger. [Figure 4D]11A to 11C are diagrams showing the operation when a desired image is designated and input with one finger. [Figure 4E] 11A to 11C are diagrams showing the operation when a desired image is designated and input with one finger. [Figure 4F] 11A to 11C are diagrams showing the operation when a desired image is designated and input with one finger. [Figure 5A] 11A and 11B are diagrams showing an operation when a desired image is designated and input with a plurality of fingers. [Figure 5B] 11A and 11B are diagrams showing an operation when a desired image is designated and input with a plurality of fingers. [Figure 5C] 11A and 11B are diagrams showing an operation when a desired image is designated and input with a plurality of fingers. [Figure 5D] 11A and 11B are diagrams showing an operation when a desired image is designated and input with a plurality of fingers. [Figure 6A] 11A and 11B are diagrams showing an operation when a desired image is designated and input by swiping a finger. [Figure 6B] 11A and 11B are diagrams showing an operation when a desired image is designated and input by swiping a finger. [Figure 6C] 11A and 11B are diagrams showing an operation when a desired image is designated and input by swiping a finger. [Figure 6D] 11A and 11B are diagrams showing an operation when a desired image is designated and input by swiping a finger. [Figure 7] 11 is a flowchart showing a processing procedure of the head-mounted information output device. [Figure 8] FIG. 11 is a block diagram showing a configuration of a head-mounted information output device according to a second embodiment. [Figure 9] FIG. 11 is a diagram illustrating an information output operation according to the second embodiment. [Figure 10A] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; [Figure 10B] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; [Figure 10C] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; [Figure 10D] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; [Figure 11] FIG. 11 is a block diagram showing a configuration of a head-mounted information output device according to a third embodiment. [Figure 12]FIG. 11 is a diagram illustrating an information output operation according to the third embodiment. [Figure 13A] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; [Figure 13B] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; [Figure 13C] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; [Figure 13D] 5A to 5C are diagrams showing a specific example of an information output operation according to the embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Several embodiments of the head-mounted information output device of the present invention will be described below with reference to the drawings. In any embodiment, when a user selects and designates a desired object from the scenery seen in front of them, the designated object is identified, and information about it is acquired and output. In this case, the embodiments will be described separately depending on how the designated object is identified. In the following embodiments, the head-mounted information output device will be referred to as an "HMD." EXAMPLES

[0012] In the first embodiment, the scenery in front of the user is photographed by a camera, and a desired object designated by the user is identified and determined based on the photographed image.

[0013] 1 is a diagram showing a typical usage form of a head-mounted information output device (HMD) 100 according to Example 1. The HMD 100 is worn on the head of a user 10, and includes a camera 111, a touch panel 112, a touch pad 113, an object identification unit 114, an object information acquisition unit 115, an information display unit 118 consisting of an information projection unit 116 and an optical combiner 117, headphones 119, a distance sensor 120, and the like. Furthermore, the HMD 100 is capable of transmitting and receiving information to and from an information server 160 and a mobile information terminal 170.

[0014] A user 10 wearing the HMD 100 directly sees a scenery image 20 of the field of view in front of the user's eyes through an optical combiner 117 consisting of a half mirror and a transparent touch panel 112. The camera 111 captures and captures the scenery image 20 of the field of view in front of the user's eyes. The touch panel 112 allows the user 10 to specify and input a position within an input area with his / her finger. At that time, the image area of ​​the scenery image (called a camera image) captured and captured by the camera 111 and the input area of ​​the touch panel 112 are assigned with position coordinates within each area in correspondence with each other (so-called mapping processing is performed). Therefore, the user 10 can select an image portion that is associated by mapping processing from the camera image captured and captured by the camera 111 by specifying and inputting a desired position on the touch panel 112.

[0015] In this case, as shown in FIG. 1, touch panel 112 is placed in the field of view of user 10 and is arranged so that the input area on touch panel 112 coincides with the image area (area surrounded by dashed lines 151, 152) of scenery image 20 seen by user 10. As a result, the positions of scenery image 20 seen by user 10 through touch panel 112 and the camera image captured and imported by camera 111 coincide with each other on touch panel 112. Therefore, by specifying the position of a desired image (e.g. object 21) visible within the field of view on touch panel 112, user 10 can select the desired object specified from the camera image captured and imported by camera 111, and can perform an operation to directly select and specify an actual object within the user's field of view.

[0016] The object identification unit 114 analyzes the part of the camera image selected and specified by the touch panel 112, and identifies and determines what object the image shows. Furthermore, the object information acquisition unit 115 acquires and generates various information (called object information) related to the object identified and determined by the object identification unit 114. When analyzing the camera image and acquiring and generating the object information, necessary information is acquired from an external information server 160. Alternatively, this may be acquired from information data stored in the HMD 100.

[0017] The information display unit 118 projects the acquired object information as text characters using the information projection unit 116, forms an image via the optical combiner 117, and displays it to the user 10. This allows the user 10 to visually recognize the displayed object information superimposed on the scenery image 20 of the field of view in front of the user 10. Meanwhile, the headphones 119 notify the user 10 of the acquired object information by voice.

[0018] In addition to the touch panel 112, a touch pad 113 disposed on the side of the HMD 100 can be used as a means for the user 10 to select and designate a desired object from the camera image. The touch pad 113 operates a mouse pointer by tracing a flat panel sensor with a finger, and allows the same designation input operation as the touch panel 112.

[0019] Furthermore, it is also possible to use a portable information terminal 170 such as a smartphone or a smartwatch, which has a touch panel on its display surface and allows designation input operations similar to those of the touch panel 112 of the HMD 100. Position information designated and input on the portable information terminal 170 is imported into the HMD 100 by short-range wireless communication.

[0020] The method of using the touch pad 113 or the mobile information terminal 170 allows the user to freely choose the posture when performing the designation input operation, and can be said to be easier to operate. For example, if the touch pad 113 is provided on both the left and right sides of the user 10, it is convenient to touch the touch pad with the left hand when holding an umbrella in the right hand in rainy weather, and to touch the touch pad with the right hand when holding an umbrella in the left hand.

[0021] On the other hand, when using the touch pad 113 or the mobile information terminal 170, although the input area is mapped with the area of ​​the camera image, it is different from the case of using the touch panel 112 of the HMD 100, where the user inputs a designation by superimposing it on the scenery image 20 seen by the user. Therefore, the position designated by the user may deviate from the desired image position. As a countermeasure, the information display unit 118 may display a marker indicating the position designated by the user. This allows the user to confirm the position designated by the user (marker position) by superimposing it on the scenery image 20 seen by the user, and allows the user to reliably select and designate the desired object. This marker indicating the position designated by the user may also be displayed when the user designates on the touch panel 112 of the HMD 100.

[0022] The distance sensor 120 detects the distance to an object present ahead, and detects the distance to an actual object selected and designated by the user. The detected distance information is used as one piece of object information.

[0023] With the above configuration, when a user selects and designates a desired object from the scenery image in the field of view in front of the user, various object information about the selected object, including distance information, can be notified to the user by text characters or voice. This allows the user to visually confirm information about the desired object in real time along with the scenery image in the field of view in front of the user, realizing an easy-to-use information output device.

[0024] Fig. 2 is a block diagram showing the configuration of the HMD shown in Fig. 1. Repeated explanations of the components explained in detail in Fig. 1 will be omitted.

[0025] The HMD 100 has a camera 111, an object designation unit 121, an information display unit 118, a distance sensor 120, an audio output unit 123, and an audio input unit 124. It further includes an object identification unit 114, an object information acquisition unit 115, a communication unit 127, a control unit 129, a memory unit 132, and a short-range communication unit 133, and each of the components is connected to each other via a bus 140.

[0026] The camera 111 is installed in front of the HMD 100, captures the scenery in front of the user's eyes, and captures it in the HMD 100 as a camera image.

[0027] The object designation unit 121 is made up of a touch panel 112 or a touch pad 113, has an input area that corresponds to the image area of ​​the camera image captured by the camera 111, and the user designates a position within the input area by touching or approaching it with a finger. Note that two touch panels or touch pads may be installed in front of the user, one on each side, allowing the user to perform touch operations with either the left or right finger.

[0028] The information display unit 118 forms and displays object information such as text characters using an information projection unit (projector) 116 and an optical combiner 117. It also displays the operating state of the HMD 100 and notification information for the user.

[0029] The distance sensor 120 may be a LiDAR (Light Detection and Ranging) that irradiates a laser and measures the scattered light to detect the distance to an object and the state of the object, or a millimeter wave radar that emits millimeter waves and captures the reflected waves to detect the distance to an object and the state of the object. In this way, the distance to the object selected and specified by the user is detected. Furthermore, the state information of the object detected by the distance sensor 120 may be used for object analysis by the object identification unit 114.

[0030] The audio output unit 123 is composed of headphones 119, speakers, etc., and converts object information into audio and outputs it to the user. It also outputs the operating state of the HMD 100 and notification information to the user. The audio input unit 124 converts audio from the user into an audio signal and inputs it.

[0031] The object identification unit 114 analyzes the image of the position selected and designated by the object designation unit 121 among the camera images, and identifies and determines what the object is. The object information acquisition unit 115 acquires and generates various object information related to the object identified and determined by the object identification unit 114. Note that an external information server 160 is used to collect and acquire the object information. Alternatively, the object information may be acquired from information data 131 stored in the HMD 100.

[0032] The communication unit 127 is a communication interface that communicates with an external information server 160 via a wireless LAN, a wired LAN, or base station communication, and for wireless communication, it connects to a network 150 via a transmitting / receiving antenna 128 to transmit and receive information. The communication unit 127 can receive information required for image analysis and object information from the information server 160, and can also transmit and receive operation control signals to and from the information server 160. Note that long-distance wireless communication such as W-CDMA (Wideband Code Division Multiple Access) or GSM (Global System for Mobile communications, registered trademark) can be used for base station communication.

[0033] The control unit 129 is composed of a CPU and the like, and controls the operation of each component of the HMD 100 by executing a program 130 stored in a memory unit 132 .

[0034] The memory unit 132 is composed of a flash memory or the like, and stores programs 130 such as an operating system (OS) and an application for controlling operation used by the control unit 129. It also stores information data 131 such as images, sounds, and signals handled by the HMD 100. The information data 131 includes, for example, images captured by the camera 111, a desired image selected and specified, detection values ​​of the distance sensor 120, and object information acquired and generated.

[0035] The short-distance communication unit 133 is a communication interface that performs short-distance wireless communication with the portable information terminal 170. It receives designated position information designated and inputted on the portable information terminal 170, and transmits and receives control information and the like to and from the portable information terminal 170. The short-distance communication unit 133 uses, for example, an electronic tag, but is not limited to this, and may also use Bluetooth (registered trademark), IrDA (Infrared Data Association), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or wireless LAN (IEEE802.11a, etc.).

[0036] Next, the configurations of the information server 160 and the mobile information terminal 170, which are communication destinations of the HMD 100, will be described. The information server 160 is configured using an information generating section 161, a memory section 162, a control section 163, a communication section 164, and a transmitting / receiving antenna 165, and each of the components is connected to one another via a bus 166.

[0037] The information generating unit 161 generates information required for analyzing a selected and specified image and object information related to an identified object in response to a request from the HMD 100. The memory unit 162 is a flash memory or the like, and stores various information generated by the information generating unit 161, various programs used by the control unit 163 in the information server 160, and the like.

[0038] The communication unit 164 is a communication interface that transmits and receives information to and from the HMD 100 via the network 150 through the transmitting and receiving antenna 165. The control unit 163 is composed of a CPU and the like, and controls the operation of the information server 160 as a whole by executing programs such as an OS and an operation control application stored in the memory unit 162.

[0039] In response to a request from the HMD 100, the control unit 163 controls each component to transmit to the HMD 100 information necessary for image analysis generated by the information generation unit 161, object information related to the object, or this information temporarily stored in the memory unit 162 after generation.

[0040] The portable information terminal 170 is configured using a display unit 171, a touch panel 172, a memory unit 173, a control unit 174, and a short-range communication unit 175, and each of the components is connected to one another via a bus 176.

[0041] Touch panel 172 provided on the surface of display unit 171 has an input area that is assigned in correspondence with the image area of ​​the camera image captured by HMD 100, and accepts a user's input of a desired position. However, when the user inputs a position on touch panel 172, the user operates while looking at scenic image 20 within the field of vision ahead.

[0042] The control unit 174 is composed of a CPU or the like, and controls each component of the portable information terminal 170. The memory unit 173 is a flash memory or the like, and stores information generated by the portable information terminal 170 and various programs used by the control unit 174. The short-range communication unit 175 is a communication interface that performs short-range wireless communication with the HMD 100.

[0043] When the user designates and inputs a desired position within the input area using the touch panel 172 with a finger, information on the designated and input position is temporarily stored in the memory unit 173 and is also transmitted to the HMD 100 via the short-range communication unit 175. As a result, the HMD 100 makes it possible to select an object within the camera image that corresponds to the position designated by the user.

[0044] FIG. 3 is a diagram showing a schematic diagram of an information output operation in this embodiment. (a) is an example of a scenery image in the field of view in front of the user 10, and (b) is a camera image captured by taking a picture of the scenery image with the camera 111. (c) is a transparent touch panel 112, through which the user 10 can view the scenery image in (a). The input area of ​​the touch panel 112 in (c) is assigned in correspondence with the area of ​​the camera image in (b) (mapping process).

[0045] Assume now that the user 10, in an attempt to obtain information about a desired image (object 21) in the scenery image of (a), touches the corresponding position on the touch panel 112 of (c) with the finger 11.

[0046] The object identification unit 114 analyzes the image portion 22 corresponding to the position specified by the user in the camera image of (b), and identifies and determines that the object indicated by the image portion 22 is an "XX mushroom." The object information acquisition unit 115 queries the information server 160 of (d) to collect information on the identified object "XX mushroom." The information server 160 searches for the object inquired about, and returns information that "it is a poisonous mushroom" as object information on this XX mushroom.

[0047] The information display unit 118 (e) or the headphones 119 (f) provide the object information received from the information server 160 to the user in the form of text or voice. For example, it may output "This is a poisonous XX mushroom" or, including distance information, "There is a poisonous mushroom 15 meters ahead."

[0048] In this way, while viewing the scenery image via touch panel 112, user 10 can specify the position on touch panel 112 where the desired image (object 21) is visible, thereby instantly obtaining information about the desired object.

[0049] Here, there are several possible methods for the user 10 to select and designate a desired image (object 21) in the scenery image using the finger 11 on the touch panel 112 of (c), and these methods will be described. 4A to 4F are diagrams showing the operation when a desired image is designated and input with one finger.

[0050] Fig. 4A is an example of a scenery image 401 in the field of view in front of the user, which is a park scene with a fountain 402 and swings 403. Fig. 4B shows touch panel 112, which has an input area associated with the image area of ​​the camera image, and scenery image 401 in front of the user is viewed through touch panel 112. With one finger 11, the user specifies a position on touch panel 112 where the desired image, fountain 402, is visible (the scenery image at the specified position is shown by a dotted line).

[0051] The object identification unit 114 identifies and determines that the object is a “fountain”, and the object information acquisition unit 115 acquires and generates information about the fountain, which is the object, from the information server 160 .

[0052] Then, as shown in Fig. 4C, information 405 saying "This is a fountain, a device that spews water" is output by voice from headphones 119. Also, as shown in Fig. 4D, information display unit 118 displays similar information 406 in text characters in accordance with scenery image 401 of the field of view.

[0053] Furthermore, in response to a user request, it is also possible to generate object information including the distance to the object detected by distance sensor 120. Then, as shown in Fig. 4E, information 407 such as "There is a fountain 3 meters ahead" is output by voice from headphones 119. Also, as shown in Fig. 4F, information display unit 118 displays similar information 408 in text characters in accordance with scenery image 401 of the field of view.

[0054] 5A to 5D are diagrams showing the operation when a desired image is designated and input with a plurality of fingers. Fig. 5A is an example of a scenery image 501 in the immediate field of view, which is a park scene with a fountain 502, a child 503, a large tree 504, and the like. Fig. 5B shows touch panel 112, in which scenery image 501 in front of the user is viewed through touch panel 112. The user specifies positions on touch panel 112 with multiple fingers 12 so as to surround the desired images of fountain 502, child 503, and large tree 504 (the scenery image at the specified positions is shown by dotted lines).

[0055] The object identification unit 114 identifies and determines that there are multiple objects, which are a "fountain," a "child," and a "large tree," and the object information acquisition unit 115 acquires and generates multiple pieces of information regarding the objects, which are the fountain, the child, and the large tree.

[0056] Then, as shown in Fig. 5C, information 505 such as "In front of a large pine tree, there is a fountain that spouts water, and to the left of it there are elementary school children" is output as voice from headphones 119. Also, as shown in Fig. 5D, information display unit 118 displays similar information 506 in text form in accordance with scenery image 501 within the field of view. If the distance to the object is included in the object information, the information might be, for example, "There is a large tree 10 meters ahead, a fountain 3 meters in front of that, and a child 2 meters ahead and to the left of that."

[0057] In this way, the user can specify multiple images (objects) within the field of view using multiple fingers, thereby allowing information on multiple objects to be output simultaneously.

[0058] 6A to 6D are diagrams showing the operation of specifying and inputting a desired image by swiping a finger. Swipe is an operation of sliding a finger on the touch surface in a specific direction. Fig. 6A is an example of a scenery image 601 in the immediate field of view, which is a scene in a park with a flower bed 602, a fountain 603, a bench 604, and the like. Fig. 6B shows the touch panel 112, in which the scenery image 601 in front of the user is viewed via the touch panel 112. By swiping (indicated by arrow 13) the user's finger 11 on the touch panel 112, the user specifies the flower bed 602, the fountain 603, and the bench 604 that are in the swipe range (the scenery image at the specified position is shown by a dotted line).

[0059] The object identification unit 114 identifies and determines that the objects are a "flower bed", a "fountain", and a "bench", and the object information acquisition unit 115 acquires and generates information regarding the objects, that is, the flower bed, the fountain, and the bench.

[0060] Then, as shown in Fig. 6C, information 605 such as "On the left there is a flower bed with flowers planted, in the middle a fountain that spouts water, and on the right there is a bench" is output as voice from headphones 119. Also, as shown in Fig. 6D, information display unit 118 displays similar information 606 in text form in accordance with the scenery image 601 within the field of view. If the distance to the object is included in the object information, the information might be, for example, "There is a flower bed 4 meters ahead to the left, a fountain 3 meters ahead in the middle, and a bench 2 meters ahead to the right."

[0061] In this way, by swiping their finger, the user can designate multiple images (objects) that exist in the direction of the swipe within their field of vision, and can also confirm and recognize the placement of the objects within the scenery image.

[0062] In the above example, the case where the touch panel 112 is used to specify an object has been described, but the same operation is performed when the touch pad 113 or the touch panel 172 of the mobile information terminal 170 is used to specify an object. However, in this case, the form of inputting the specification by superimposing it on the scenery image seen by the user is different. Therefore, as described above, if the position specified by the user is displayed as a marker on the information display unit 118, the user can easily understand the relationship between the scenery image and the specified position. In this case, when multiple fingers are used as in FIG. 5B, multiple markers may be displayed, and when a finger is swiped as in FIG. 6B, a line-shaped marker may be displayed at the swipe position. This marker may also be displayed when the user specifies an object on the touch panel 112 of the HMD 100.

[0063] 7 is a flowchart showing the processing procedure of the HMD described above. Here, the flow from when a desired image (object) is designated by the user to when the object information is acquired and output is shown, and the following steps are performed by the control unit 129.

[0064] S301: The camera 111 captures the field of view in front of the user, and captures an image of the scenery seen by the user (camera image). S302: It is determined whether the user has selected and designated a desired image through the object designation unit 121 (such as the touch panel 112). If no designation has been input, the process waits until the designation is input, during which time the photographing in S301 is continued to capture the latest camera image.

[0065] S303: When a designation input is made, the object identification unit 114 refers to the camera image, analyzes the desired image corresponding to the position designated by the object designation unit 121, and identifies and judges the object shown in the desired image. At that time, when the user designates with one finger (as shown in FIG. 4B), one image is analyzed and one object is identified and judged. When the designation is with multiple fingers (as shown in FIG. 5B), the image within the range surrounded by the multiple fingers is analyzed, and when the designation is made by swiping a finger (as shown in FIG. 6B), the image within the swipe range is analyzed and multiple objects are identified and judged.

[0066] S304: The object information acquisition unit 115 acquires and generates object information regarding the object identified by the object identification unit 114. For this purpose, the information is collected by inquiring of the information server 160. Alternatively, the object information may be collected from the information data 131 stored in the HMD 100.

[0067] S305: When the selection specification input is made in S302, it is determined whether the user has requested distance detection to the object. If distance detection has been requested, the process proceeds to S306, and if distance detection has not been requested, the process proceeds to S308.

[0068] S306: The distance to the object is detected by the distance sensor 120. When there are multiple objects, this is performed for each object. At this time, the distance to each object included in the forward scenery image is measured two-dimensionally by the distance sensor 120, but since the scenery image and the object designation unit 121 (touch panel) are associated with each other in-area positions, the distance data of the designated object can be easily extracted. S307: The object information acquisition unit 115 adds the detected distance information to the object information acquired and generated in S304.

[0069] S308: The information display unit 118 or the voice output unit 123 displays the acquired and generated object information as text characters or outputs it as voice to notify the user. Of course, the information may be notified by both text characters and voice. S309: The above sequence is repeated until the selection input is completed.

[0070] In addition, the user's operation when requesting distance detection in S305 can be distinguished from the operation of selecting and inputting the desired image in S302 by pressing harder when specifying and inputting the desired image with a finger, pressing longer than a predetermined time, or rotating the finger around the desired image position.

[0071] Furthermore, when a user wants to know more detailed information about an object, the user can combine operations such as repeatedly specifying and inputting a desired image with one finger or specifying the image with one finger for a long period of time as a selection and designation operation.

[0072] Specifically, for example, in FIG. 4B, when a user designates the object once with one finger, the object information is described as "This is a fountain," and when a user designates the object twice with one finger, the object information is described in detail as "This is a fountain in the shape of a pelican." When a user designates the object three times, the object information is described as "A fountain in the shape of a pelican, with water spraying upwards from its beak," and when a user designates the object four times, the object information is described in greater detail as the number of designations increases. This allows the user to obtain sufficient object information by increasing the number of designation inputs until the user is satisfied.

[0073] Also, the voice input unit 124 in the HMD 100 may be used to capture the user's voice, and a detailed explanation may be given in the form of a response to the user's voice question. For example, if the user utters "Tell me more" after performing a designation input operation with one finger, text or voice will be provided, such as "It is a fountain in the shape of a white pelican, and water shoots upward from its beak in a varying intensity." This allows the user to know the object information in an even more convenient manner.

[0074] Furthermore, it is possible to mount a stereo microphone as the voice input unit 124, collect the voice of the object specified by the finger with the stereo microphone, and analyze it together with the camera image. For example, if there is a crow cawing at the position specified by the finger in a park, etc., object information such as "It's a crow cawing" is generated including the vocalization of the object, and notified by text characters or voice. This makes it possible to recognize the appearance and state of the object, including the vocalization from the object.

[0075] In this way, the head-mounted information output device (HMD) of this embodiment allows the user to confirm and recognize information about an object specified by a finger in text or voice while looking at the view ahead. In particular, for users with visual or hearing impairments, this is a convenient tool because it allows them to easily know information about objects in front of them. EXAMPLES

[0076] In the second embodiment, the view in front of the user is not photographed by a camera, but map data for the current position is acquired, and a desired object designated by the user is identified and determined based on the map data.

[0077] Fig. 8 is a block diagram showing the configuration of a head-mounted information output device (HMD) 100' according to Example 2. In Fig. 8, the parts denoted with the same reference numerals as in Example 1 (Fig. 2) are the same as those described above, and detailed descriptions thereof will be omitted and only different parts will be described.

[0078] The HMD 100' includes a 3D map acquisition unit 201 and a field of view map image generation unit 202 instead of the camera 111 in the first embodiment, and further includes a position sensor 203 and a geomagnetic sensor 204. The HMD 100' is also connected to a map providing server 180 via a network 150. The other configurations are the same as those in the first embodiment (FIG. 2).

[0079] The 3D map acquisition unit 201 collects and acquires a 3D map, which is map data that adds three-dimensional information in the height direction to a planar map, from an external map providing server 180 using the network 150, a cloud service, or the like. Note that the map providing server 180 from which the map data is acquired may be the same as the information server 160 from which the object information is subsequently acquired. The three-dimensional information of the 3D map includes information such as the elevation of the land and the height of buildings, which allows the user to know the topography and shape of buildings in a desired area.

[0080] The field of view map image generating unit 202 extracts map data within the user's field of view from the 3D map and generates a "field of view map image." The field of view map image is a map data of the range visible from the user's position arranged in correspondence with the position of the scenery image seen by the user.

[0081] The input area of ​​the object designation unit 121 (such as the touch panel 112) is assigned in correspondence with the image area of ​​the field of view map image generated by the field of view map image generation unit 202, and the user can specify a position to designate an object at the corresponding position in the field of view map image.

[0082] The position sensor 203 is, for example, a GPS receiver that receives signals from GPS (Global Positioning System) satellites in the sky, and detects the current position of the HMD 100' (that is, the current position of the user).

[0083] The geomagnetic sensor 204 detects the magnetic force of the earth and detects the direction in which the HMD 100' is facing (i.e., the direction in which the user is facing).

[0084] The distance sensor 120 detects the distance to an object selected and designated by the user, and also detects the height of the HMD 100' from the ground (that is, the height of the location of the user from the ground).

[0085] FIG. 9 is a diagram showing a schematic diagram of an information output operation in this embodiment. (a) is an example of a scenery image of the field of view in front of the user 10, and (b1) is a 3D map acquired by the 3D map acquisition unit 201 from an external information server 160. On this 3D map, the direction in which the user is facing detected by the geomagnetic sensor 204 is determined based on the position of the user detected by the position sensor 203 and the height position from the ground of the location of the user detected by the distance sensor 120.

[0086] (b2) is a view map image, which is generated by the view map image generating unit 202 by cutting out map data corresponding to the view ahead that can be seen by the user at the base point from the 3D map. At this time, the cut-out map data is arranged according to the user's view direction, so that the view image (a) seen by the user and the view map image (b2) correspond in position within the image. Here, the objects included in the view map image (b2) are shown in the same illustration as the objects in the view image (a), but in reality, the three-dimensional information included in the 3D map is arranged at the corresponding position.

[0087] (c) is a transparent touch panel 112, and the user 10 can view the scenery image of (a) through the touch panel 112. The input area of ​​the touch panel 112 of (c) is assigned in correspondence with the area of ​​the field of view map image of (b2) (mapping process). Here, assume that the user 10 touches the corresponding position on the touch panel 112 of (c) with his / her finger 11 in order to obtain information about a desired image (object 21) in the scenery image of (a).

[0088] The object identification unit 114 analyzes the image portion 22 corresponding to the position specified by the user in the field of view map image (b2) and identifies and determines what the specified object is. At that time, by referring to the map data of the 3D map (b1), it becomes clear that the object is simply a "school" or a "XX City XX Elementary School."

[0089] 3(d) to (f) in the first embodiment, the object information acquisition unit 115 queries the information server 160 to collect information on the identified object "XX Elementary School". The object information received from the information server 160 is then provided to the user by the information display unit 118 or headphones 119 in the form of text characters or voice.

[0090] Thus, in this embodiment too, the user 10 can instantly obtain information about a desired image (object) by specifying the position on the touch panel 112 where the desired image (object 21) is visible while viewing the scenery image via the touch panel 112.

[0091] 10A to 10D are diagrams showing a specific example of the information output operation of this embodiment. 10A shows a state in which a user 10 wearing an HMD 100' is positioned on a hill 700 and is looking at the surrounding scenery. A tower 702, buildings 703, 704, and the like are present in a surrounding scenery image 701. At this time, the HMD 100' obtains a 3D map from an external map providing server 180. Then, based on the position of the user 10 and the direction 705 in which the user 10 is facing, a field of view map image corresponding to the scenery image 701 that the user is looking at is generated. However, the generated field of view map image is not displayed, and is therefore invisible to the user 10.

[0092] 10B shows touch panel 112 having an input area corresponding to the image area of ​​the view map image, and a scenery image 701 in front of the user is viewed through touch panel 112. The user uses finger 11 to specify a position on touch panel 112 where a desired image, tower 702, can be seen (the scenery image at the specified position is shown by a dotted line).

[0093] The object identification unit 114 identifies and determines that the object is a “tower”, and the object information acquisition unit 115 acquires and generates information about the tower, which is the object, from the information server 160 .

[0094] Then, as shown in Fig. 10C, information 706 saying "This is a tower 1 kilometer ahead to the right, a broadcast relay station" is output as voice from headphones 119. Also, as shown in Fig. 10D, similar information 707 is displayed as text by information display unit 118.

[0095] In this embodiment, the user 10 can also specify a desired image in the scenery image by using multiple fingers as shown in Fig. 5B or swiping a finger as shown in Fig. 6B. Also, as described in the first embodiment, it is possible to display a marker indicating the position specified by the user.

[0096] In this way, in this embodiment as well, the user can confirm and recognize information about an object specified by a finger through text characters or voice while looking at the scenery ahead. However, in this embodiment, in order to recognize and determine the object specified by the user, a 3D map is acquired from the map providing server 180, and the map data is used to identify and determine the object. For example, fixed objects such as buildings and roads are registered in the 3D map as unchanging map data, so this can be used, and there is no need to take a picture of the scenery in front of you with a camera as in the first embodiment. In the above description, 3D map data is used, but similar operations can be achieved by using 2D map, i.e., two-dimensional map data. EXAMPLES

[0097] In the third embodiment, an image captured by a surveillance camera is acquired from an external device, and a desired object designated by a user is identified and determined based on the image captured by the surveillance camera.

[0098] FIG. 11 is a block diagram showing a configuration of a head-mounted information output device (HMD) 100″ according to the third embodiment. In FIG. 11, the parts denoted with the same reference numerals as in the first embodiment (FIG. 2) and the second embodiment (FIG. 8) are the same as those described above, and detailed description thereof will be omitted and only the different parts will be described.

[0099] The HMD 100" is configured using a field of view surveillance video acquisition unit 211 instead of the camera 111 in the first embodiment and the 3D map acquisition unit 201 and field of view map image generation unit 202 in the second embodiment. The HMD 100" is also connected to a surveillance video server 190 via a network 150. The other configurations are similar to those of the first and second embodiments.

[0100] The field of view surveillance video acquisition unit 211 acquires video captured by a surveillance camera installed outside from an external surveillance video server 190 via a network 150, a cloud service, etc. The surveillance video server 190 from which the video is acquired at this time may be the same as the information server 160 from which the object information is subsequently acquired.

[0101] Here, the surveillance video server 190 holds and manages video footage from numerous surveillance cameras that are installed widely throughout the area. The HMD 100" then sends information such as the user's position, height, and facing direction detected by the position sensor 203, geomagnetic sensor 204, and distance sensor 120 to the surveillance video server 190. The field of view video generation unit 191 of the surveillance video server 190 then extracts and generates surveillance camera video of the area in front of the user from the surveillance camera video it holds, and sends it to the HMD 100". The surveillance camera video acquired by the field of view surveillance video acquisition unit 211 is therefore arranged in accordance with the scenic image visible from the user's position, and will hereinafter be referred to as "field of view surveillance video."

[0102] The input area of ​​the object designation unit 121 (such as the touch panel 112) is assigned in correspondence with the image area of ​​the field of view monitoring image acquired by the field of view monitoring image acquisition unit 211, and the user can specify a position to designate an object at the corresponding position in the field of view monitoring image.

[0103] FIG. 12 is a diagram showing a schematic diagram of an information output operation in this embodiment. (a) is an example of a scenery image of the field of view in front of the user 10, and (b) is a field of view monitoring video acquired by the field of view monitoring video acquisition unit 211 from the monitoring video server 190. This field of view monitoring video is cut out from the video of each monitoring camera based on the position, height, and facing direction of the user 10. Therefore, the scenery image seen by the user in (a) and the field of view monitoring video in (b) correspond in position within the image. However, since the position of the user and the position of the monitoring camera are different, there may be objects that are invisible to the user but captured by the monitoring camera, or conversely, objects that are visible to the user but not captured by the monitoring camera.

[0104] (c) is a transparent touch panel 112, and the user 10 can view the scenery image of (a) through the touch panel 112. The input area of ​​the touch panel 112 of (c) is assigned in correspondence with the area of ​​the field of view monitoring image of (b) (mapping process).

[0105] Assume now that the user 10, in an attempt to obtain information about a desired image (object 21) in the scenery image of (a), touches the corresponding position on the touch panel 112 of (c) with the finger 11.

[0106] The object identification unit 114 analyzes the image portion 22 corresponding to the position specified by the user in the field of view monitoring image of (b) and identifies and determines what the specified object is. In this example, it turns out to be a "car."

[0107] 3(d) to (f) in the first embodiment, the object information acquisition unit 115 queries the information server 160 to collect information on the identified object "automobile". The object information received from the information server 160 is then provided to the user by the information display unit 118 or headphones 119 in the form of text characters or voice.

[0108] Thus, in this embodiment too, the user 10 can instantly obtain information about a desired image (object) by specifying the position on the touch panel 112 where the desired image (object 21) is visible while viewing the scenery image via the touch panel 112.

[0109] 13A to 13D are diagrams showing a specific example of the information output operation of this embodiment. FIG. 13A shows a view image 801 of the view ahead as seen by the user 10, in which a building 802 and a moving car 803 are present. At this time, the HMD 100″ requests surveillance camera images from the external surveillance video server 190. In response, the information server 160 extracts, generates, and provides a field of view surveillance image corresponding to the view seen ahead by the user from the surveillance camera images it holds. However, the HMD 100″ does not display the received field of view surveillance image, so it is not visible to the user.

[0110] 13B shows touch panel 112 having an input area corresponding to the image area of ​​the received visibility monitoring image, and a scenery image 801 in front of the user is viewed through touch panel 112. The user uses finger 11 to specify a position on touch panel 112 where a desired image, that of a car 803, is visible (the scenery image at the specified position is shown by a dotted line).

[0111] The object identification unit 114 identifies and determines that the object is a "moving automobile", and the object information acquisition unit 115 acquires and generates information about the automobile, which is the object, from the information server 160.

[0112] Then, as shown in Fig. 13C, information 806 such as "This is a minivan type car made by XX" is output as voice from headphones 119. Also, as shown in Fig. 13D, similar information 807 is displayed in text form by information display unit 118.

[0113] In this embodiment, the user 10 can also specify a desired image in the scenery image by using multiple fingers as shown in Fig. 5B or swiping a finger as shown in Fig. 6B. Also, as described in the first embodiment, it is possible to display a marker indicating the position specified by the user.

[0114] In this embodiment, the user can confirm and recognize information about an object specified by a finger by text characters or voice while looking at the view ahead. However, in this embodiment, in order to recognize and determine the object specified by the user, the image captured by the surveillance camera is used to identify and determine the object. Therefore, not only fixed objects as in the second embodiment, but also moving or changing objects such as cars and pedestrians can be confirmed and recognized in real time. Even if the object moves and is hidden behind a building and cannot be seen by the user, it may be possible for the surveillance camera installed near the object to capture it. Furthermore, since a 360-degree surveillance camera can capture images with almost no blind spots, it can be used as an effective means for tracking an object when an incident occurs, for example. In addition, information about an object that is not visible to the user may be confirmed and recognized by text characters or voice upon request from the user.

[0115] The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0116] 10: user, 100, 100', 100": head-mounted information output device (HMD), 111: camera, 112: touch panel, 113: touch pad, 114: object identification unit, 115: object information acquisition unit, 116: information projection unit, 117: optical combiner, 118: information display unit, 119: headphones, 120: distance sensor, 121: object designation unit, 123: audio output unit, 124: audio input unit, 12 7: communication unit, 129: control unit, 132: memory unit, 133: short-range communication unit, 150: network, 160: information server, 161: information generation unit, 170: portable information terminal, 171: display unit, 172: touch panel, 180: map providing server, 190: surveillance video server, 201: 3D map acquisition unit, 202: field of view map image generation unit, 203: position sensor, 204: geomagnetic sensor, 211: field of view surveillance video acquisition unit.

Claims

1. A head-mounted information output device that a user wears on his / her head and visually recognizes images in real space or virtual space, A camera that captures an image of the real space; a transmission type panel provided on a surface in a line of sight of a user when the head-mounted information output device is mounted on the user's head; an object designation unit that is provided integrally with the head-mounted information output device, has an input area to which position coordinates corresponding to an area of ​​an image captured by the camera are assigned, and that designates a position of a desired object in the real space that is viewed by the user through the transmissive panel; an object identification unit that analyzes an image portion corresponding to a position designated by a user using the object designation unit within a captured image captured by the camera, and identifies the object designated by the user; an object information acquisition unit that acquires object information regarding the object identified by the object identification unit; an information output unit that outputs the object information acquired by the object information acquisition unit as text characters from an information display unit or as voice from a voice output unit; A head-mounted information output device characterized in that the object designation unit allows the user to designate the position of the object while visually recognizing the real space within their field of vision, without displaying to the user the image captured by the camera.

2. 2. The head-mounted information output device according to claim 1, The head-mounted information output device is characterized in that the object designation unit is configured to designate an object within a range surrounded by a plurality of fingers by a user's contact with or proximity to the object designation unit.

3. 2. The head-mounted information output device according to claim 1, The head-mounted information output device is characterized in that the object designation unit allows a user to slide and swipe a finger to designate an object within a swipe range.

4. 2. The head-mounted information output device according to claim 1, A head-mounted information output device characterized in that when a user repeatedly specifies the same position to the object designation unit, or specifies the same position for longer than a predetermined period of time, the object information acquisition unit acquires more detailed object information regarding the object.

5. 2. The head-mounted information output device according to claim 1, Further, a distance sensor for detecting a distance to an object within the user's field of view is provided, A head-mounted information output device characterized in that, when a user requests distance information to an object, the object information acquisition unit acquires object information including distance information to the object detected by the distance sensor.

6. 2. The head-mounted information output device according to claim 1, Further, the device has a communication unit for transmitting and receiving information to and from an external information server, The head-mounted information output device, wherein the object information acquisition unit acquires object information relating to the object from the information server via the communication unit.

7. 2. The head-mounted information output device according to claim 1, Further, a voice input unit for inputting the voice of a user is provided, The head-mounted information output device is characterized in that the object information acquisition unit acquires more detailed object information about the object based on the user's question voice captured by the voice input unit.

8. 2. The head-mounted information output device according to claim 1, Further, a voice input unit is provided for collecting a voice uttered by an object designated by a user, The head-mounted information output device is characterized in that the object information acquisition unit acquires object information including audio information collected by the audio input unit.

9. 2. The head-mounted information output device according to claim 1, Further, the device has a short-range communication unit for transmitting and receiving information to and from an external mobile information terminal, A head-mounted information output device, characterized in that a user can specify the position of a desired object using a touch panel of the portable information terminal.

10. 2. The head-mounted information output device according to claim 1, The head-mounted information output device is characterized in that the information output unit displays a marker indicating a position designated by a user on the object designation unit.

11. A head-mounted information output device that a user wears on his / her head and visually recognizes images in real space or virtual space, a 3D map acquisition unit that acquires a 3D map, which is map data that adds three-dimensional information in the height direction to a planar map; a view field map image generating unit that extracts map data within the user's view field from the acquired 3D map to generate a view field map image; a position sensor for detecting a current position of the head-mounted information output device; a geomagnetic sensor for detecting a direction in which the head-mounted information output device is facing; a distance sensor for detecting a height of the head-mounted information output device from the ground; a transmission type panel provided on a surface in a line of sight of a user when the head-mounted information output device is mounted on the user's head; an object designation unit that is provided integrally with the head-mounted information output device, has an input area to which position coordinates corresponding to an image area of ​​the field of view map image generated by the field of view map image generation unit are assigned, and designates a position of a desired object in the real space that is visually recognized by the user through the transparent panel; an object identification unit that analyzes an image portion corresponding to a position designated by a user by the object designation unit within the field of view map image generated by the field of view map image generation unit, and identifies the object designated by the user; an object information acquisition unit that acquires object information regarding the object identified by the object identification unit; an information output unit that outputs the object information acquired by the object information acquisition unit as text characters from an information display unit or as voice from a voice output unit; The field of view map image generating unit extracts map data of a range visible in a direction detected by the geomagnetic sensor at the current position detected by the position sensor and the height from the ground detected by the distance sensor from the 3D map acquired by the 3D map acquiring unit, and generates a field of view map image; A head-mounted information output device characterized in that the object designation unit allows the user to designate the position of the object while visually recognizing the real space within their field of vision, without displaying to the user the field of vision map image generated by the field of vision map image generation unit.

12. A head-mounted information output device is connected to a surveillance video server that stores surveillance camera images, and is worn by a user on the head to view images of real space and virtual space. A communication unit for transmitting and receiving information to and from the surveillance video server; a visibility monitoring video acquisition unit that acquires visibility monitoring video from the monitoring video server; a position sensor for detecting a current position of the head-mounted information output device; a geomagnetic sensor for detecting a direction in which the head-mounted information output device is facing; a distance sensor for detecting a height of the head-mounted information output device from the ground; a transmission type panel provided on a surface in a line of sight of a user when the head-mounted information output device is mounted on the user's head; an object designation unit that is provided integrally with the head-mounted information output device, has an input area to which position coordinates corresponding to the field of view monitoring image acquired by the field of view monitoring image acquisition unit are assigned, and designates a position of a desired object in the real space that is visually recognized by the user through the transmissive panel; an object identification unit that analyzes an image portion corresponding to a position designated by a user by the object designation unit in the field of view monitoring image acquired by the field of view monitoring image acquisition unit, and identifies an object designated by the user; an object information acquisition unit that acquires object information regarding the object identified by the object identification unit; an information output unit that outputs the object information acquired by the object information acquisition unit as text characters from an information display unit or as voice from a voice output unit; The field of view surveillance video acquisition unit transmits information on the user's position, direction, and height detected by the position sensor, the geomagnetic sensor, and the distance sensor to the surveillance video server, and acquires from the surveillance video server a field of view surveillance video within the user's field of view that is generated by cutting out surveillance camera video held by the surveillance video server, A head-mounted information output device characterized in that the object designation unit allows the user to designate the position of the object while visually recognizing the real space within their field of vision, without displaying to the user the field of vision monitoring video acquired by the field of vision monitoring video acquisition unit.

Citation Information

Patent Citations

  • Head mount display

    JP2011198304A

  • Information display system, information display method and program for information display

    JP2013054661A

  • Head-mounted display, mobile information terminal, image processing apparatus, display control program, display control method, and display system

    JP2016081476A

  • Sound collection system and sound collection method

    JP2016146547A

  • Human body gesture-based region and volume selection for hmd

    JP2016514298A