Head-mounted information presentation device

Abstract

To provide a head-mounted information presentation device which can be easily manufactured, and can reduce manufacturing cost.SOLUTION: An information presentation device includes: a body 2; a wide angle image imaging part 3 which is provided on an upper edge part 11b of a front part 11 of the body 2, and images a face 10c including eyes 10b of a wearer 10, a body 13 and peripheral environment; a visual line detection part 6 for detecting the visual line of the wearer 10; a watching object detection part 7 for detecting a watching object existing in a region including the visual line of the wearer 10; a pair of information display parts 4 which are arranged on each of a pair of side parts 12 of the body 2, and display information related to the watching object; and a pair of half mirrors 5 which permeate light beyond the visual line of the wearer 10, and reflect information from the pair of information display parts 4 to both of the eyes 10b of the wearer 10.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a head-mounted information presentation device. [Background technology]

[0002] The head-mounted information presentation device described in Patent Document 1 includes a main body that covers both eyes of the wearer and a pair of electronic viewfinders attached to the main body for detecting the wearer's line of sight and displaying information to the wearer. Each electronic viewfinder includes a plate-shaped transmission element positioned in front of each eye of the wearer, with multiple openings formed in the transmission element that are inclined toward the center of the wearer's eye. Each electronic viewfinder also includes a light-projecting unit that projects infrared light onto one of the wearer's eyes through one of the openings, and a light-receiving unit that receives light irradiated from the light-projecting unit onto the eye through another of the openings. These light-projecting unit and light-receiving unit are used to detect the wearer's line of sight. Each electronic viewfinder also includes a display unit, and light from the display unit is focused by a lens and reaches the wearer's eyes through the multiple openings of the transmission element. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-144598 Summary of the Invention [Problem to be solved by the invention]

[0004] The head-mounted information presentation device of Patent Document 1 requires a dedicated light-receiving unit and light-projecting unit to detect the line of sight of one of the wearer's eyes. Therefore, two sets of light-receiving units and light-projecting units are required to detect the line of sight of both eyes, which increases the number of parts in the information presentation device and raises the risk of increased manufacturing costs. Furthermore, the information presentation device requires multiple openings in the transmission element of the electronic viewfinder to detect the wearer's line of sight and display information to the wearer, which complicates the manufacture of the information presentation device and increases manufacturing costs.

[0005] The present invention has been devised in consideration of the above technical problems, and one of its objects is to provide a head-mounted information presentation device that can be easily manufactured and that can reduce manufacturing costs. [Means for solving the problem]

[0006] The present invention relates to a head-mounted information presentation device comprising a main body, an imaging unit provided on the upper edge of the front part of the main body for capturing images of the wearer's face including the eyes, body and surrounding environment, a gaze detection unit for detecting the wearer's gaze, a gaze object detection unit for detecting a gaze object that exists in an area including the wearer's gaze, a pair of display units respectively provided on a pair of side parts of the main body for displaying information related to the gaze object, and a pair of half mirrors that transmit light in the direction of the wearer's gaze and reflect information from the pair of display units to both eyes of the wearer. [Effects of the Invention]

[0007] According to the present invention, the line of sight of the wearer is detected by one imaging unit, and information on an object being gazed at ahead of the line of sight is displayed on a pair of display units provided on a pair of side sections of the main body and directed to the wearer's eyes via a pair of half mirrors. This simplifies the configuration of the head-mounted information presentation device, making it easier to manufacture the information presentation device and reducing manufacturing costs. [Brief explanation of the drawings]

[0008] [Figure 1] 1A and 1B show a head-mounted information display device according to a first embodiment in a state where the device is mounted on the head of a wearer, in which FIG. 1A is a side view of the information display device, and FIG. 1B is a front view of the information display device. [Figure 2] 1 is a block diagram of a head-mounted information presentation device according to a first embodiment. [Figure 3] 1 is a flowchart showing a processing flow according to the first embodiment. [Figure 4] FIG. 10 is a block diagram of a head-mounted information presentation device according to a second embodiment. [Figure 5] FIG. 10 is an explanatory diagram showing transmission of information from a remote device to a head-mounted information presentation device of a second embodiment. [Figure 6] 10 is a flowchart showing the flow of processing in the second embodiment. [Figure 7] FIG. 10 is a block diagram of a head-mounted information presentation device according to a third embodiment. [Figure 8] 10 is a flowchart showing the flow of processing in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a head-mounted information presentation device according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0010] A head-mounted information presentation device (hereinafter referred to as "information presentation device") includes a wearing device 1 (FIG. 2) such as a helmet. As shown in FIGS. 1(a) and 1(b), the wearing device 1 includes a main body 2, a wide-angle image capturing unit 3, a pair of information display units 4, and a pair of half mirrors 5. As shown in FIG. 2, the information presentation device includes, in addition to the wearing device 1, a gaze detection unit 6, a gaze target detection unit 7, and an image search unit 8. The gaze detection unit 6, the gaze target detection unit 7, and the image search unit 8 are provided on a server 9 external to the wearing device 1. The gaze detection unit 6 and the like may be provided in a control device capable of communicating with the wearing device 1, rather than on the server 9.

[0011] The main body 2 is provided in a wearing device 1 that is worn on the head 10a of a wearer 10 working in a workplace, and is formed into a glasses-like shape as a whole, as shown in FIGS. 1(a) and 1(b). The workplace may be, for example, a factory or a construction site. In this embodiment, the workplace is an automobile factory, such as a door assembly workshop. The main body 2 has a front portion 11 that covers both eyes 10b of the wearer 10, and a pair of side portions 12 that extend rearward from both sides of the front portion 11 perpendicular to the front portion 11 and cover the sides of both eyes 10b of the wearer 10. The front portion 11 has a rectangular plate shape with a long side 11a extending in the left-right direction (the left-right direction in FIG. 1(b)). As shown in FIG. 1(b), the length of the long side 11a of the front portion 11 in the left-right direction is greater than the length of the face 10c of the wearer 10 in the left-right direction. The side portions 12 are rectangular plates with long sides 12a shorter than the long sides 11a of the front portion 11. When the wearer 10 wears the wearing device 1, a predetermined gap is provided between the eyes 10b of the wearer 10 and the front portion 11.

[0012] The wide-angle image capturing unit 3 is a spherical camera capable of capturing an image within a 180-degree range. The wide-angle image capturing unit 3 is provided in a central position in the left-right direction of the upper edge 11b of the front part 11 of the main body 2, with the lens facing downward. The wide-angle image capturing unit 3 captures an image of the face 10c including the eyes 10b (both eyes 10b) of the wearer 10 and the body 13 inside the main body 2, i.e., on the side of the eyes 10b of the wearer 10, and captures an image of the surrounding environment of the wearer 10 outside the main body 2, i.e., on the opposite side of the eyes 10b. The combined image capturing range inside the main body 2 and the image capturing range outside the main body 2 is a 180-degree range. The wide-angle image capturing unit 3 constantly captures the face 10c, body 13, and surrounding environment while the wearer 10 is working.

[0013] The pair of information display units 4 have the same configuration and are displays that display information in text format related to an object being gazed at by the wearer 10, which in this embodiment is related to parts and tools used in automobile factories. As shown in FIG. 1(b), the pair of information display units 4 are respectively disposed on a pair of side units 12 of the main body 2. Also, as shown in FIG. 1(a), the information display unit 4 is disposed at a position closer to the wide-angle image capturing unit 3 in the front-to-rear direction of the side units 12 (the left-to-right direction in FIG. 1(a)).

[0014] The pair of half mirrors 5 transmit light in the line of sight of the wearer 10 and reflect information from the pair of information display units 4 to the eyes 10b of the wearer 10. The pair of half mirrors 5 are similar rectangular plates that are larger than the eyes 10b of the wearer 10. The pair of half mirrors 5 are disposed in a pair of rectangular openings 11c that are provided in the front part 11 of the main body 2 at positions facing the eyes 10b of the wearer 10. The pair of half mirrors 5 are disposed so as to be inclined at an appropriate angle relative to the main body 2 when viewed from the vertical direction (the vertical direction in FIGS. 1(a) and 1(b)). Information from the pair of information display units 4 is reflected to the eyes 10b of the wearer 10 via the pair of half mirrors 5 disposed at this appropriate angle.

[0015] The gaze detection unit 6 acquires an image including both eyes 10b of the wearer 10 captured by the wide-angle image capturing unit 3, and uses this image as an input to detect the gaze direction of the wearer 10 using a pre-trained model for detecting a person's gaze direction. The pre-trained model is configured by using a plurality of images including both eyes of a person to pre-train pairs of both eyes in each image and the corresponding gaze direction for each eye.

[0016] The gazed object detection unit 7 detects a gazed object present in an area including the line of sight of both eyes 10b of the wearer 10. More specifically, the gazed object detection unit 7 crops an area including the line of sight of both eyes 10b detected by the line of sight detection unit 6 from the surrounding environment in the image captured by the wide-angle image capture unit 3, and detects an object present in this area as a gazed object by using an image recognition model, which in this embodiment is an image recognition model using deep learning. In this embodiment, the gazed object detection unit 7 crops an area in the line of sight of both eyes 10b of the wearer 10 from the image of the interior of a door assembly workshop captured by the wide-angle image capture unit 3, and detects door parts and assembly tools present in this area as a gazed object by using an image recognition model using deep learning. Note that the image recognition model does not have to be an image recognition model using deep learning, but may be another image recognition model using template matching, etc.

[0017] The image search unit 8 acquires information on the Internet about the gaze object detected from the image by the gaze object detection unit 7. In this embodiment, the information about the gaze object is a manual describing the weight and handling method of door parts and assembly tools, as well as the procedures for using these parts and tools. The information acquired by the image search unit 8 is displayed in text format on the information display unit 4 of the mounting device 1.

[0018] Next, the processing flow of the first embodiment will be described with reference to FIG.

[0019] First, in step S1, an image is acquired by the wide-angle image capturing unit 3. That is, an image including the face 10c including the eyes 10b of the wearer 10, the body 13, and the surrounding environment, which in this embodiment is the environment inside an automobile factory, is acquired by the wide-angle image capturing unit 3. Note that in this embodiment, the image of the body 13 is not directly related to the detection of the gazed object, but for example, the hands, which are part of the body 13 of the wearer 10, grasp the gazed object while working, so an image of the body 13 is also acquired.

[0020] Next, in step S2, the gaze detection unit 6 acquires the gazes of the eyes 10b of the wearer 10 as a process. When acquiring the gazes, as described above, an image including the eyes 10b of the wearer 10 and a model that has been trained in advance are used to acquire the gazes of the eyes 10b of the wearer 10.

[0021] Then, in step S3, the gazed-up object of the wearer 10 is detected as a process performed by the gazed-up object detection unit 7. When detecting this gazed-up object, as described above, an area including the line of sight of both eyes 10b of the wearer 10 is cropped from the surrounding environment in the image captured by the wide-angle image capturing unit 3, and the gazed-up object is detected by using an image recognition model using deep learning.

[0022] Next, in step S4, the image search unit 8 acquires information related to the gazed-up object (gazed-up object related information) on the Internet as a process.

[0023] Then, in step S5, information related to the gaze object is displayed in text format on the pair of information display units 4. The information displayed on the pair of information display units 4 is reflected to both eyes 10b of the wearer 10 via the pair of half mirrors 5.

[0024] As described above, in the first embodiment, the information presentation device includes the main body 2, the wide-angle image capture unit 3 provided on the upper edge 11b of the front part 11 of the main body 2 and capturing images of the face 10c including the eyes 10b of the wearer 10, the body 13, and the surrounding environment, and the gaze detection unit 6 that detects the gaze of the wearer 10. Therefore, the only component provided in the wearing device 1 for detecting the gaze of the wearer 10 is the wide-angle image capture unit 3 provided on the upper edge 11b of the front part 11, and there is no need to provide dedicated light-projecting units and light-receiving units in front of each eye to detect the gaze of each eye, as in the prior art. Therefore, in this embodiment, the number of components required for gaze detection is smaller than in the prior art, making it easier to manufacture the information presentation device and reducing manufacturing costs.

[0025] In addition, in the conventional technology, a display unit is disposed in an electronic viewfinder placed in front of the user's eyes. Furthermore, this display unit and the transmission element placed in front of the user's eyes have multiple openings formed therein for detecting the user's line of sight and displaying information to the user. As described above, in the conventional technology, the structure of the part of the information presentation device facing the user's eyes is complex, which makes manufacturing relatively difficult and increases the risk of manufacturing costs increasing.

[0026] In contrast to this, in this embodiment, a pair of information display units 4 are arranged on a pair of side sections 12 of the main body 2 where there is ample space, and further, a pair of plate-shaped half mirrors 5 are arranged in front of both eyes 10b to reflect information from the information display units 4. Therefore, since it is only necessary to arrange the pair of information display units 4 and the pair of half mirrors 5 without processing any components, the structure of the information presentation device can be simplified and manufacturing costs can be reduced.

[0027] As shown in Fig. 4, the information presentation device of the second embodiment is configured by eliminating the image search unit 8 from the information presentation device of the first embodiment shown in Fig. 2 and newly adding a sound collection unit 14, an automatic lip reading unit 15, a voice recognition unit 16, and a remote device 17. In the second embodiment, the wide-angle image capturing unit 3 captures an image of the face 10c of the wearer 10, including the eyes 10b (both eyes 10b) as well as the lips 10d (see Fig. 1(a)), which are part of the face 10c. The other imaging ranges of the wide-angle image capturing unit 3 are the same as those in the first embodiment.

[0028] The sound collection unit 14 is a microphone that collects sounds emitted by the wearer 10. The sound collection unit 14 is provided in the wearing device 1 at a position that does not obstruct the wide-angle image capturing unit 3 from capturing an image of the lips 10d.

[0029] The automatic lip reading unit 15 reads the voice coming from the wearer 10 by applying a well-known automatic lip reading method that reads the content of the voice (content of speech) of the wearer 10 from the movement of the lips 10d to an image including the movement of the lips 10d of the wearer 10 captured by the wide-angle image capturing unit 3. The automatic lip reading unit 15 is provided on a server 9 external to the wearing device 1.

[0030] The voice recognition unit 16 recognizes the voice information of the wearer 10 based on the voice collected by the sound collection unit 14 and the voice read by the automatic lip reading unit 15. In this embodiment, the voice information of the wearer 10 is voice information uttered to receive instructions from others on how to handle a part or tool, which is an object of gaze. Furthermore, when recognizing the voice information, estimation is performed assuming that the name of the object of gaze acquired through processing in the gaze detection unit 6 and the gaze object detection unit 7 and words contained in a manual related to the object of gaze are spoken. The voice recognition unit 16 is provided on a server 9 external to the wearing device 1.

[0031] The remote device 17 generates communication information as instructions for the wearer 10 based on the voice information acquired by the voice recognition unit 16, and transmits this communication information to the information display unit 4. The remote device 17 is configured as a personal computer installed in a location remote from the wearer 10 working in the workplace. A person other than the wearer 10 is waiting in front of the remote device 17, and this person generates the communication information for the wearer 10 in text format based on the voice information from the voice recognition unit 16. As shown in FIG. 5 , the remote device 17 transmits the communication information to the information display unit 4 via a relay device 18.

[0032] Next, the processing flow of the second embodiment will be described with reference to FIG.

[0033] First, in step S11, an image is acquired by the wide-angle image capturing unit 3. That is, the wide-angle image capturing unit 3 captures an image including the face 10c including the eyes 10b and lips 10d of the wearer 10, the body 13, and the surrounding environment.

[0034] Next, in step S2, the gaze detection unit 6 acquires the gazes of both eyes 10b of the wearer 10, and in step S3, the gazed-upon object detection unit 7 detects the object that the wearer 10 is gazing at.

[0035] After the gaze target is detected, in step S12, the movement of the lips 10d of the wearer 10 is acquired as a process performed by the automatic lip reading unit 15. Then, by applying well-known automatic lip reading to the movement of the lips 10d, the voice coming from the wearer 10 is read.

[0036] In parallel with step S12, in step S13, the sound emitted by the wearer 10 is acquired by the sound collection unit 14.

[0037] Next, in step S14, the voice recognition unit 16 performs processing to acquire voice information of the wearer 10 based on the voice of the wearer 10 read by the automatic lip reading unit 15 and the voice of the wearer 10 collected by the sound collection unit 14.

[0038] Then, in step S15, the remote device 17 acquires voice information from the wearer 10. Based on this voice information, a person waiting in front of the remote device 17 inputs information to be communicated to the wearer 10 into the remote device 17.

[0039] Next, in step S16, the remote device 17 transmits the communication information to the wearer 10, and in step S17, the communication information is displayed on the information display unit 4.

[0040] As described above, in the second embodiment, the voice recognition unit 16 recognizes voice information of the wearer 10 based on the voice read by the automatic lip reading unit 15 and the voice collected by the sound collection unit 14. Therefore, when the environment in which the wearer 10 is located is a noisy environment, even if the sound collection unit 14 is unable to sufficiently acquire the live voice uttered by the wearer 10, the lack of voice is compensated for by the voice read by the automatic lip reading unit 15. Therefore, the voice of the wearer 10 can be efficiently acquired even in a noisy environment.

[0041] Furthermore, in this embodiment, the remote device 17 generates instructions, i.e., communication information, for the wearer 10 based on the voice information recognized by the voice recognition unit 16, and transmits this communication information to the pair of information display units 4. This allows the wearer 10 to receive communication information from the remote device 17 even if the voice is made in a noisy environment, thereby improving work efficiency.

[0042] As shown in Figure 7, the information presentation device of the third embodiment is configured by eliminating the image search unit 8 from the information presentation device of the first embodiment shown in Figure 2, and newly adding a personal identification unit 19, an environment recognition unit 20, a motion recognition model switching unit 21, a posture detection unit 22, and a motion evaluation unit 23.

[0043] The personal identification unit 19 stores face data, such as facial photographs, captured in advance for all workers working in the automobile factory. The personal identification unit 19 compares the face data of each worker stored in the personal identification unit 19 with the face 10c of the wearer 10 captured by the wide-angle image capturing unit 3 while working, thereby identifying the wearer 10 as a specific individual. The personal identification unit 19 is provided on a server 9 external to the wearing device 1.

[0044] The environment recognition unit 20 stores data relating to a plurality of different surrounding environments acquired in advance for all work areas in the automobile factory, more specifically, data relating to a plurality of different surrounding environments including the shapes and arrangements of parts, tools, equipment, etc. The environment recognition unit 20 compares the data relating to the plurality of different surrounding environments stored in the environment recognition unit 20 with the surrounding environment captured by the wide-angle image capturing unit 3 during work, thereby identifying the captured surrounding environment, which in this embodiment is the surrounding environment in which the wearer 10 is located in the automobile factory. The environment recognition unit 20 is provided on a server 9 external to the wearing device 1.

[0045] The action recognition model switching unit 21 receives as input the individual identified by the individual identification unit 19, the surrounding environment recognized by the environment recognition unit 20, the gaze of the wearer 10 detected by the gaze detection unit 6, and the gazed object detected by the gazed object detection unit 7. The action recognition model switching unit 21 switches the action recognition model that identifies the action of a general worker (including the wearer 10) based on at least one of the individual identified by the individual identification unit 19 and the surrounding environment recognized by the environment recognition unit 20. The action recognition model switching unit 21 is provided on a server 9 external to the wearing device 1.

[0046] The action recognition model includes a series of basic actions performed by a general worker using each part and tool in an automobile factory, such as the start to finish of an automobile door assembly job.

[0047] The reason for using an individual as a parameter for switching between action recognition models is that, for example, if the individual wearer 10 has relatively little work experience, it is presumed that the wearer 10 is working in a workplace where the work is relatively easy, such as a workplace for assembling automobile doors, and this makes it possible to identify the location of the wearer 10 within an automobile factory. Furthermore, because the line of sight and gazed-up object of the wearer 10 are input to the action recognition model switching unit 21, it is possible to determine what type of tool the individual wearer 10 is using and on what part in the identified location, and to infer the posture of the individual while working.

[0048] Furthermore, if the wearer 10 is an expert, he or she may be in charge of multiple work areas, and therefore, if only the individual is used as a parameter for switching between action recognition models, it is difficult to identify the location of the wearer 10 within the automobile factory. Therefore, by using the individual and the surrounding environment as parameters for switching between action recognition models and further taking into account the line of sight and gaze target of the wearer 10, it is possible to determine what type of tool the wearer 10 is using and on what part in a specific surrounding environment within the automobile factory, and to infer the posture of the individual during this work.

[0049] Furthermore, by understanding the surrounding environment in which the wearer 10 is located and further taking into account the line of sight and gaze of the wearer 10, it is possible to determine what type of tool the wearer 10 is using and what part he is working on in the surrounding environment, and to infer the posture of the wearer 10 while working.

[0050] The posture detection unit 22 analyzes the body 13 of the wearer 10 imaged by the wide-angle image capturing unit 3 and extracts points such as the shoulders, elbows, and wrists of the wearer 10 to detect the posture of the wearer 10, including the positions of each part such as the arms. The posture detection unit 22 is provided on a server 9 external to the wearing device 1.

[0051] The action evaluation unit 23 evaluates the action of the wearer 10, more specifically the appropriateness of the action, by comparing the posture of the wearer 10 detected by the posture detection unit 22 with the action recognition model switched by the action recognition model switching unit 21. Here, the "appropriateness of the action" is determined to be appropriate when the work performed by the wearer 10 is a planned work, and further, when the multiple work steps constituting the work performed by the wearer 10 are performed in the correct order. The action evaluation unit 23 is provided on a server 9 external to the wearing device 1.

[0052] Next, the processing flow of the third embodiment will be described with reference to FIG.

[0053] First, in step S1, an image is acquired by the wide-angle image capturing unit 3. That is, the wide-angle image capturing unit 3 captures an image including the face 10c including the eyes 10b of the wearer 10, the body 13, and the surrounding environment.

[0054] Next, in step S2, the gaze detection unit 6 acquires the gazes of both eyes 10b of the wearer 10, and in step S3, the gazed-upon object detection unit 7 detects the object that the wearer 10 is gazing at.

[0055] In parallel with step S2, in step S31, the personal identification unit 19 performs processing to identify the wearer 10 as a specific individual based on the data relating to the face of each worker stored in the personal identification unit 19 and the face 10c of the wearer 10 captured by the wide-angle image capturing unit 3 during work.

[0056] Furthermore, in parallel with step S2 and step S31, in step S32, the environment recognition unit 20 performs processing to identify the surrounding environment in which the wearer 10 is located within the automobile factory based on data relating to the surrounding environment stored in the environment recognition unit 20 and the surrounding environment captured by the wide-angle image capturing unit 3 during work.

[0057] Next, in step S33, the action recognition model switching unit 21 switches the action recognition model based on at least one of the individual identified by the individual identification unit 19 and the surrounding environment recognized by the environment recognition unit 20. When switching the action recognition model, the line of sight and gaze target of the wearer 10 input to the action recognition model switching unit 21 are also taken into consideration.

[0058] After switching the action recognition model, in step S34, the posture detection unit 22 analyzes the body 13 of the wearer 10 imaged by the wide-angle image capturing unit 3, and detects the posture of the wearer 10.

[0059] Next, in step S35, the action evaluation unit 23 performs processing to evaluate the action of the wearer 10 based on the posture of the wearer 10 detected by the posture detection unit 22 and the action recognition model switched by the action recognition model switching unit 21. When evaluating this action, the appropriateness of the action of the wearer 10 is evaluated based on whether the wearer 10 performed the specified task and whether the order of multiple task steps that make up this task is correct.

[0060] Then, in step S36, the information display unit 4 displays the detection result, that is, whether the work performed by the wearer 10 was appropriate or not.

[0061] As described above, in the third embodiment, the individual identification unit 19 identifies the wearer 10 as a specific individual by comparing the data relating to the face of each worker stored in the individual identification unit 19 with the face 10c of the wearer 10 captured by the wide-angle image capturing unit 3 while working. By identifying the individual wearer 10 in this way, it is possible to determine whether the wearer 10 is a beginner or an expert. Furthermore, since beginners and those close to beginners are particularly likely to be in charge of a limited number of work areas, it is possible to easily limit the work area in which the wearer 10 is located.

[0062] Furthermore, in this embodiment, the environment recognition unit 20 identifies the surrounding environment in which the wearer 10 is located within the automobile factory by comparing data relating to the surrounding environment stored in the environment recognition unit 20 with the surrounding environment captured by the wide-angle image capturing unit 3 during work. If the surrounding environment is known, the work that the wearer 10 will perform in that surrounding environment is limited, and the posture of the wearer 10 can be easily estimated.

[0063] Furthermore, in this embodiment, the posture detection unit 22 analyzes the body 13 of the wearer 10 captured by the wide-angle image capturing unit 3 and extracts points such as the shoulders, elbows, and wrists of the wearer 10 to detect the posture of the wearer 10. Therefore, a series of movements of the wearer 10 while working on a gaze target can be acquired.

[0064] Furthermore, in this embodiment, the action recognition model switching unit 21 switches the action recognition model for identifying the action of a general worker (including the wearer 10) based on at least one of the individual identified by the individual identification unit 19 and the surrounding environment recognized by the environment recognition unit 20. This makes it possible to set a series of basic movements that a general worker will make in a specific surrounding environment. Note that the surrounding environment can be limited by the identified individual when the individual is a beginner or someone close to being a beginner, as described above.

[0065] Furthermore, in this embodiment, the action evaluation unit 23 evaluates the action of the wearer 10, more specifically, the appropriateness of the action, by comparing the posture of the wearer 10 detected by the posture detection unit 22 with the action recognition model switched by the action recognition model switching unit 21. This makes it possible to determine whether the wearer 10 performed a plurality of work steps in the correct order in a set task, and to discover missing work steps (work steps that were not performed). [Explanation of symbols]

[0066] 1. Mounting device 2. Main unit 3. Wide-angle image capture unit 4...Information display section 5. Half mirror 6. Gaze detection unit 7. Gaze object detection unit 8. Image search section 14...Sound collection section 15...Automatic lip reading section 16. Voice recognition unit 17. Remote device 19. Personal Identification Unit 20...Environmental recognition department 21. Motion recognition model switching unit 22. Posture detection unit 23. Operation evaluation section

Claims

1. a main body having a front portion that covers both eyes of the wearer and a pair of side portions that extend rearward from both sides of the front portion and cover the sides of both eyes of the wearer; an imaging unit provided on an upper edge of a front portion of the main body, for capturing images of the wearer's face including the eyes, body, and surrounding environment; a gaze detection unit that detects the gaze of the wearer captured by the imaging unit; a gaze object detection unit that detects a gaze object that exists in an area including the wearer's line of sight; a pair of display units disposed on a pair of side portions of the main body, respectively, for displaying information related to the gaze object; a pair of half mirrors respectively arranged in positions facing the wearer's eyes in the front part of the main body, the pair of half mirrors transmitting light ahead of the wearer's line of sight and reflecting the information from the pair of display units to the wearer's eyes; A head-mounted information presentation device comprising:

2. a lip reader that reads the voice generated by the wearer based on the movement of lips, which are part of the face of the wearer, captured by the imaging unit; a sound collection unit that collects sounds generated by the wearer; a voice recognition unit that recognizes voice information of the wearer based on the voice read by the lip reading unit and the voice collected by the sound collecting unit; The head-mounted information presentation device according to claim 1, further comprising:

3. 3. The head-mounted information presentation device according to claim 2, further comprising a remote device that generates instructions for the wearer based on the voice information and transmits the instructions to the pair of display units.

4. The head-mounted information presentation device of claim 1, characterized in that the main body, the pair of half mirrors, the imaging unit and the pair of display units are provided in a wearing device that is worn on the wearer's head, and the gaze detection unit and the gaze object detection unit are provided outside the wearing device.

5. 2. The head-mounted information presentation device according to claim 1, further comprising an individual identification unit that identifies the wearer as a specific individual based on the face of the wearer captured by the imaging unit.

6. The head-mounted information presentation device according to claim 5, further comprising an environment recognition unit that recognizes the imaged surrounding environment based on the surrounding environment imaged by the imaging unit and data relating to a plurality of different surrounding environments acquired in advance.

7. 7. The head-mounted information presentation device according to claim 6, further comprising a posture detection unit that detects a posture of the wearer based on an image of the wearer's body captured by the imaging unit.

8. a motion recognition model having basic motions of a general worker including the wearer; The head-mounted information presentation device of claim 7, further comprising an action recognition model switching unit that switches the action recognition model based on at least one of the individual identified by the individual identification unit and the surrounding environment recognized by the environment recognition unit.

9. The head-mounted information presentation device of claim 8, further comprising an action evaluation unit that evaluates the appropriateness of the wearer's actions based on the posture of the wearer detected by the posture detection unit and the action recognition model switched by the action recognition model switching unit.

Images