Work support system, head-mounted display, and work support method

Abstract

To support work without using a monitor.SOLUTION: A work support system includes: a setting information acquisition unit 101 which acquires setting information including information indicating relative coordinates with respect to a reference position, the coordinates being coordinates of one or more work places, for each model; a reference position specifying unit 102 which specifies the reference position; a model specifying unit 103 which specifies a model; a line-of-sight position detection unit 104 which detects a position of a line of sight of a worker; a line-of-sight position determination unit 106 which determines the position of the line of sight is normal or not on the basis of the setting information acquired by the setting information acquisition unit 101, the reference position specified by the reference position specifying unit 102, and a positional relationship between the current work place based on the model specified by the model specifying unit 103 and the position of the line of sight detected by the line-of-sight position detection unit 104; and a work status notification unit 108 which displays, when the line-of-sight position determination unit 106 determines that the position of the line of sight is abnormal, information indicating anomaly on a screen of a head-mounted display 1.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a work assistance system, a head mounted display, and a work assistance method that assist a worker in performing work. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a work support system that supports a worker's work is known (see, for example, Patent Document 1).

[0003] On the other hand, in the tightening operation using a torque wrench, since it is necessary to balance the tightening, the order in which the tightening is performed is determined for the multiple tightening target locations. In the conventional tightening operation support system for supporting the tightening operation, an AR marker is attached to the end of the torque wrench, and a camera that photographs the area where the tightening operation is performed and a monitor that displays information about the tightening operation are provided. In this tightening operation support system, the position of the AR marker is detected from the image photographed by the camera, and the position of the AR marker is determined to be located at the location that is the current tightening target, thereby determining whether the tightening is good or bad. Here, when the tightening operation support system determines that the tightening is incorrect, it displays on the monitor a message indicating that the tightening is incorrect. In addition, the tightening operation support system repeats the above determination according to the tightening order, thereby making it possible to determine whether the tightening operation has been performed in the specified order for the specified tightening target locations. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2010-61566 A Summary of the Invention [Problem to be solved by the invention]

[0005] In this way, the conventional tightening operation support system supports the tightening operation using a monitor. However, in this case, the worker needs to perform the tightening operation while checking the monitor, which is cumbersome. The same can be said about the conventional operation support system that supports operations other than the tightening operation.

[0006] The present disclosure has been made to solve the above-mentioned problems, and has an object to provide a work assistance system that is capable of assisting work without using a monitor. [Means for solving the problem]

[0007] The work assistance system disclosed herein is characterized by comprising a setting information acquisition unit that acquires setting information including information indicating the coordinates of one or more work locations for each model, the coordinates being relative to a reference position; a reference position identification unit that identifies the reference position; a model identification unit that identifies the model; a gaze position detection unit that detects the position of the worker's gaze; a gaze position determination unit that determines whether the gaze position is normal based on the setting information acquired by the setting information acquisition unit, the reference position identified by the reference position identification unit, and the positional relationship between the current work location based on the model identified by the model identification unit and the gaze position detected by the gaze position detection unit; and a work status notification unit that, if the gaze position determination unit determines that the gaze position is not normal, displays an indication that the gaze position is not normal on a screen of the head-mounted display. Effect of the Invention

[0008] According to the present disclosure, with the above-mentioned configuration, it becomes possible to assist with work without using a monitor. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing a configuration example of a fastening operation support system according to a first embodiment; [Diagram 2] FIG. 2 is a diagram illustrating a configuration example of a head mounted display according to the first embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a configuration of a server according to the first embodiment. [Figure 4] 4 is a time chart showing an example of the operation of the fastening operation support system according to the first embodiment. [Diagram 5] 5 is a diagram illustrating an example of the operation of the fastening operation support system according to the first embodiment. FIG. [Figure 6] 5 is a diagram illustrating an example of the operation of the fastening operation support system according to the first embodiment. FIG. [Figure 7] 7A and 7B are diagrams illustrating an example of a hardware configuration of the head mounted display and the server according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings. Embodiment 1 1 is a diagram showing a configuration example of a fastening operation support system according to embodiment 1. That is, in the following, a case will be shown in which the operation support system is a fastening operation support system for supporting a worker in fastening operation using a torque wrench 4. As shown in FIG. 1, the tightening operation support system includes a head mounted display 1, a server 2, a receiver 3, and a torque wrench 4.

[0011] The head mounted display 1 is a device worn by a worker, and is a device that assists the worker in performing a tightening operation using a torque wrench 4. If the head mounted display 1 determines that the line of sight of the worker is not normal based on the line of sight of the worker and a torque-up signal received by the server 2, it displays a message indicating that the line of sight is not normal, thereby assisting the worker in performing a tightening operation using the torque wrench 4. The torque-up signal is a signal that indicates the tightening by the torque wrench 4. This torque-up signal includes, for example, information that enables the torque wrench 4 to be identified and information that indicates the date and time of the tightening by the torque wrench 4.

[0012] This head mounted display 1 may be any display that can be worn by a worker and display a screen superimposed on the worker's field of vision, such as an MR (Mixed Reality) device, AR (Augmented Reality) glasses, or VR (Virtual Reality) glasses. A configuration example of the head mounted display 1 will be described later.

[0013] The server 2 is an information processing device that manages a tightening operation using a torque wrench 4 by communicating with the head mounted display 1 and the receiver 3. The server 2 receives the torque-up signal received by the receiver 3. The server 2 is, for example, a PC. An example of the configuration of this server 2 will be described later.

[0014] The receiver 3 receives a torque-up signal from the torque wrench 4 .

[0015] The torque wrench 4 is a tool used by an operator during work. The torque wrench 4 is connected to the receiver 3, generates a torque-up signal in response to a tightening operation performed by the operator, and transmits the torque-up signal to the receiver 3.

[0016] Next, a configuration example of the head mounted display 1 in the first embodiment will be described with reference to FIG. As shown in FIG. 2, the head mounted display 1 in embodiment 1 includes a setting information acquisition unit 101, a reference position identification unit 102, a model identification unit 103, a gaze position detection unit 104, a signal acquisition unit 105, a gaze position determination unit 106, a notification transmission / reception unit 107, a work status notification unit 108, an image transmission unit 109, and a work log transmission unit 110.

[0017] The setting information acquisition unit 101 acquires setting information. At this time, for example, the setting information acquisition unit 101 acquires the setting information from the server 2 by making an acquisition request to the server 2. Alternatively, the setting information acquisition unit 101 may acquire the setting information held inside the head mounted display 1. The setting information includes, for example, information indicating the coordinates of one or more locations to be tightened for each model. The coordinates of the locations to be tightened are relative coordinates with respect to a reference position.

[0018] In addition, when there are multiple locations to be tightened that are associated with the model, information indicating the tightening order of the tightening locations is added to the setting information.

[0019] The reference position identifying unit 102 identifies a reference position. At this time, for example, the reference position specifying unit 102 detects a QR marker attached to a workbench or the like on which a worker performs work, and specifies the position of the QR marker as the reference position.

[0020] The model identification unit 103 identifies the model. At this time, for example, the model identification unit 103 detects a 2D code attached to a work instruction sheet or the like, and identifies the model from the 2D code.

[0021] The gaze position detection unit 104 detects the position of the worker's gaze. At this time, for example, the gaze position detection unit 104 detects the position of the worker's gaze by eye tracking.

[0022] The signal acquisition unit 105 acquires a torque-up signal from the server 2.

[0023] The gaze position determining unit 106 determines, based on the gaze position detected by the gaze position detecting unit 104, whether the gaze position is normal. At this time, the gaze position determination unit 106 determines whether the gaze position is normal based on the positional relationship between the current tightening point based on the setting information acquired by the setting information acquisition unit 101, the reference position identified by the reference position identification unit 102, and the model identified by the model identification unit 103, and the gaze position detected by the gaze position detection unit 104. That is, the gaze position determination unit 106 determines whether the gaze position is normal by determining whether the gaze position is within a threshold value for the current tightening point. The threshold value can be set appropriately.

[0024] The notification transmission / reception unit 107 transmits and receives notifications to and from the server 2 . For example, the notification transmitting / receiving unit 107 transmits a work start notification to the server 2 based on the model identified by the model identification unit 103. This work start notification includes, for example, information that enables the head mounted display 1 to be identified and information that indicates the model.

[0025] Also, for example, the notification transmitting / receiving unit 107 receives a notification from the server 2 that the image has been saved. Also, for example, the notification transmitting / receiving unit 107 receives a notification from the server 2 that the work log has been saved.

[0026] The work status notifying unit 108 displays the status of the fastening work. When the gaze position determination unit 106 determines that the gaze position is not normal, the work status notification unit 108 displays on the screen of the head mounted display 1 a message indicating that the gaze position is not normal. At this time, for example, the work status notification unit 108 may display, based on the determination result by the gaze position determination unit 106, on the screen of the head mounted display 1, an indication of the distance between the gaze position and the area that is currently being tightened.

[0027] In addition, the work status notification unit 108 may display, on the screen of the head mounted display 1, the location that is currently being tightened, based on the setting information acquired by the setting information acquisition unit 101, the reference position identified by the reference position identification unit 102, and the model identified by the model identification unit 103. At this time, for example, the work status notifying unit 108 first searches for the same model as the above model from the setting information, and extracts information indicating the coordinates of the location that is the current tightening target associated with the searched model. Then, the work status notifying unit 108 displays the location that is the current tightening target at a position on the screen based on the coordinates indicated by the extracted information and the reference position.

[0028] In addition, when there are multiple locations to be tightened that are associated with the model, the work status notification unit 108 performs the above display while switching to the next location to be tightened each time the tightening work is completed (for example, each time the notification transmission / reception unit 107 receives a notification that the image has been saved), in accordance with the tightening order of the locations to be tightened.

[0029] Furthermore, the work status notifying unit 108 may perform a display indicating the gaze position on the screen of the head mounted display 1 based on the gaze position detected by the gaze position detecting unit 104.

[0030] In addition, the work status notification unit 108 may display, on the screen of the head mounted display 1, the number of the location that is currently being tightened, based on the setting information acquired by the setting information acquisition unit 101 and the model identified by the model identification unit 103.

[0031] Furthermore, when the tightening work on the part to be tightened is completed (for example, when the notification transmission / reception unit 107 receives a notification of completion of image storage), the work status notification unit 108 may notify the user that the tightening work is completed, based on the setting information acquired by the setting information acquisition unit 101 and the model identified by the model identification unit 103. At this time, the work status notification unit 108 may display on the screen of the head mounted display 1 that the tightening work is completed, or may output a sound by an audio output device (not shown) of the head mounted display 1 that indicates that the tightening work is completed.

[0032] In addition, when a torque-up signal is acquired by the signal acquisition unit 105 and the gaze position determination unit 106 determines that the gaze position is not normal, the work status notification unit 108 may display on the screen of the head-mounted display 1 a message indicating that the work location may be incorrect. Furthermore, at this time, the work status notifying unit 108 may output a sound, through a sound output device (not shown) of the head mounted display 1, indicating that the work location may be incorrect.

[0033] When the gaze position determination unit 106 determines that the gaze position is normal, the image transmission unit 109 acquires an image showing the gaze position and transmits the image to the server 2. Note that information that enables the head mounted display 1 to be identified is added to the image. At this time, for example, the image transmitting unit 109 acquires an image displayed on the screen of the head mounted display 1, which image indicates the gaze position used in the determination by the gaze position determining unit 106.

[0034] In addition, even if the gaze position determination unit 106 determines that the gaze position is not normal, the image transmission unit 109 may obtain an image showing the gaze position and transmit the image to the server 2.

[0035] When the tightening work is completed, the work log transmission unit 110 creates a work log including traceability data and transmits the work log to the server 2. Examples of the traceability data include information that identifies the worker, information indicating the time required for the work, and information indicating the number of redoes. Note that information that identifies the head mounted display 1 is added to the work log.

[0036] 2 shows a case where image transmission unit 109 and work log transmission unit 110 are provided in head mounted display 1 (tightening work support system). However, image transmission unit 109 and work log transmission unit 110 are not essential components of head mounted display 1 (tightening work support system), and image transmission unit 109 and work log transmission unit 110 do not have to be provided in head mounted display 1 (tightening work support system).

[0037] Next, a configuration example of the server 2 in the first embodiment will be described with reference to FIG. As shown in FIG. 3, the server 2 in the first embodiment includes a setting information transmitting unit 201, a signal linking unit 202, a notification transmitting / receiving unit 203, an image saving unit 204, and a log saving unit 205.

[0038] The setting information transmission unit 201 detects an acquisition request from the head mounted display 1, and transmits setting information to the head mounted display 1 in response to the acquisition request.

[0039] The signal linking unit 202 receives a torque-up signal from the receiver 3 and transmits the torque-up signal to the head mounted display 1. The server 2 holds in advance information indicating the head mounted display 1 and torque wrench 4 used by the worker, and the signal linking unit 202 transmits the torque-up signal to the corresponding head mounted display 1 based on this information.

[0040] The notification transmitting / receiving unit 203 transmits and receives notifications to and from the head mounted display 1. For example, when the image storage unit 204 has completed storage of the image, the notification transmission / reception unit 203 transmits a storage completion notification of the image to the head mounted display 1 which is the transmission source of the image. Note that, when the image storage unit 204 has failed to store the image, the notification transmission / reception unit 203 transmits a notification to that effect to the head mounted display 1. Furthermore, for example, when the log storage unit 205 has completed storing the work log, the notification transmission / reception unit 203 transmits a storage completion notification of the work log to the head mounted display 1 that is the transmission source of the work log. Note that, when the log storage unit 205 has failed to store the work log, the notification transmission / reception unit 203 transmits a notification to that effect to the head mounted display 1. Also, for example, the notification transmitting / receiving unit 203 receives a work start notification from the head mounted display 1.

[0041] The image storage unit 204 acquires the image transmitted by the image transmission unit 109 of the head mounted display 1, and stores the image in a storage device (not shown) provided inside or outside the server 2. At this time, for example, the image storage unit 204 stores the image in a storage device in association with information indicating the model indicated in the work start notification received by the notification transmission / reception unit 203 based on information capable of identifying the head mounted display 1.

[0042] The log storage unit 205 acquires the work log transmitted by the work log transmission unit 110 of the head mounted display 1, and stores the work log in a storage device (not shown) provided inside or outside the server 2. At this time, for example, the log storage unit 205 stores the above work log in a storage device in association with information indicating the model indicated in the work start notification received by the notification transmission / reception unit 203 based on information capable of identifying the head mounted display 1. Furthermore, the log storage unit 205 may store not only the operation log but also a system log in the storage device.

[0043] 3 shows a case where the setting information transmission unit 201, the image storage unit 204, and the log storage unit 205 are provided in the server 2 (tightening work support system). However, the setting information transmission unit 201, the image storage unit 204, and the log storage unit 205 are not essential components of the server 2 (tightening work support system), and the setting information transmission unit 201, the image storage unit 204, and the log storage unit 205 do not have to be provided in the server 2 (tightening work support system).

[0044] 2 shows a case where the head mounted display 1 includes a setting information acquisition unit 101, a reference position identification unit 102, a model identification unit 103, a gaze position detection unit 104, a signal acquisition unit 105, a gaze position determination unit 106, a notification transmission / reception unit 107, a work status notification unit 108, an image transmission unit 109, and a work log transmission unit 110. However, the present invention is not limited to this, and in the tightening work assistance system, some or all of the above components may be provided outside the head mounted display 1, for example, in the server 2.

[0045] 3 shows a case where the server 2 includes a setting information transmission unit 201, a signal linking unit 202, a notification transmission / reception unit 203, an image storage unit 204, and a log storage unit 205. However, the present invention is not limited to this, and in the fastening operation assistance system, some or all of the above components may be provided outside the server 2, for example, in the head mounted display 1.

[0046] Next, an example of the operation of the fastening operation support system according to the first embodiment shown in FIGS. 1 to 3 will be described with reference to FIG. In the following, the components of the server 2 other than the signal linking unit 202 are realized by a web server and a control unit. The setting information acquisition unit 101 is assumed to have acquired setting information in advance. The gaze position detection unit 104 is assumed to detect the gaze position of the worker constantly or at predetermined intervals. The gaze position determination unit 106 is assumed to determine the normality of the gaze position constantly or at predetermined intervals. In the following, a case is shown in which there are multiple locations to be tightened that are associated with the model (n is 2 or more). In addition, a case is shown in which the work status notification unit 108 displays the current location to be tightened and the gaze position of the worker.

[0047] In an operation example of the tightening operation assistance system according to the first embodiment shown in FIGS. 1 to 3, first, as shown in FIG. 4, for example, in the head mounted display 1, the reference position identifying unit 102 identifies a reference position (step ST101). At this time, for example, the reference position specifying unit 102 detects a QR marker attached to a workbench on which a worker performs work, and specifies the position of the QR marker as a reference position. This enables the head mounted display 1 to associate the coordinates of the screen of the head mounted display 1 with the coordinates of the real space.

[0048] Next, in the head mounted display 1, the model identification unit 103 identifies the model (step ST102). At this time, for example, the model identification unit 103 detects a 2D code attached to a work instruction sheet or the like, and identifies the model from the 2D code.

[0049] Furthermore, the head mounted display 1 initializes (n=1) the number of the location to be tightened (step ST103).

[0050] Next, in the head mounted display 1, the notification transmitting / receiving unit 107 transmits a work start notification to the server 2 based on the model identified by the model identification unit 103, and in the server 2, the notification transmitting / receiving unit 203 receives the work start notification (steps ST104 and ST105). This work start notification includes, for example, information that enables the head mounted display 1 to be identified and information indicating the model.

[0051] Next, in the head mounted display 1, the work status notification unit 108 displays, on the screen of the head mounted display 1, an indication of the area that is currently being tightened, based on the setting information acquired by the setting information acquisition unit 101, the reference position identified by the reference position identification unit 102, and the model identified by the model identification unit 103 (step ST106). At this time, for example, the work status notifying unit 108 first searches for the same model as the above model from the setting information, and extracts information indicating the coordinates of the location that is the current tightening target associated with the searched model. Then, the work status notifying unit 108 displays the location that is the current tightening target at a position on the screen based on the coordinates indicated by the extracted information and the reference position.

[0052] For example, as shown by reference numeral 51 or 52 in Fig. 5, the work status notification unit 108 displays an icon indicating the location that is the current tightening target at a position corresponding to the location on the screen of the head mounted display 1. In the example of Fig. 5, a circle surrounding the location that is the current tightening target shown by reference numeral 51 and an arrow pointing to the location that is the current tightening target shown by reference numeral 52 are displayed.

[0053] Furthermore, the work status notifying unit 108 performs a display showing the gaze position on the screen of the head mounted display 1 based on the gaze position detected by the gaze position detecting unit 104 (step ST107).

[0054] For example, as indicated by reference numeral 53 in FIG. 5, the work status notifying unit 108 may display an icon indicating the viewpoint at a position on the screen of the head mounted display 1 that corresponds to the viewpoint.

[0055] In addition, the work status notification unit 108 may display, on the screen of the head mounted display 1, the number of the location that is currently being tightened, based on the setting information acquired by the setting information acquisition unit 101 and the model identified by the model identification unit 103.

[0056] For example, as indicated by reference numeral 54 in Fig. 5, the work status notifying unit 108 may display, on the screen of the head mounted display 1, the ordinal number of the current tightening target among all the tightening targets. In the example of Fig. 5, it is displayed that the current tightening target is the Nth tightening target among four tightening targets.

[0057] Furthermore, when the gaze position determination unit 106 determines that the gaze position is not normal, the work status notification unit 108 displays on the screen of the head mounted display 1 a message indicating that the gaze position is not normal. At this time, for example, the work status notification unit 108 may display, based on the determination result by the gaze position determination unit 106, on the screen of the head mounted display 1, an indication of the distance between the gaze position and the area that is currently being tightened.

[0058] For example, as shown by reference numeral 51 or 52 in Fig. 5, the work status notifying unit 108 displays an icon indicating the distance between the position of the viewpoint and the location at a position on the screen of the head mounted display 1 that corresponds to the location that is currently being tightened. In this case, for example, the work status notifying unit 108 expresses distance by displaying an icon having a different display form, such as a color, depending on whether the position of the viewpoint and the location are farther away or closer than a threshold value. For example, the work status notifying unit 108 may display the icon in red when the position of the gaze and the location are farther away than the threshold value (when the gaze position determining unit 106 determines that the gaze position is not normal), and may display the icon in green when the position of the gaze and the location are closer than the threshold value (when the gaze position determining unit 106 determines that the gaze position is normal). Also, for example, when the position of the viewpoint and the above-mentioned location are farther than a threshold value, particularly when the above-mentioned location does not exist within the screen of the head-mounted display 1, the work status notification unit 108 may display an arrow indicating the direction in which the above-mentioned location is located on the screen of the head-mounted display 1.

[0059] Thereafter, the worker uses the torque wrench 4 to perform the tightening operation. The torque wrench 4 generates a torque-up signal in response to a tightening operation by an operator, and transmits the torque-up signal to the receiver 3, which receives the torque-up signal.

[0060] Next, in the server 2, the signal linking unit 202 receives the torque-up signal from the receiver 3 and transmits the torque-up signal to the head mounted display 1, and in the head mounted display 1, the signal acquiring unit 105 acquires the torque-up signal (steps ST108 to ST110). The server 2 holds in advance information indicating the head mounted display 1 and torque wrench 4 used by the worker, and the signal linking unit 202 transmits the torque-up signal to the corresponding head mounted display 1 based on this information.

[0061] Next, in the head mounted display 1, the gaze position determining unit 106 determines whether the gaze position is normal, based on the gaze position detected by the gaze position detecting unit 104 (step ST111). At this time, the gaze position determination unit 106 determines whether the gaze position is normal based on the positional relationship between the current tightening point based on the setting information acquired by the setting information acquisition unit 101, the reference position identified by the reference position identification unit 102, and the model identified by the model identification unit 103, and the gaze position detected by the gaze position detection unit 104. That is, the gaze position determination unit 106 determines whether the gaze position is normal by determining whether the gaze position is within a threshold value for the current tightening point.

[0062] In this step ST111, if the gaze position determination unit 106 determines that the gaze position is normal, the image transmission unit 109 in the head mounted display 1 acquires an image showing the gaze position and transmits the image to the server 2, and in the server 2, the image storage unit 204 acquires the image (steps ST112 to ST114). At this time, for example, the image transmitting unit 109 obtains an image displayed on a screen of the head mounted display 1, which indicates the gaze position used in the determination by the gaze position determining unit 106. For example, as shown in Fig. 5, the image transmitting unit 109 obtains an image in which an icon indicating the gaze position, indicated by the reference numeral 53, is displayed. Note that information enabling the head mounted display 1 to be identified is added to the image.

[0063] Next, in the server 2, the image storage unit 204 stores the acquired image in a storage device (not shown) provided inside or outside the server 2 (step ST115). At this time, for example, the image storage unit 204 stores the image in a storage device in association with information indicating the model indicated in the work start notification received by the notification transmission / reception unit 203 based on information capable of identifying the head mounted display 1.

[0064] Next, in the server 2, the notification transmitting / receiving unit 203 transmits a storage completion notification of the image to the head mounted display 1 which is the transmission source of the image, and in the head mounted display 1, the notification transmitting / receiving unit 107 receives the storage completion notification of the image (steps ST116, ST117). Note that, if the image storage unit 204 fails to store the image, the notification transmitting / receiving unit 203 transmits a notification indicating that fact to the head mounted display 1.

[0065] Next, in the head mounted display 1, the work status notifying unit 108 notifies the user that the tightening work has been completed based on the setting information acquired by the setting information acquiring unit 101 and the model identified by the model identifying unit 103 (step ST118). At this time, the work status notifying unit 108 may display on the screen of the head mounted display 1 that the tightening work has been completed, or may output a sound by an audio output device (not shown) of the head mounted display 1 that the tightening work has been completed.

[0066] Furthermore, the head mounted display 1 updates the number of the part to be tightened to the next number (n=n+1) (step ST119). Then, the sequence returns to step ST 106. Then, the work status notifying unit 108 switches to the next part to be fastened and displays it according to the fastening order of the parts to be fastened. If the head mounted display 1 cannot update the number of the part to be tightened to the next number, that is, if the number of the part to be tightened currently is the final number, the sequence proceeds to step ST121.

[0067] On the other hand, if in step ST111 the gaze position determination unit 106 determines that the gaze position is not normal, in the head mounted display 1 the work status notification unit 108 displays on the screen of the head mounted display 1 a message indicating that the work location may be incorrect (step ST120). Furthermore, at this time, the work status notifying unit 108 may output a sound, through a sound output device (not shown) of the head mounted display 1, indicating that the work location may be incorrect.

[0068] For example, as indicated by reference numeral 61 in Fig. 6, the work status notifying unit 108 displays a message indicating that the work location may be incorrect on the screen of the head mounted display 1. Note that the display indicated by reference numeral 61 in Fig. 6 also displays a "Confirm" button to prompt the worker to make a confirmation.

[0069] In addition, even if the gaze position determination unit 106 determines in step ST111 that the gaze position is not normal, the image transmission unit 109 may obtain an image showing the gaze position and transmit the image to the server 2.

[0070] Then, the worker redoes the tightening work. In the example of Fig. 6, the worker presses the "Confirm" button. The sequence then returns to step ST108.

[0071] Next, in the head mounted display 1, when the tightening work is completed, the work log transmission unit 110 creates a work log including traceability data and transmits the work log to the server 2, and in the server 2, the log storage unit 205 acquires the work log (steps ST121, 122). Examples of the traceability data include information that can identify the worker, information indicating the time required for the work, and information indicating the number of redoes. Note that information that can identify the head mounted display 1 is added to the work log.

[0072] Next, in the server 2, the log storage unit 205 stores the acquired work log in a storage device (not shown) provided inside or outside the server 2 (step ST123). At this time, for example, the log storage unit 205 stores the above work log in a storage device in association with information indicating the model indicated in the work start notification received by the notification transmission / reception unit 203 based on information capable of identifying the head mounted display 1. Furthermore, the log storage unit 205 may store not only the operation log but also a system log in the storage device.

[0073] Next, in the server 2, the notification transmitting / receiving unit 203 transmits a storage completion notification of the work log to the head mounted display 1 which is the transmission source of the work log, and in the head mounted display 1, the notification transmitting / receiving unit 107 receives the storage completion notification of the work log (steps ST124, 125). Note that, if the log storage unit 205 fails to store the work log, the notification transmitting / receiving unit 203 transmits a notification indicating that fact to the head mounted display 1. After that, the sequence returns to step ST102.

[0074] In this way, the fastening operation support system according to the first embodiment uses the head mounted display 1, and when it is determined that the line of sight of the worker is not normal, a display is made to indicate that the line of sight is not normal. This makes it possible for the fastening operation support system according to the first embodiment to support the fastening operation without using a monitor.

[0075] Moreover, in the fastening work assistance system according to embodiment 1, it is possible to reduce the need for the worker to move his / her eyes to check work procedures, etc., compared to conventional systems, thereby enabling the speed of work to be improved and the physical burden to be reduced. That is, in the past, work instructions were displayed on a monitor. In this case, the worker had to check the monitor for each task. This required neck movement, which placed a physical burden on the worker. In addition, the worker had to move his / her line of sight a lot, and the large line of sight movement slowed down the worker's work speed. In contrast, the fastening operation support system according to the first embodiment makes it possible to display only the currently required information in the space in front of the worker. As a result, the fastening operation support system according to the first embodiment can always display information within the worker's field of vision regardless of the worker's posture, thereby reducing the physical burden on the worker. Furthermore, the fastening operation support system according to the first embodiment makes it possible for the worker to concentrate on the work, thereby enabling the work to be performed at a faster speed.

[0076] In addition, the fastening operation support system according to embodiment 1 ensures workability and reliability by working in conjunction with camera recognition and validation of the operation using external sensors, etc., thereby making it possible to improve workability compared to conventional systems. That is, conventionally, information obtained from tools, cameras, switches, etc. is displayed on a monitor. In this case, even if real-time values ​​such as current values ​​are displayed on the monitor, the worker cannot see the real-time values ​​while working. Also, even if an NG display is displayed on the monitor, the worker may overlook it. Therefore, the worker needs to check each work process. Also, working without looking at the screen may cause human error. In contrast, the fastening operation support system according to the first embodiment makes it possible to display information in the field of view at all times. Furthermore, the fastening operation support system according to the first embodiment makes it possible to display an alert in front of the operator when an NG occurs. Therefore, the above-mentioned information is always within the line of sight, so that it is not overlooked. Furthermore, the fastening operation support system according to the first embodiment makes it possible to issue a warning even in situations where mistakes are likely to occur, such as during a changeover.

[0077] In addition, the fastening work assistance system according to embodiment 1 can realize a simple system that is not affected by the size or shape of the work object and does not require devices such as a camera or external sensor, thereby simplifying the equipment. Conventionally, displays are made using cameras or PCs. In this case, camera positioning is limited depending on the shape or size of the work object. In addition, multiple cameras may be required. Furthermore, the number of devices required increases, making the system complicated and increasing maintenance costs. Also, the hardware and software must be reviewed each time a new model is added. In contrast, the tightening operation support system according to the first embodiment can display any shape and size on the real object. Furthermore, the tightening operation support system according to the first embodiment can be supported by the head mounted display 1, so that the maintenance cost is low. Furthermore, the tightening operation support system according to the first embodiment can be supported by only updating the software of the head mounted display 1.

[0078] Furthermore, in the fastening operation assistance system according to the first embodiment, it is possible to issue a warning in real time by utilizing the line of sight information, and it is also possible to store the information as evidence. That is, in the past, the use of the work instruction system was dependent on the worker. In this case, as the worker becomes accustomed to the work, he or she stops looking at the monitor. As a result, mistakes in work may occur. Also, the worker may not know whether he or she is looking at the things he or she should be looking at while working. As a result, when a mistake is discovered, it is up to the worker to report it himself or herself. In contrast, the fastening operation support system according to the first embodiment can always store the information as a monitoring log by tracking the line of sight. Therefore, when a mistake is discovered, it can be confirmed. Furthermore, the fastening operation support system according to the first embodiment can monitor the line of sight and display a warning when the line of sight is not on the work location.

[0079] Moreover, the fastening operation support system according to the first embodiment can be used to provide work procedure guidance to new employees, which is expected to reduce training costs. That is, in the past, it was necessary to confirm the work procedures based on an instructor or a procedure manual. In this case, it was difficult to secure the man-hours and time for both the instructor and the newcomer. Also, the procedure manual can be difficult to understand, or it can get in the way. Also, there is a difference in the level of instruction given by the instructor. Also, it is difficult to understand how an experienced worker moves their eyes. In contrast, the tightening operation support system according to the first embodiment can display the work procedure on the actual object. As a result, the tightening operation support system according to the first embodiment allows a new employee to check the procedure by himself. Furthermore, the tightening operation support system according to the first embodiment is easy to understand because it can display on the actual object. Furthermore, the tightening operation support system according to the first embodiment makes it possible to provide training of uniform quality. Furthermore, the tightening operation support system according to the first embodiment can also display where the worker is looking based on line-of-sight information.

[0080] In the above description, the case has been described in which the work performed by the worker is a tightening work, and the work assistance system is a tightening work assistance system that assists in the tightening work. However, the work assistance system is not limited to this, and may be a work assistance system that assists in work other than the fastening work.

[0081] As described above, according to the first embodiment, the work support system includes a setting information acquisition unit 101 that acquires setting information including information indicating the coordinates of one or more work locations for each model, the coordinates being relative to a reference position, a reference position identification unit 102 that identifies the reference position, a model identification unit 103 that identifies the model, a gaze position detection unit 104 that detects the position of the worker's gaze, a gaze position determination unit 106 that determines whether the gaze position is normal based on the positional relationship between the current work location based on the setting information acquired by the setting information acquisition unit 101, the reference position identified by the reference position identification unit 102, and the model identified by the model identification unit 103 and the gaze position detected by the gaze position detection unit 104, and a work status notification unit 108 that displays a message indicating that the gaze position is not normal on the screen of the head mounted display 1 when the gaze position determination unit 106 determines that the gaze position is not normal. As a result, the tightening work support system according to the first embodiment can support the work without using a monitor.

[0082] Finally, with reference to FIG. 7, a hardware configuration example of the head mounted display 1 and the server 2 according to the first embodiment will be described. The functions of the setting information acquisition unit 101, the reference position identification unit 102, the model identification unit 103, the gaze position detection unit 104, the signal acquisition unit 105, the gaze position determination unit 106, the notification transmission / reception unit 107, the work status notification unit 108, the image transmission unit 109, and the work log transmission unit 110 in the head mounted display 1 are realized by a processing circuit 501. The processing circuit 501 may be dedicated hardware as shown in Fig. 7A, or may be a CPU (also called a Central Processing Unit, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor)) 502 that executes a program stored in a memory 503 as shown in Fig. 7B.

[0083] When the processing circuit 501 is a dedicated hardware, the processing circuit 501 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. The functions of each of the setting information acquisition unit 101, the reference position identification unit 102, the model identification unit 103, the gaze position detection unit 104, the signal acquisition unit 105, the gaze position determination unit 106, the notification transmission / reception unit 107, the work status notification unit 108, the image transmission unit 109, and the work log transmission unit 110 may be realized by the processing circuit 501 individually, or the functions of each unit may be realized by the processing circuit 501 collectively.

[0084] When the processing circuit 501 is a CPU 502, the functions of the setting information acquisition unit 101, the reference position identification unit 102, the model identification unit 103, the gaze position detection unit 104, the signal acquisition unit 105, the gaze position determination unit 106, the notification transmission / reception unit 107, the work status notification unit 108, the image transmission unit 109, and the work log transmission unit 110 are realized by software, firmware, or a combination of software and firmware. The software and firmware are described as programs and stored in the memory 503. The processing circuit 501 realizes the functions of each unit by reading and executing the programs stored in the memory 503. That is, the head mounted display 1 includes a memory 503 for storing a program that, when executed by the processing circuit 501, results in the execution of each step shown in FIG. 4, for example. These programs can also be said to cause a computer to execute procedures and methods of the setting information acquisition unit 101, the reference position identification unit 102, the model identification unit 103, the gaze position detection unit 104, the signal acquisition unit 105, the gaze position determination unit 106, the notification transmission / reception unit 107, the work status notification unit 108, the image transmission unit 109, and the work log transmission unit 110. Here, examples of the memory 503 include non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically EPROM), magnetic disks, flexible disks, optical disks, compact disks, mini disks, and DVDs (Digital Versatile Discs).

[0085] The functions of the setting information acquisition unit 101, the reference position identification unit 102, the model identification unit 103, the gaze position detection unit 104, the signal acquisition unit 105, the gaze position determination unit 106, the notification transmission / reception unit 107, the work status notification unit 108, the image transmission unit 109, and the work log transmission unit 110 may be partially realized by dedicated hardware and partially realized by software or firmware. For example, the setting information acquisition unit 101 may be realized by a processing circuit 501 as dedicated hardware, and the reference position identification unit 102, the model identification unit 103, the gaze position detection unit 104, the signal acquisition unit 105, the gaze position determination unit 106, the notification transmission / reception unit 107, the work status notification unit 108, the image transmission unit 109, and the work log transmission unit 110 may be realized by the processing circuit 501 reading out and executing a program stored in the memory 503.

[0086] Thus, the processing circuitry 501 can realize each of the above-described functions by hardware, software, firmware, or a combination of these.

[0087] Furthermore, the functions of the setting information transmission unit 201, the signal linking unit 202, the notification transmission / reception unit 203, the image storage unit 204, and the log storage unit 205 in the server 2 are realized by a processing circuit 501. The processing circuit 501 may be dedicated hardware as shown in Fig. 7A, or may be a CPU 502 that executes a program stored in a memory 503 as shown in Fig. 7B.

[0088] When the processing circuit 501 is a dedicated hardware, the processing circuit 501 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination of these. The functions of each of the setting information transmission unit 201, the signal linking unit 202, the notification transmission / reception unit 203, the image storage unit 204, and the log storage unit 205 may be realized by the processing circuit 501, or the functions of each unit may be realized by the processing circuit 501 collectively.

[0089] When the processing circuit 501 is a CPU 502, the functions of the setting information transmission unit 201, the signal linking unit 202, the notification transmission / reception unit 203, the image storage unit 204, and the log storage unit 205 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 503. The processing circuit 501 realizes the functions of each unit by reading and executing the programs stored in the memory 503. That is, the server 2 includes a memory 503 for storing a program that, when executed by the processing circuit 501, results in the execution of each step shown in FIG. 4, for example. It can also be said that these programs cause a computer to execute the procedures and methods of the setting information transmission unit 201, the signal linking unit 202, the notification transmission / reception unit 203, the image storage unit 204, and the log storage unit 205. Here, memory 503 may be, for example, a RAM, a ROM, a flash memory, a non-volatile or volatile semiconductor memory such as an EPROM or an EEPROM, a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD.

[0090] It should be noted that the functions of the setting information transmission unit 201, the signal linking unit 202, the notification transmission / reception unit 203, the image storage unit 204, and the log storage unit 205 may be partially realized by dedicated hardware and partially realized by software or firmware. For example, the function of the setting information transmission unit 201 may be realized by a processing circuit 501 as dedicated hardware, and the functions of the signal linking unit 202, the notification transmission / reception unit 203, the image storage unit 204, and the log storage unit 205 may be realized by the processing circuit 501 reading out and executing a program stored in a memory 503.

[0091] Thus, the processing circuitry 501 can realize each of the above-described functions by hardware, software, firmware, or a combination of these.

[0092] Any of the components of the embodiments may be modified or omitted.

[0093] Various aspects of the present disclosure are summarized below as appendices.

[0094] (Appendix 1) a setting information acquisition unit that acquires setting information including information indicating coordinates of one or more work locations for each model relative to a reference position; a reference position specifying unit for specifying a reference position; A model identification unit that identifies a model; a gaze position detection unit for detecting the gaze position of a worker; a gaze position determination unit that determines whether a gaze position is normal based on a positional relationship between a current work location based on the setting information acquired by the setting information acquisition unit, the reference position identified by the reference position identification unit, and the model identified by the model identification unit, and the gaze position detected by the gaze position detection unit; and a work status notifying unit that, when the gaze position determining unit determines that the gaze position is not normal, displays a message indicating that the gaze position is not normal on a screen of the head mounted display; A work support system equipped with (Appendix 2) The work area is the area to be tightened. 2. A work support system according to claim 1. (Appendix 3) A signal acquisition unit for acquiring a torque-up signal is provided, When the signal acquisition unit acquires a torque-up signal and the gaze position determination unit determines that the gaze position is not normal, the work status notification unit displays, on a screen of the head-mounted display, a message indicating that the work location may be incorrect. 3. A work support system according to claim 2. (Appendix 4) The work status notifying unit displays a current work location on a screen of the head mounted display based on the setting information acquired by the setting information acquiring unit, the reference position identified by the reference position identifying unit, and the model identified by the model identifying unit. 4. The work support system according to claim 1, wherein: (Appendix 5) an image transmission unit that, when the gaze position determination unit determines that the gaze position is normal, obtains an image showing the gaze position and transmits the image; an image storage unit that acquires the image transmitted by the image transmission unit and stores the image in a storage device; 5. The work support system according to claim 1, (Appendix 6) a work log transmission unit that creates a work log including traceability data when the tightening work is completed and transmits the work log; a log storage unit that acquires the work log transmitted by the work log transmission unit and stores the work log in a storage device; 6. A work support system according to any one of claims 1 to 5, (Appendix 7) a setting information acquisition unit that acquires setting information including information indicating coordinates of one or more work locations for each model relative to a reference position; a reference position specifying unit for specifying a reference position; A model identification unit that identifies a model; a gaze position detection unit for detecting the gaze position of a worker; a gaze position determination unit that determines whether a gaze position is normal based on a positional relationship between a current work location based on the setting information acquired by the setting information acquisition unit, the reference position identified by the reference position identification unit, and the model identified by the model identification unit, and the gaze position detected by the gaze position detection unit; and a work status notifying unit that, when the gaze position determining unit determines that the gaze position is not normal, displays a message indicating that the gaze position is not normal on a screen of the own device; Equipped with a head mounted display. (Appendix 8) A setting information acquisition unit acquires setting information including information indicating coordinates of one or more work locations for each model relative to a reference position; A reference position specifying unit specifies a reference position; A model identification unit identifies a model; A step in which a gaze position detection unit detects a gaze position of a worker; a gaze position determination unit determining whether the gaze position is normal based on the setting information acquired by the setting information acquisition unit, the reference position identified by the reference position identification unit, and the positional relationship between a current working location based on the model identified by the model identification unit and the gaze position detected by the gaze position detection unit; a step of, when the gaze position determination unit determines that the gaze position is not normal, displaying on a screen of the head mounted display, a message indicating that the gaze position is not normal, by a work status notification unit; The work support method includes the steps of: [Explanation of symbols]

[0095] 1 Head Mounted Display, 2 Server, 3 Receiver, 4 Torque Wrench, 101 Setting Information Acquisition Unit, 102 Reference Position Identification Unit, 103 Model Identification Unit, 104 Gaze Position Detection Unit, 105 Signal Acquisition Unit, 106 Gaze Position Determination Unit, 107 Notification Transmission / Reception Unit, 108 Work Status Notification Unit, 109 Image Transmission Unit, 110 Work Log Transmission Unit, 201 Setting Information Transmission Unit, 202 Signal Linkage Unit, 203 Notification Transmission / Reception Unit, 204 Image Storage Unit, 205 Log Storage Unit, 501 Processing Circuit, 502 CPU, 503 Memory.

Claims

1. a setting information acquisition unit that acquires setting information including information indicating coordinates of one or more work locations for each model relative to a reference position; a reference position specifying unit for specifying a reference position; A model identification unit that identifies a model; A gaze position detection unit that detects the gaze position of a worker; a gaze position determination unit that determines whether a gaze position is normal based on a positional relationship between a current work location based on the setting information acquired by the setting information acquisition unit, the reference position identified by the reference position identification unit, and the model identified by the model identification unit, and the gaze position detected by the gaze position detection unit; and a work status notifying unit that, when the gaze position determining unit determines that the gaze position is not normal, displays a message indicating that the gaze position is not normal on a screen of the head mounted display; A work support system equipped with

2. The work area is the area to be tightened.

2. The work support system according to claim 1.

3. A signal acquisition unit for acquiring a torque-up signal is provided, When the signal acquisition unit acquires a torque-up signal and the gaze position determination unit determines that the gaze position is not normal, the work status notification unit displays, on a screen of the head-mounted display, a message indicating that the work location may be incorrect.

3. The work support system according to claim 2.

4. The work status notifying unit displays a current work location on a screen of the head mounted display based on the setting information acquired by the setting information acquiring unit, the reference position identified by the reference position identifying unit, and the model identified by the model identifying unit.

2. The work support system according to claim 1.

5. an image transmission unit that, when the gaze position determination unit determines that the gaze position is normal, obtains an image showing the gaze position and transmits the image; an image storage unit that acquires the image transmitted by the image transmission unit and stores the image in a storage device; 2. The work support system according to claim 1.

6. a work log transmission unit that creates a work log including traceability data when the tightening work is completed and transmits the work log; a log storage unit that acquires the work log transmitted by the work log transmission unit and stores the work log in a storage device; 2. The work support system according to claim 1.

7. a setting information acquisition unit that acquires setting information including information indicating coordinates of one or more work locations for each model relative to a reference position; a reference position specifying unit for specifying a reference position; A model identification unit that identifies a model; A gaze position detection unit that detects the gaze position of a worker; a gaze position determination unit that determines whether a gaze position is normal based on a positional relationship between a current work location based on the setting information acquired by the setting information acquisition unit, the reference position identified by the reference position identification unit, and the model identified by the model identification unit, and the gaze position detected by the gaze position detection unit; and a work status notifying unit that, when the gaze position determining unit determines that the gaze position is not normal, displays a message indicating that the gaze position is not normal on a screen of the own device; Equipped with a head mounted display.

8. A setting information acquisition unit acquires setting information including information indicating coordinates of one or more work locations for each model relative to a reference position; A reference position specifying unit specifies a reference position; A model identification unit identifies a model; A step in which a gaze position detection unit detects a gaze position of a worker; a gaze position determination unit determining whether the gaze position is normal based on the setting information acquired by the setting information acquisition unit, the reference position identified by the reference position identification unit, and the positional relationship between a current working location based on the model identified by the model identification unit and the gaze position detected by the gaze position detection unit; a step of, when the gaze position determination unit determines that the gaze position is not normal, displaying on a screen of the head mounted display, a message indicating that the gaze position is not normal, by a work status notification unit; The work support method includes the steps of:

Images