Work support system and work support method

Abstract

To provide a work support system or the like of high convenience.SOLUTION: The work support system comprises: a terminal device 10 including an imaging unit 11b for capturing an image of the vicinity of a region of a field of view of a worker and a first display unit 11c for displaying prescribed work instruction information; and a management device 30 which receives device information of a work object imaged by the imaging unit 11b from the terminal device 10 and transmits work instruction information corresponding to the device information to the terminal device 10. The work instruction information includes: work items of which the work contents are indicated by character information; and work item moving images corresponding to the work items. The management device 30 generates a work instruction information group in which a plurality of pieces of work instruction information are arranged in work order, and transmits the first piece of work instruction information in the work instruction information group to the terminal device 10 and, on the basis of an imaging result of the imaging unit 11b which is transmitted from the terminal device 10, repeats processing of transmitting the next piece of work instruction information to the terminal device 10 each time the worker finishes prescribed work based on the work instruction information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a work assistance system and the like. [Background technology]

[0002] As a technology for supporting work such as operation, inspection, and maintenance of machinery and equipment, for example, the one described in Patent Document 1 is known. That is, Patent Document 1 describes a work support system that includes "a display device that displays information visible to a worker, a reception unit that receives information regarding the progress of work from the worker, and a display control unit that displays work procedure information, which is information regarding a work procedure, on the display device according to the information received by the reception unit." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-114111 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, information on work procedures according to the progress of work is displayed on a display device, but there is room for improvement in terms of convenience.

[0005] Therefore, an object of the present disclosure is to provide a highly convenient work assistance system, etc. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the present disclosure comprises a terminal device having an imaging unit that images an area near a worker's field of view and a first display unit that displays specified work instruction information, and a management device that receives equipment information of a work object imaged by the imaging unit from the terminal device and transmits the work instruction information corresponding to the equipment information to the terminal device, wherein the work instruction information includes work items, which are text information indicating work content, and work item videos corresponding to the work items, and the management device generates a work instruction information group in which a plurality of the work instruction information are arranged in order of work, transmits a first work instruction information in the work instruction information group to the terminal device, and repeats the process of transmitting the next work instruction information to the terminal device each time the worker completes a specified task based on the work instruction information based on the imaging result of the imaging unit transmitted from the terminal device. Effect of the Invention

[0007] According to the present disclosure, a highly convenient work assistance system and the like can be provided. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram of a work support system according to a first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram of a terminal device and a management device included in the work support system according to the first embodiment. [Diagram 3] FIG. 2 is an explanatory diagram of a manufacturing device which is an example of a work target in the work support system according to the first embodiment. [Figure 4] 2 is a functional block diagram of a terminal device included in the work support system according to the first embodiment. FIG. [Diagram 5] 2 is a functional block diagram of a main body of a management device included in the work support system according to the first embodiment. FIG. [Figure 6] FIG. 2 is an explanatory diagram relating to a plurality of pieces of work instruction information of the work support system according to the first embodiment. [Figure 7] FIG. 2 is an explanatory diagram relating to a work instruction information group of the work support system according to the first embodiment. [Figure 8A] 5 is a flowchart showing the processing of a terminal device and a management device in the work support system according to the first embodiment. [Figure 8B] 5 is a flowchart showing the processing of a terminal device and a management device in the work support system according to the first embodiment. [Figure 9] 4 is a display example of confirmation instruction information in the work support system according to the first embodiment. [Figure 10] FIG. 11 is a functional block diagram of a main body of a management device included in a work support system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] First Embodiment FIG. 1 is an explanatory diagram of a task support system 100 according to the first embodiment. The work support system 100 is a system for supporting a worker in performing a specific task. The facility at the work site may be a manufacturing plant or a chemical plant, or may be a power generation facility, a substation facility, a power distribution facility, a railway facility, a communication facility, an air conditioning facility, a medical facility, a gas facility, or a water facility, or may be other facilities such as computer facilities such as a server, OA facilities such as an office multifunction printer, food processing facilities, kitchen facilities, and amusement facilities. The type of work performed by the worker may be the operation of a specific device, inspection or maintenance of the device, or other specific tasks.

[0010] 1, the work support system 100 includes a terminal device 10 and a relay device 21 at the work site, and also includes a relay device 22 and a management device 30 in a management room. The terminal device 10 is a device carried by a worker at the work site.

[0011] The relay device 21 is a device that relays communication between the terminal device 10 and the network N1, and is installed at a work site. For example, a router is used as such a relay device 21. As shown in Fig. 1, the relay device 21 is connected to the terminal device 10 by wire or wirelessly. Note that in data communication via the relay device 21, communication standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark) may be used as appropriate. The network N1 shown in Fig. 1 is a wide area communication network such as the Internet. The other relay device 22 is a device that relays communication between the management device 30 and the network N1, and is installed in a management room. As shown in Fig. 1, the relay device 22 is connected to the management device 30 by wire or wirelessly.

[0012] The terminal device 10 has a function of photographing the worker working and generating data of the photographed image. The photographed image data generated by the terminal device 10 is transmitted to the management device 30 via the relay device 21, the network N1, and the relay device 22 in this order. The management device 30 generates predetermined work instruction information based on the photographed image data. The work instruction information generated by the management device 30 is transmitted to the terminal device 10 via the relay device 22, the network N1, and the relay device 21 in this order. In this manner, predetermined data is transmitted from one of the terminal device 10 and the management device 30 to the other via the network N1, etc.

[0013] FIG. 2 is an explanatory diagram of the terminal device 10 and the management device 30 included in the work support system 100. As shown in FIG. As shown in FIG. 2, the terminal device 10 includes smart glasses 11 and a control unit 12. The smart glasses 11 are wearable devices worn by a worker. By using such a wearable device, it is possible to capture images of the worker's work while both hands are free, and to display predetermined work instruction information. The type of wearable device is not limited to the smart glasses 11, and may be a headset or VR (Virtual Reality) goggles. As shown in FIG. 2, the smart glasses 11 include a frame 11a, an imaging unit 11b, a first display unit 11c, and an audio output unit 11d.

[0014] The frame 11a is a frame for allowing the eyeglass-like smart glasses 11 to be worn on both ears of the worker and for holding the imaging unit 11b, the first display unit 11c, and the audio output unit 11d. The imaging unit 11b is an eye-level camera for capturing images of the area in the worker's field of view. In the example of FIG. 2, the imaging unit 11b is installed near the center of the width of the frame 11a (near the area between the eyes of the worker wearing the smart glasses 11). The imaging unit 11b captures images of the area the worker is looking at from moment to moment, and image data representing the captured images are transmitted to the management device 30. During the worker's work, the imaging unit 11b continuously transmits the captured images (video data) to the management device 30 in real time via the network N1 (see FIG. 1) or the like.

[0015] The first display unit 11c is a display for displaying work instruction information and the like received from the management device 30 via the network N1 or the like (see FIG. 1), and is installed on the frame 11a. The first display unit 11c displays predetermined work instruction information so that it is within the field of view (for example, the gaze position) of a worker wearing the smart glasses 11 (wearable device). Note that since the first display unit 11c is transparent or semi-transparent, the worker can see the outside even when the work instruction information is displayed on the first display unit 11c.

[0016] The audio output unit 11d is a speaker that outputs a predetermined sound. That is, the audio output unit 11d outputs the work instruction information generated by the management device 30 as a sound. The audio output unit 11d is installed on the frame 11a so as to be provided near the ears of a worker wearing the smart glasses 11.

[0017] The control unit 12 relays communication between the relay device 21 (see FIG. 1) and the smart glasses 11 and supplies power to the smart glasses 11, and is connected to the smart glasses 11 (wearable device) by wire. Details of the control unit 12 will be described later.

[0018] The management device 30 includes a main body 31 and a second display unit 32. Although not shown in the figure, the main body 31 includes electronic circuits such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and various interfaces as a hardware configuration. A program stored in the ROM is read out and expanded in the RAM, and the CPU executes various processes.

[0019] The second display unit 32 is a display for displaying the imaging results of the imaging unit 11b of the smart glasses 11. Note that the second display unit 32 may be omitted when there is no particular need for a skilled worker to view the image on the second display unit 32 and give predetermined instructions to workers at the work site.

[0020] FIG. 3 is an explanatory diagram of a manufacturing facility M1 that is an example of a work target. In addition, Fig. 3 shows a case where the machine equipment to be worked on is a manufacturing equipment M1 for secondary batteries (not shown). In the example of Fig. 3, a plurality of processing members P1 for performing a predetermined processing on the secondary batteries are installed at a predetermined interval on a belt conveyor B1 and move integrally with the belt conveyor B1. Each of the plurality of processing members P1 is provided with a fitting portion P1a into which a secondary battery in the middle of manufacturing is fitted.

[0021] The worker inspects the manufacturing equipment M1 for defects such as loose bolts and worn bearings, and replaces parts as necessary. When the worker performs the work, he or she wears the smart glasses 11 (see FIG. 2) and carries the control unit 12 (see FIG. 2) in his or her pocket.

[0022] FIG. 4 is a functional block diagram of the terminal device 10. As shown in FIG. 4, the terminal device 10 includes smart glasses 11 (see also FIG. 2) and a control unit 12 (see also FIG. 2). As described above, the smart glasses 11 include the imaging unit 11b, the first display unit 11c, and the audio output unit 11d, as well as a communication unit 11e and a signal conversion unit 11f. The communication unit 11e performs predetermined communication with the communication unit 12a of the control unit 12. The communication unit 11e also transmits image data output from the imaging unit 11b to the communication unit 12a of the control unit 12 via the communication unit 11e.

[0023] The signal conversion unit 11f converts the work instruction information received by the communication unit 11e into a predetermined image signal and outputs this image signal to the first display unit 11c, and also converts the work instruction information into a predetermined audio signal and outputs this audio signal to the audio output unit 11d.

[0024] As shown in Fig. 4, the control unit 12 includes a communication unit 12a, a signal conversion unit 12b, a storage unit 12c, and a power supply unit 12d. The communication unit 12a transmits and receives data to and from a relay device 21 (see Fig. 1), and transmits and receives data to and from the smart glasses 11. The communication unit 12a is wirelessly connected to the communication relay device 21 (see Fig. 1). The signal conversion unit 12b converts data received via the communication unit 12a into a predetermined data, and stores the converted data in the storage unit 12c. The storage unit 12c temporarily stores image information that is the imaging result of the imaging unit 11b, as well as temporarily stores work instruction information received from the management device 30 (see Fig. 1).

[0025] The power supply unit 12d supplies power from a battery (not shown) to the smart glasses 11 (wearable device). When the smart glasses 11 and the control unit 12 are not connected via a power line, power is wirelessly supplied from the battery (not shown) to the smart glasses 11 via the power supply unit 12d.

[0026] FIG. 5 is a functional block diagram of the main body unit 31 of the management device 30. As shown in FIG. 5, the main body 31 of the management device 30 includes a communication unit 311, a signal conversion unit 312, a storage unit 313, and a processing unit 314. The communication unit 311 transmits and receives data to and from the relay device 22 (see FIG. 1). The signal conversion unit 312 converts data received via the communication unit 311 into a predetermined format and outputs the converted data to the processing unit 314. When data generated by the processing unit 314 is transmitted to the terminal device 10 (see FIG. 4), the data is converted into a predetermined format by the signal conversion unit 312, and the converted data is transmitted via the communication unit 311.

[0027] In addition to a predetermined program being stored in advance in the memory unit 313, a device information database 313a, a work instruction information database 313b, and a work specification database 313c are also stored in advance. The device information database 313a is a database in which device information is stored, and is used when identifying the work target of the worker. For example, if the work target is a device such as a control panel, the device is identified based on the serial number, etc. Also, if the work target is a part, the part is identified based on the model number and device identification information of the part.

[0028] However, the method of identifying the work object is not limited to this. For example, a worker may use a three-dimensional measuring sensor (not shown) to three-dimensionally measure the shape of the work object. Three-dimensional measurement data (e.g., point cloud data) generated by the three-dimensional measuring sensor is transmitted to management device 30 via network N1 or the like (see FIG. 1). Management device 30 identifies the work object by comparing the three-dimensional measurement data with design data of the plant. In this case, it is assumed that the design data of the plant is stored in equipment information database 33a. Such three-dimensional measurement is also included in "imaging" the work object.

[0029] 5 is a database used to generate predetermined work instruction information. Note that the work instruction information is information for instructing a worker to perform a predetermined work.

[0030] FIG. 6 is an explanatory diagram relating to a plurality of pieces of work instruction information. The equipment information α, β, γ, δ, ε, and η shown arranged vertically on the left edge of the page in FIG. 6 are each identification information of the equipment (or parts of the equipment) that is the work target. The first work instruction information, second work instruction information, ..., nth work instruction information shown in FIG. 6 are information indicating the content of the work that the worker should perform on a specific work target, and are arranged in the order of the work. That is, the first work instruction information indicates the content of the work that the worker should perform first. The second work instruction information indicates the content of the work that the worker should perform second. Also, the nth work instruction information indicates the content of the work that the worker should perform last.

[0031] In this way, a series of tasks to be performed by a worker on a specific work object is divided into n tasks, and the first to nth work instruction information corresponding to these n tasks are arranged in the order of the tasks. Note that, although the example in Fig. 6 shows a case where n pieces of work instruction information are associated with each of the equipment information α, β, γ, δ, ε, and η, the number of pieces of work instruction information may be different for each work object (i.e., for each equipment information).

[0032] Each piece of work instruction information includes a work item, which is text information indicating the work content, and a work item video corresponding to the work item. For example, the first work instruction information corresponding to the device information α includes the work item α 1a And, Work Item Video α 1b and, 1a is a predetermined character information indicating the work content. For example, when a sentence such as "Please turn on the switch numbered XXX1 on the control panel" is displayed on the first display unit 11c (see FIG. 2) of the smart glasses 11 (see FIG. 2), the work item α 1a The text data is used.

[0033] Work item video α 1b is data used to show workers the correct way to do a task using video, and work item α 1a For example, a skilled worker has previously assigned task item α 1a The data recorded when the above tasks were performed is Work Item Video α 1b In addition, a predetermined animation showing the correct way to perform a task may be used as the task item video α 1b Such a task item video α 1b When a worker performs a predetermined task on a task object identified by the device information α, a task item video α is displayed on the first display unit 11c (see FIG. 2) of the smart glasses 11 (see FIG. 2). 1b The remaining second to nth work instruction information are displayed in the same manner.

[0034] Then, every time a worker finishes a task based on predetermined work instruction information, the management device 30 (see FIG. 2) transmits the next work instruction information to the terminal device 10 (see FIG. 2).

[0035] The work specification database 313c shown in Fig. 5 is used to judge whether or not work is being performed appropriately on a given work object. For example, in addition to information indicating the original work location of the work object, a combination of information (called work action data) characterizing the work content performed by the worker, such as the presence or absence and position of the work object (for example, when replacing a plate material secured with four screws, whether the plate material is secured with four screws after replacement), the correct position of the worker's hands, the correct acceleration of the hands (including the direction in which the hands are moved), and the correct way of moving the tool held by the worker, is previously set as work specification information in association with each work item.

[0036] As shown in FIG. 5, the processing unit 314 includes a device information matching unit 314a, an instruction information generating unit 314b, and a work content matching unit 314c. The device information matching unit 314a extracts device information of the work object based on the imaging result of the terminal device 10. Then, the device information matching unit 314a performs matching between the extracted device information and multiple pieces of device information included in the device information database 313a. If the device information of the work object is included in the device information database 313a, this device information is used for processing by the instruction information generating unit 314b. If the device information of the work object is not included in the device information database 313a, a predetermined error signal is transmitted to the terminal device 10 (see FIG. 2). In this case, the device information is imaged again in the terminal device 10.

[0037] The instruction information generating unit 314b shown in FIG. 5 generates a predetermined work instruction information group based on the result of the matching by the device information matching unit 314a.

[0038] FIG. 7 is an explanatory diagram regarding the work instruction information group. The work instruction information group shown in Fig. 7 corresponds to the multiple pieces of work instruction information in the device information α in Fig. 6. In the example of Fig. 7, the work instruction information group is made up of first to fourth pieces of work instruction information. The first work instruction information includes work item α, which is text information indicating the work content. 1a And, the work item video α to show the correct way to do the work 1bThe same is true for the second to fourth work instruction information. The first to fourth work instruction information are arranged in the order of work to be performed to form a work instruction information group.

[0039] 5 judges whether the work actually performed by the worker at the work site is appropriate for each work item based on the information in the work specification database 313c. Note that AI (Artificial Intelligence) may be used as appropriate to judge whether the work is being performed appropriately by the worker.

[0040] 8A and 8B are flowcharts showing the processing of the terminal device and the management device (also see FIGS. 4 and 5 as appropriate). In step S101, the terminal device 10 captures the device information by the imaging unit 11b. As described above, the device information is identification information for identifying the work target. For example, the device information may be written at a predetermined position on the work target, and the worker wearing the smart glasses 11 may view the device information of the work target and then perform a predetermined operation (such as pressing a predetermined button) that serves as a trigger for transmitting the device information to the management device 30. In step S102, the terminal device 10 transmits image information including the equipment information to the management device 30. That is, the terminal device 10 captures an image of the equipment information of the object to be worked on by the worker, and transmits the equipment information to the management device 30 (equipment information transmission step).

[0041] In step S103, the management device 30 receives the device information from the terminal device 10. Then, the management device 30 extracts the device information from the image information by performing pattern matching or the like. In step S104, the management device 30 generates a work instruction information group by the instruction information generation unit 314b. To explain in more detail, first, the management device 30 identifies the work target based on the image data of the equipment information received from the terminal device 10. Then, the management device 30 generates the work instruction information group by extracting multiple pieces of work instruction information from the work instruction information database 313b. As described above, the work instruction information group is data in which multiple pieces of work instruction information corresponding to equipment information are arranged in the order of work from the first work instruction information to the nth work instruction information.

[0042] Next, in step S105, the management device 30 transmits the first work instruction information to the terminal device 10. In this manner, the management device 30 receives the equipment information of the work object imaged by the imaging unit 11b from the terminal device 10 (S103), and transmits the work instruction information corresponding to the equipment information to the terminal device 10 (S105: work instruction information transmission step). In step S106, the terminal device 10 receives the first work instruction information from the management device 30.

[0043] In step S107, the terminal device 10 displays predetermined work instruction information including the work items and the work item videos on the first display unit 11c (work instruction information display step). For example, when the terminal device 10 receives the first work instruction information corresponding to the equipment information α shown in FIG. 6 from the management device 30, the terminal device 10 displays the work item α, which is text information, on the first display unit 11c. 1a is displayed on the first display unit 11c, and a work item video α showing the correct way to perform the work is displayed. 1b is also displayed on the first display unit 11c. This allows the worker to understand at a glance how to proceed with the work.

[0044] In addition, in the first display section 11c, the work item α 1a And work item video α 1b is preferably displayed at the gaze position of the worker. This allows the worker to 1a And work item video α 1bThe movement of the pupils of the worker's eyes may be identified by the smart glasses 11, and the display position of the work instruction information on the smart glasses 11 may be adjusted according to the movement of the pupils.

[0045] Next, in step S108, the terminal device 10 captures the worker's work using the imaging unit 11b. It is preferable that the work is captured as a video so that the worker's hand movements can be seen. In step S109, the terminal device 10 transmits the imaging results of the work to the management device 30. In step S110, the management device 30 receives the imaging results of the work from the terminal device 10.

[0046] Next, in step S111 of FIG. 8B, the management device 30 uses the work content collation unit 314c to determine whether or not there is an error in the work of the worker. That is, the management device 30 generates work action data that characterizes the work content performed by the worker based on the imaging result of the imaging unit 11b. Then, the management device 30 determines whether or not each piece of work action data is within a predetermined allowable range. For example, if each piece of work action data is within the allowable range, the management device 30 determines that there is no error in the work of the worker. Also, if any piece of work action data is outside the allowable range, the management device 30 determines that there is an error in the work of the worker. Note that forgetting to perform a task by the worker is also considered to be an error in the work. If there is an error in the work in step S111 (S111: No), the process of the management device 30 proceeds to step S112.

[0047] In step S112, management device 30 transmits confirmation instruction information for pointing out the work error to terminal device 10. The confirmation instruction information includes text information and audio information for pointing out the work error, and is stored in advance in storage unit 313 in association with the content of the error. In addition, the confirmation instruction information may include a work item video showing the correct work content. This work item video may be the same as the work item video included in the first work instruction information transmitted in step S105 (see FIG. 8A), or may have a specified portion highlighted to correct the work error.

[0048] In this way, when the management device 30 determines that the worker's work is incorrect with respect to the original work content based on the imaging result of the imaging unit 11b (S111: No), it transmits confirmation instruction information indicating this error to the terminal device 10 (S112). After performing the process of step S112, the management device 30 again receives the imaging result of the worker's work from the terminal device 10 (S110 in FIG. 8A) and again determines whether or not there is an error in the work (S111 in FIG. 8B).

[0049] In step S113 of FIG. 8B, the terminal device 10 receives, from the management device 30, confirmation instruction information for pointing out an error in the work. Next, in step S114, the terminal device 10 displays the confirmation instruction information. That is, when the terminal device 10 receives the confirmation instruction information, it displays an image included in the confirmation instruction information on the first display unit 11c. For example, the terminal device 10 displays text information or video for pointing out an error in the work on the first display unit 11c. Also, when the terminal device 10 receives the confirmation instruction information, the audio included in the confirmation instruction information may be output from the audio output unit 11d. This allows the worker to notice the error in the work and also makes it easier to understand the original correct work content.

[0050] FIG. 9 is an example of a display of confirmation instruction information. The image of the confirmation instruction information shown in FIG. 9 is displayed on the first display unit 11c (see FIG. 2) of the smart glasses 11 (see FIG. 2). If the worker's work is incorrect, for example, the confirmation instruction information shown in FIG. 9 is displayed on the first display unit 11c to correct the mistake. In the example of FIG. 9, in addition to the message "The work content is incorrect. Please perform the work according to the work item video," a work item video (recorded image) of a skilled worker tightening the nut U1 using the torque wrench T1 is displayed. By displaying such confirmation instruction information, the worker can understand that the previous work was incorrect and can redo the work.

[0051] 8B (S111: Yes), the management device 30 proceeds to step S115. That is, if the work action data is within the allowable range, the management device 30 proceeds to step S115. In step S115, the management device 30 determines whether or not all of the first to nth work instruction information have been erased from the work instruction information group (see FIG. 7). In step S115, if work instruction information remains in the work instruction information group (S115: No), the processing of the management device 30 proceeds to step S116.

[0052] In step S116, the management device 30 deletes the first work instruction information from the work instruction information group, and moves it up in the order. Specifically, the management device 30 moves the order of the original second work instruction information up to make it the first work instruction information, and moves the order of the original nth work instruction information up to make it the (n-1)th work instruction information, and so on, moving the order up one by one. In this way, the management device 30 deletes work instruction information that has been used in the current series of work, and moves the order of the work instruction information up one by one. This not only prevents the work instruction information from being sent in the wrong order, but also allows the management device 30 to have some spare memory capacity.

[0053] After performing the process of step S116, the process of the management device 30 returns to step S105 (see FIG. 8A). Then, in step S105, the management device 30 transmits new first work instruction information to the terminal device 10. That is, if the work action data characterizing the work content performed by the worker is within the allowable range, the management device 30 transmits the next work instruction information to the terminal device 10. For example, in the work instruction information group corresponding to the equipment information α in FIG. 6, the work item α 2a And work item video α 2b The new first work instruction information including the above is transmitted from the management device 30 to the terminal device 10.

[0054] In this way, the management device 30 generates a work instruction information group in which a plurality of pieces of work instruction information are arranged in the order of work, transmits the first piece of work instruction information in the work instruction information group to the terminal device 10, and repeats the process of transmitting the next piece of work instruction information to the terminal device 10 every time the worker completes a predetermined task based on the work instruction information based on the imaging result of the imaging unit 11b transmitted from the terminal device 10. Specifically, when the worker correctly performs the task corresponding to the task item of the first work instruction information, the management device 30 deletes the first work instruction information from the work instruction information group, advances the task order number associated with the work instruction information, and repeats the process of transmitting new first work instruction information to the terminal device 10.

[0055] Furthermore, in step S115, if all work instruction information included in the work instruction information group has been erased (S115: Yes), the processing of the management device 30 proceeds to step S117. In step S117, the management device 30 transmits a work end signal to the terminal device 10 and ends the series of processes (END). Note that the work end signal is a signal indicating that a series of tasks related to the work object has been completed. In this way, if the management device 30 has erased all work instruction information included in the work instruction information group (S115: Yes), it transmits a work end signal to the terminal device 10 (S117).

[0056] Meanwhile, in step S118, the terminal device 10 determines whether or not a work end signal has been received from the management device 30. If a work end signal has not been received from the management device 30 (S118: No), the process of the terminal device 10 returns to step S106 (see FIG. 8A). Then, in step S106, the terminal device 10 receives new first work instruction information from the management device 30 after the order has been moved up.

[0057] Furthermore, if a work end signal is received from the management device 30 in step S118 (S118: Yes), the terminal device 10 displays a work end message on the first display unit 11c in step S119. For example, the terminal device 10 displays a message such as "The work of operating the control panel has been completed" on the first display unit 11c. This allows the worker to confirm that a series of works related to the work object has been completed. After performing the process of step S119, the terminal device 10 ends the series of processes (END).

[0058] <Effects> According to the first embodiment, every time a worker finishes a task based on predetermined work instruction information, the next work instruction information is displayed on the first display unit 11c. This allows the worker to proceed with the task while checking the work contents one by one, thereby improving convenience and reliability when performing the task. Furthermore, if the worker makes an error in the task, confirmation instruction information for pointing out the error is displayed on the first display unit 11c. This makes it possible to prevent the worker from making an error in the task even if the worker is a beginner. Furthermore, since there is no particular need for another experienced worker to be present beside the worker to give advice, the effort and cost required for the task can be reduced.

[0059] Second Embodiment The second embodiment differs from the first embodiment in that the management device 30A (see FIG. 10) does not particularly include a work specification database 313c (see FIG. 5) or a work content comparison unit 314c (see FIG. 5). The second embodiment also differs from the first embodiment in that a skilled worker (advisor) in a management room watches the worker's work and, if there is an error in the work, points it out to the worker by voice or the like. The rest of the second embodiment is similar to the first embodiment. Therefore, only the parts that differ from the first embodiment will be described, and explanations of the overlapping parts will be omitted.

[0060] FIG. 10 is a functional block diagram of a main body 31A of a management device 30A included in the work support system according to the second embodiment. As shown in Fig. 10, the management device 30A includes a communication unit 311, a signal conversion unit 312, a storage unit 313A, and a processing unit 314A. The storage unit 313A also stores in advance a device information database 313a and a work instruction information database 313b. The device information database 313a and the work instruction information database 313b are similar to those in the first embodiment (see Fig. 5), and therefore a description thereof will be omitted. The work specification database 313c (see Fig. 5) described in the first embodiment is not particularly stored in the storage unit 313A.

[0061] The processing unit 314A also includes a device information matching unit 314a and an instruction information generating unit 314b. The device information matching unit 314a and the instruction information generating unit 314b are the same as those in the first embodiment (see FIG. 5), and therefore a description thereof will be omitted. The processing unit 314A does not particularly include the task content matching unit 314c (see FIG. 5) described in the first embodiment.

[0062] In the management room, the imaging results of the imaging unit 11b (see FIG. 2) of the smart glasses 11 (see FIG. 2) are displayed in real time as a video on the second display unit 32 (see FIG. 2) of the management device 30A. A skilled worker checks the work content of the worker one by one by looking at the second display unit 32, and if the work content is incorrect, points out the error and teaches the correct work content.

[0063] The process performed by the terminal device 10 is similar to that described in the first embodiment (see Figs. 8A and 8B), and therefore will not be described. The process performed by the management device 30A differs from that in the first embodiment in steps S111 and S112 (see Fig. 8B). That is, in the second embodiment, a skilled worker determines whether or not there is an error in the work performed by the worker, instead of the management device 30A.

[0064] Also, if there is an error in the work of the worker, the experienced worker corrects the error. For example, the experienced worker may output a predetermined advice by voice from the voice output unit 11d of the smart glasses 11 (see FIG. 2). That is, when a voice signal of the experienced worker (advisor) who confirms that there is an error in the work after seeing the work of the worker displayed on the second display unit 32 (see FIG. 2) is transmitted from the management device 30 to the terminal device 10, the voice signal may be output from the voice output unit 11d as a predetermined voice.

[0065] Furthermore, when information is shared between a skilled worker in the control room and a worker at the work site, a communication tool such as Teams (registered trademark) or Zoom (registered trademark) may be used as appropriate. In this case, in order to convey the correct work content to the worker, predetermined image information may be shared by Teams (registered trademark) or Zoom (registered trademark), and this image information may be displayed on the first display unit 11c of the smart glasses 11. This can improve the convenience of the worker at the work site when receiving advice.

[0066] <Effects> According to the second embodiment, an experienced worker checks the work content of the worker in the control room and, if there is an error in the work content, points it out by voice, etc. This allows the worker to receive advice from the experienced worker and, if necessary, to share certain image information.

[0067] <<Variations>> Although the work assistance system 100 and the work assistance method according to the present disclosure have been described in the above embodiments, the present disclosure is not limited to these descriptions and may be modified in various ways. For example, in each embodiment, a wearable device such as smart glasses 11 (see FIG. 2) is used as the terminal device 10 (see FIG. 2), but the present disclosure is not limited to this. In other words, a smartphone or a tablet may be used as the terminal device 10.

[0068] In each embodiment, a case has been described in which an eye-line camera is used as the imaging unit 11b (see FIG. 2) that captures an image of the area in the worker's field of view and the vicinity thereof, but this is not limiting. For example, a camera (imaging unit) that captures an image of the area in the worker's field of view and the vicinity thereof may be attached to the worker's head (helmet, etc.) or to a predetermined part of the worker's body.

[0069] In addition, in each embodiment, the case where the sound is output from the sound output unit 11d (see FIG. 2) of the smart glasses 11 (see FIG. 2) has been described, but the present invention is not limited to this. For example, the worker may wear earphones (not shown) and a predetermined sound may be output from the earphones. Also, the worker may wear a microphone (not shown). If the worker has any questions while performing the work, the worker may speak to that effect into the microphone. In this case, the management device 30 may answer the questions from the inventor, or the skilled worker may answer by voice. When the management device 30 answers, AI (Artificial Intelligence) may be used as appropriate.

[0070] Also, an inspection device such as a sensor probe (not shown) may be included in the terminal device 10, and momentary detection values ​​of this inspection device may be transmitted to the management device 30 via a network N1 or the like (see FIG. 1). In this case, the management device 30 judges whether the work content is appropriate based on the work content of the worker and the detection values ​​of the inspection device. Also, the worker may wear a motion sensor (not shown), and momentary detection values ​​of the motion sensor may be used in addition to the captured image to judge whether the work content is appropriate.

[0071] In the second embodiment, the processing unit 314A of the management device 30A (see FIG. 10) is provided with the device information collating unit 314a, but the device information collating unit 314a may be omitted. In this case, a skilled worker (advisor) in the management room visually checks the imaging result of the device information displayed on the second display unit 32 (see FIG. 2) and inputs the device information.

[0072] In each embodiment, the case where the main body unit 31 and the second display unit 32 are separate in the management device 30 (see FIG. 2) has been described, but this is not limited thereto. For example, the main body unit 31 and the second display unit 32 may be integrated as in a smartphone or tablet. In addition, in each embodiment, the image of the work instruction information is displayed at the gaze position of the worker on the first display unit 11c, but this is not limited to the above. For example, the image of the work instruction information may be displayed at a position away from the gaze position of the worker on the first display unit 11c.

[0073] In each embodiment, when the work of the first work instruction information is completed, the management device 30 deletes the first work instruction information from the work instruction information group and moves the order of the work instruction information up by one, but the following process may be performed. That is, each time the work of a specific work item is completed, the management device 30 may sequentially read out the first work instruction information, the second work instruction information, ..., the n-th work instruction information, and transmit the read out work instruction information to the terminal device 10. In this case, there is no particular need to delete the work instruction information or move it up in order.

[0074] It is also possible to combine the first and second embodiments. That is, the management device 30 judges whether or not there is an error in the work content of the worker, and the experienced worker may also check the work content while looking at the second display unit 32 (see FIG. 2) and give advice to the worker as necessary.

[0075] In addition, the present disclosure is not limited to each embodiment, and various modified examples are included. For example, each embodiment has been described in detail to clearly explain the present disclosure, and is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another example to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0076] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0077] 10 Terminal Equipment 11 Smart glasses (wearable devices) 11a Frame 11b Imaging section 11c 1st display section 11d Audio output section 12 Control section 12a Communications Department 12b Signal conversion section 12c storage section 12d Power supply section 21,22 Relay device 30,30A management device 31,31A Main body 32 2nd display section 100 Work Support System 313,313A storage section 313a Equipment Information Database 313b Work instruction information database 313c Work Specification Database 314,314A Processing section 314a Equipment information verification unit 314b Instruction information generation unit 314c Work Content Verification Department M1 Manufacturing equipment (work object) N1 Network S102 Step (Device Information Transmission Step) S105 step (work instruction information sending step) S107 step (work instruction information display step)

Claims

1. A terminal device having an imaging unit that images an area in the field of view of a worker and a first display unit that displays predetermined work instruction information; a management device that receives equipment information of the work object captured by the imaging unit from the terminal device and transmits the work instruction information corresponding to the equipment information to the terminal device, The work instruction information includes a work item, which is text information indicating a work content, and a work item video corresponding to the work item, The management device generates a work instruction information group in which a plurality of pieces of work instruction information are arranged in order of work, transmits a first piece of work instruction information in the work instruction information group to the terminal device, and repeats the process of transmitting the next piece of work instruction information to the terminal device each time the worker completes a specified task based on the work instruction information based on the imaging result of the imaging unit transmitted from the terminal device.

2. the terminal device includes a sound output unit that outputs a predetermined sound; When the management device determines that the worker's work is incorrect with respect to the original work content based on the imaging result of the imaging unit, the management device transmits confirmation instruction information indicating the error to the terminal device; When the terminal device receives the confirmation instruction information, the terminal device outputs a sound included in the confirmation instruction information from the sound output unit.

2. The work support system according to claim 1,

3. When the management device determines that the worker's work is incorrect with respect to the original work content based on the imaging result of the imaging unit, the management device transmits confirmation instruction information indicating the error to the terminal device; When the terminal device receives the confirmation instruction information, an image included in the confirmation instruction information is displayed on the first display unit.

2. The work support system according to claim 1,

4. the terminal device includes a sound output unit that outputs a predetermined sound; the management device includes a second display unit on which an imaging result of the imaging unit is displayed, When an advisor who has viewed the work of the worker displayed on the second display unit and confirmed that the work is incorrect sends a voice signal from the management device to the terminal device, the voice signal is output from the voice output unit as a predetermined voice.

2. The work support system according to claim 1,

5. The management device includes: generating a group of work instruction information in which the work instruction information is arranged in order of work from a first work instruction information to an n-th work instruction information as the plurality of work instruction information corresponding to the device information; when the work corresponding to the work item of the first work instruction information is correctly performed by the worker, the first work instruction information is deleted from the work instruction information group, and new first work instruction information is transmitted to the terminal device; and When all of the work instruction information included in the work instruction information group is deleted, a work completion signal is transmitted to the terminal device.

2. The work support system according to claim 1,

6. The terminal device includes a wearable device worn by the worker and a control unit connected to the wearable device by a wired connection; the wearable device includes the first display unit that displays the work instruction information so as to be within the field of view of the worker wearing the wearable device, an audio output unit that is provided near the worker's ear, and the imaging unit; The control unit has a power supply unit that supplies power to the wearable device, and a communication unit that wirelessly connects to a communication relay device.

2. The work support system according to claim 1,

7. The management device generates work action data characterizing the work content to be performed by the worker based on the imaging result, and transmits the next work instruction information to the terminal device if the work action data is within an allowable range.

2. The work support system according to claim 1,

8. a device information transmission step in which the terminal device captures device information of the object to be worked on by the worker using an imaging unit and transmits the device information to the management device; a work instruction information transmitting step in which the management device receives the device information from the terminal device and transmits work instruction information corresponding to the device information to the terminal device; a work instruction information display step of causing the terminal device to display the work instruction information on a first display unit, The work instruction information includes a work item, which is text information indicating a work content, and a work item video corresponding to the work item, In the work instruction information sending step, a group of work instruction information is generated in which a plurality of pieces of work instruction information are arranged in order of task, a first piece of work instruction information in the group of work instruction information is sent to the terminal device, and based on the imaging result of the imaging unit sent from the terminal device, each time the worker completes a specified task based on the work instruction information, the process of sending the next piece of work instruction information to the terminal device is repeated.

Images