Work assistance device, work assistance method, and work assistance system

Abstract

A work assistance device (2) is configured to be provided with: a work specifying unit (10) that specifies a work to be assisted; and a character information acquisition unit (13) that acquires character information of a work sound, which is a sound indicating the work specified by the work specification unit (10).

Description

Technical Field

[0001] This disclosure relates to work assistance devices, work assistance methods, and work assistance systems. Background Technology

[0002] There are work assistance devices that assist workers in their work.

[0003] As such an operational assistance device, for example, Patent Document 1 discloses a fork positioning assistance device equipped with a camera and an image display unit.

[0004] The camera captures an image of the area in front of the forklift and outputs image data representing that area to an image display unit. The area captured by the camera includes the pallet being handled by the forklift and the forklift's forks. The image display unit displays the image shown by the image data output from the camera.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2013-86959 Summary of the Invention

[0008] The problem the invention aims to solve

[0009] While the fork positioning aid disclosed in Patent Document 1 displays an image of the area including the pallet and forks on a monitor, it does not detect whether the forks have been correctly positioned relative to the pallet. Therefore, the following problem exists: the operator or the monitor of the operation may sometimes be unable to confirm whether the positioning operation they are performing or the positioning operation being monitored is being performed correctly.

[0010] This disclosure was made to solve the problems mentioned above, and its purpose is to provide a work assistance device that enables the operator or the monitor of the work to confirm whether the work they are performing or the work being monitored is being performed correctly.

[0011] means for solving problems

[0012] The work assistance device disclosed herein includes: a work determination unit that determines a work to be assisted; and a text information acquisition unit that acquires text information of a work sound, the work sound being a sound representing the work determined by the work determination unit.

[0013] Invention Effects

[0014] According to this disclosure, the operator or the monitor of the operation can confirm whether the operation they are performing or the operation being monitored is being performed correctly. Attached Figure Description

[0015] Figure 1 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 1.

[0016] Figure 2 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to Embodiment 1.

[0017] Figure 3 This is a hardware structure diagram of a computer when the work assistance device 2 is implemented through software or firmware.

[0018] Figure 4 This is a flowchart illustrating the work assistance method as the processing steps of the work assistance device 2.

[0019] Figure 5 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 2.

[0020] Figure 6 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to Embodiment 2.

[0021] Figure 7 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 3.

[0022] Figure 8 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 3.

[0023] Figure 9 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 4.

[0024] Figure 10 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 4.

[0025] Figure 11 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 5.

[0026] Figure 12 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 5.

[0027] Figure 13 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 6.

[0028] Figure 14 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 6.

[0029] Figure 15This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 7.

[0030] Figure 16 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 7. Detailed Implementation

[0031] Hereinafter, in order to illustrate this disclosure in more detail, the manner in which this disclosure is carried out will be described with reference to the accompanying drawings.

[0032] Implementation method 1.

[0033] Figure 1 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 1.

[0034] Figure 2 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to Embodiment 1.

[0035] exist Figure 1 In this context, sensor 1 is implemented, for example, through a camera or a lidar device.

[0036] Sensor 1 detects the position of the work object being worked on by the operator, i.e., the work position.

[0037] Specifically, sensor 1 detects, for example, the position of the operator's hand relative to the work object or the position of the operator's foot relative to the work object as the work position.

[0038] Sensor 1 outputs position data indicating the work location to work assistance device 2.

[0039] In the case where sensor 1 is implemented by, for example, a camera, sensor 1 outputs coordinate data representing the position of the operator's hand, etc., as position data to the work assistance device 2 in the image captured by the camera.

[0040] exist Figure 1 In the illustrated job assistance system, sensor 1 detects the position of the operator's hand relative to the work object as the job position. Alternatively, sensor 1 can detect the job position based on job instruction data indicating the next task the operator will perform. This job instruction data can be stored in the internal memory of sensor 1, provided externally to the job assistance system, or provided by the job assistance device 2. For example, the internal memory of sensor 1 may store a correspondence between one or more possible tasks the operator might perform and the corresponding job positions. If such a correspondence exists in the internal memory, sensor 1 refers to the internal memory to determine the job position corresponding to the task indicated by the job instruction data and outputs the position data representing the job position to the job assistance device 2.

[0041] Alternatively, if sensor 1 is implemented by a camera, sensor 1 can also capture the work position corresponding to the work indicated by the work instruction data, and output the coordinate data, such as the position of the worker's hand reflected in the captured image, as position data to the work assistance device 2.

[0042] When the work location is detected or determined based on pre-prepared work instruction data rather than data such as images acquired during the operation, the work location may include a location or area other than the actual work location. Such an area could be, for example, a larger area that includes both the vicinity of the actual work location and the actual work location itself.

[0043] Furthermore, when the work assistance system detects or determines the work location based on pre-prepared work instruction data and other data, sensor 1 can be omitted. The pre-prepared data is static data.

[0044] For example, the work position can be detected based on work instruction data by a position data acquisition unit 11 included in the work assistance device 2, instead of sensor 1. In such a case, the expression "position data output from sensor 1" in the following description is replaced by "position data determined based on work instruction data".

[0045] Furthermore, the work assistance system can also identify the next task based on the sequence of tasks in a pre-prepared work process, rather than the task location. In this case, the work assistance system can detect a task identifier instead of the task location as an indicator for task identification. In this case, terms such as "position data output from sensor 1" and "work location indicated by the position data" in the following description are replaced with "task identifier determined based on work instruction data" and "task indicated by the task identifier." Hereinafter, the means of determining tasks as an assistance object will sometimes be collectively referred to as the "task determination unit."

[0046] The objects of the operation include, for example, products in the process of manufacturing, products in the process of assembling, components installed on products, products in the process of disassembling, or equipment in factories, etc.

[0047] As an operator, there are operations such as manufacturing products, assembling products, disassembling products, inspecting products / equipment, or handling parts.

[0048] The work assistance device 2 includes a position data acquisition unit 11, a sound registration unit 12, and a text information acquisition unit 13, which are examples of the work determination unit 10.

[0049] The work assistance device 2 acquires textual information of the work sound, which is the sound indicating the work when the work is being carried out at the work position shown by the position data output from the sensor 1.

[0050] The work assistance device 2 outputs the text information of the work sound to the display device 3.

[0051] exist Figure 1 In the illustrated work assistance system, the work assistance device 2 acquires the text information of the work audio and outputs it to the display device 3. The work assistance device 2 may also have functions such as work process management or work safety management. To achieve these functions, the work assistance device 2 may store, for example, a work process management table, work instructions, a baseline overview, manuals related to the work, design data of the work object, three-dimensional data representing the shape of the work object, or map data representing the location of the work object.

[0052] Display device 3 is achieved, for example, through a smartphone, tablet, or AR (Augmented Reality) glasses.

[0053] Display device 3 displays text representing the operation sound according to the text information output from operation assistance device 2.

[0054] The task determination unit 10 is equipped with a location data acquisition unit 11.

[0055] The task determination unit 10 determines the tasks that will be used as auxiliary objects.

[0056] Location data acquisition unit 11, for example, by Figure 2 The position data acquisition circuit 31 shown is implemented.

[0057] The position data acquisition unit 11 acquires position data indicating the work position from the sensor 1. This work position can be a position detected by the sensor 1 based on work instruction data, or a position detected independently of the work instruction data. Furthermore, as described above, the position data acquisition unit 11 can also acquire position data indicating the work position based on work instruction data.

[0058] The location data acquisition unit 11 outputs the location data to the text information acquisition unit 13.

[0059] For example, by the Onomatopoeia Registration Department 12 Figure 2 The onomatopoeic registration circuit 32 shown is implemented.

[0060] The onomatopoeia registration unit 12 registers onomatopoeia corresponding to the work sounds generated during operations that can be performed on the work object. The onomatopoeia registration unit 12 is, for example, a database that registers onomatopoeia corresponding to the following work sounds: work sounds corresponding to operations performed in multiple work procedures shown in work instruction data, etc., or work sounds performed at each work position when operations are performed at multiple work positions according to work procedures prepared in advance for the work object.

[0061] exist Figure 1 In the work assistance device 2 shown, the onomatopoeia registration unit 12 registers onomatopoeia corresponding to the work sounds performed in each work process or work position. However, this is only one example; the actual work sounds performed in the above-mentioned work can also be registered in the onomatopoeia registration unit 12. For example, the work sound collected by a microphone when performing a certain work could be considered an actual work sound.

[0062] In the onomatopoeia registration unit 12, textual information of the onomatopoeia corresponding to the onomatopoeia of ...

[0063] Figure 1 The work assistance device 2 shown has a built-in onomatopoeia registration unit 12. However, this is just one example, and the onomatopoeia registration unit 12 can also be provided outside the work assistance device 2.

[0064] The text information acquisition unit 13 includes a voice acquisition unit 13a and a text information output unit 13b.

[0065] The text information acquisition unit 13 acquires location data from the location data acquisition unit 11.

[0066] The text information acquisition unit 13 acquires text information of the operation sound, which is the sound indicating the operation when the operation is carried out at the operation position shown in the position data.

[0067] The text information acquisition unit 13 outputs the text information of the operation sound to the display device 3.

[0068] For example, by the sound acquisition section 13a of the work. Figure 2 The operation sound acquisition circuit 33 shown is implemented.

[0069] The sound acquisition unit 13a acquires position data from the position data acquisition unit 11.

[0070] The operation sound acquisition unit 13a acquires the operation sound when the operator is performing the operation based on the location data.

[0071] Specifically, the work sound acquisition unit 13a acquires the onomatopoeia corresponding to the work position shown in the position data from the onomatopoeia registration unit 12, and uses it as the work sound when the work is performed by the operator.

[0072] The sound acquisition unit 13a outputs the sound data representing the onomatopoeia to the text information output unit 13b.

[0073] exist Figure 1 In the work assistance device 2 shown, the work sound acquisition unit 13a acquires the simulated sound corresponding to the work position shown in the position data from the simulated sound registration unit 12, as the work sound when the work is performed by the operator. However, this is only one example; the work sound acquisition unit 13a may also provide the position data to a learning model (not shown) and acquire the simulated sound corresponding to the work position shown in the position data from the learning model.

[0074] When the learning model is learning, if it is provided with training data representing position data and onomatopoeia corresponding to the job position shown by the position data, it learns the onomatopoeia corresponding to the job position.

[0075] When the learning model is inferring, if position data is provided from the work sound acquisition unit 13a, it outputs sound data representing the onomatopoeia corresponding to the work position shown in the position data to the display device 3.

[0076] exist Figure 1 In the work assistance device 2 shown, the work sound acquisition unit 13a acquires the onomatopoeia corresponding to the work position shown in the position data from the onomatopoeia registration unit 12, and uses it as the work sound when the work is performed by the operator. When the onomatopoeia corresponding to the work position is registered in the onomatopoeia registration unit 12 in place of the onomatopoeia corresponding to the work position, the work sound acquisition unit 13a acquires the text information of the work sound corresponding to the work position shown in the position data from the onomatopoeia registration unit 12, and outputs the text information to the text information output unit 13b.

[0077] Text information output unit 13b, for example, via Figure 2 The text information output circuit 34 shown is used to achieve this.

[0078] The text information output unit 13b obtains sound data representing onomatopoeia from the working sound acquisition unit 13a.

[0079] The text information output unit 13b converts the onomatopoeia shown in the sound data into text information.

[0080] The text information output unit 13b outputs text information to the display device 3.

[0081] exist Figure 1In the work assistance device 2 shown, the text information output unit 13b obtains sound data representing onomatopoeia from the work sound acquisition unit 13a. When the text information of the work sound is output from the work sound acquisition unit 13a, the text information output unit 13b obtains the text information from the work sound acquisition unit 13a and outputs the text information to the display device 3.

[0082] exist Figure 1 In the work assistance device 2 shown, the text information output unit 13b outputs text information to the display device 3. However, this is just one example; the text information output unit 13b can also output the onomatopoeic sound data shown in the text information to a speaker (not shown).

[0083] exist Figure 1 In this context, it is assumed that the position data acquisition unit 11, the onomatopoeia registration unit 12, the work sound acquisition unit 13a, and the text information output unit 13b, which are components of the work assistance device 2, are respectively transmitted through... Figure 2 This is implemented using dedicated hardware as shown. That is, it is assumed that the work assistance device 2 is implemented through a position data acquisition circuit 31, a sound registration circuit 32, a work sound acquisition circuit 33, and a text information output circuit 34.

[0084] Here, the onomatopoeic registration circuit 32 corresponds, for example, to non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), disks, floppy disks, optical disks, compact disks, mini disks, or DVDs (Digital Versatile Discs).

[0085] In addition, the location data acquisition circuit 31, the operation sound acquisition circuit 33, and the text information output circuit 34 are each, for example, equivalent to a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a circuit composed of these.

[0086] The components of the work assistance device 2 are not limited to being implemented through dedicated hardware; the work assistance device 2 can also be implemented through software, firmware, or a combination of software and firmware.

[0087] Software or firmware is stored as a program in the computer's memory. A computer refers to the hardware that executes programs, such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor).

[0088] Figure 3 This is a hardware structure diagram of a computer when the work assistance device 2 is implemented through software or firmware.

[0089] When the work assistance device 2 is implemented by software or firmware, the onomatopoeia registration unit 12 is configured in the computer's memory 51. Programs for causing the computer to execute the processing steps of the position data acquisition unit 11, the work tone acquisition unit 13a, and the text information output unit 13b are stored in the memory 51. Then, the computer's processor 52 executes the program stored in the memory 51.

[0090] In addition, Figure 2 The diagram shows an example where the components of the work assistance device 2 are implemented using dedicated hardware. Figure 3 The example shown is an example of a work assistance device 2 implemented by software or firmware. However, this is just one example; it is also possible that some components of the work assistance device 2 are implemented by dedicated hardware, while the remaining components are implemented by software or firmware.

[0091] Next, for Figure 1 The operation of the work assistance system shown is explained.

[0092] Figure 4 This is a flowchart illustrating the work assistance method as the processing steps of the work assistance device 2.

[0093] Sensor 1 detects the position of the work object being worked on by the operator, i.e., the work position.

[0094] Specifically, sensor 1 detects, for example, the position of the operator's hand relative to the work object as the work position.

[0095] As mentioned above, sensor 1 can also detect the work location based on work instruction data.

[0096] The position data acquisition unit 11 of the work assistance device 2 acquires position data indicating the work position from the sensor 1. Figure 4 Step ST1). The location data acquisition unit 11 may also acquire location data representing the work location based on the work instruction data, as described above. Alternatively, the location data acquisition unit 11 may also acquire a work identifier that identifies the work, as described above, based on the work instruction data.

[0097] The location data acquisition unit 11 outputs either the location data or the job identifier to the job sound acquisition unit 13a.

[0098] The onomatopoeia registration unit 12 registers onomatopoeia corresponding to the work sounds generated during work that can be performed on the work object. The onomatopoeia registration unit 12 registers onomatopoeia corresponding to the following work sounds: work sounds for work performed separately in multiple work procedures shown in work instruction data, etc., or work sounds for work performed at each work position when work is performed at multiple work positions according to work procedures prepared in advance for the work object.

[0099] As for the work on the object, for example, if there are 3 work (1) to (3), work (1) is, for example, tightening a screw at position A of the object using an electric screwdriver, work (2) is, for example, embedding a connector at position B of the object, and work (3) is, for example, drilling a hole at position C of the object using an electric drill.

[0100] In this case, as an onomatopoeia corresponding to the onomatopoeia of the work performed in each of the work (1) to (3), the onomatopoeia as shown below is registered in the onomatopoeia registration section 12.

[0101] Homework (1) → Onomatopoeia for "da da"

[0102] Homework (2) → Onomatopoeia for "click"

[0103] Homework (3) → Onomatopoeia for "Zzzzz"

[0104] The operation sound acquisition unit 13a acquires either the position data or the operation identifier from the position data acquisition unit 11.

[0105] The work sound acquisition unit 13a acquires from the sound registration unit 12 the sound corresponding to the work performed at the work position shown in the position data or the work indicated by the work identifier, as the work sound when the work is performed by the operator. Figure 4 Step ST2).

[0106] If the job performed at the job location shown in the location data or the job identified by the job identifier is, for example, job (1), then the job sound acquisition unit 13a acquires “da da” as an onomatopoeia.

[0107] If the job performed at the job location shown in the location data or the job identified by the job identifier is, for example, job (2), then the job sound acquisition unit 13a acquires "click" as an onomatopoeia.

[0108] If the job performed at the job location shown in the location data or the job identified by the job identifier is, for example, job (3), then the job sound acquisition unit 13a acquires “zzzz” as an onomatopoeia.

[0109] The sound acquisition unit 13a outputs the acquired onomatopoeic sound data to the text information output unit 13b. Furthermore, the method for determining the task is merely one example and is not limited to the method described above; other known methods may also be used.

[0110] The text information output unit 13b obtains sound data representing onomatopoeia from the working sound acquisition unit 13a.

[0111] The text information output unit 13b converts the onomatopoeia shown in the sound data into text information. Figure 4 Step ST3). The process of converting onomatopoeia into text information is a well-known technique, so detailed explanation is omitted.

[0112] The text information output unit 13b outputs text information to the display device 3.

[0113] Therefore, text information is displayed on display device 3.

[0114] If the work performed at the work location indicated by the location data, or the work indicated by the work identifier, is, for example, work (1), then text information indicating "click-click" is displayed on the display device 3. The text information on the display device 3 can also be displayed using, for example, onomatopoeic representations of comics or dialogue boxes from comics, in an AR format. The display location can be, for example, the work location indicated by the location data, or a location predetermined as the location for displaying the sound of the work performed at the work location indicated by the location data, or the work indicated by the work identifier. Alternatively, a location near the work location that does not obstruct the operator's work can also be identified and displayed.

[0115] If, during actual work, a sound similar to the displayed onomatopoeic "da-da" is produced, the worker can determine that their work has been performed correctly. If, during actual work, a sound unlike "da-da" is produced, the worker can determine that their work has not been performed correctly.

[0116] In the above-described embodiment 1, the work assistance device 2 is configured to include: a work determination unit 10, which determines the work to be assisted; and a text information acquisition unit 13, which acquires text information of the work sound, which is the sound indicating the work determined by the work determination unit 10. Therefore, in the work assistance device 2, the worker or the monitor who is monitoring the work can confirm whether the work they are performing or the work being monitored is being performed correctly.

[0117] Implementation method 2.

[0118] The work assistance device 2 equipped with a sound data acquisition unit 15 will be described. The sound data acquisition unit 15 acquires sound data representing the work sound from a microphone 4 that collects the work sound of the worker.

[0119] Figure 5 This is a structural diagram showing a work assistance system including the work assistance device 2 of Embodiment 2. Figure 5 In, with Figure 1 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0120] Figure 6 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to Embodiment 2. Figure 6 In, with Figure 2 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0121] Microphone 4 collects the sounds of the operator performing the work.

[0122] Microphone 4 outputs sound data representing the work sound to work assistance device 2.

[0123] Figure 5 The work assistance device 2 shown includes a position data acquisition unit 11, a sound data acquisition unit 15, a sound recording unit 16, and a text information acquisition unit 17.

[0124] The sound data acquisition unit 15, for example, is composed of... Figure 6 The sound data acquisition circuit 35 shown is implemented.

[0125] The sound data acquisition unit 15 acquires sound data representing the working sound from the microphone 4.

[0126] The sound data acquisition unit 15 outputs the sound data representing the operation sound to the text information acquisition unit 17.

[0127] For example, by the Onomatopoeia Registration Department 16 Figure 6 The onomatopoeic registration circuit 36 ​​shown is implemented.

[0128] The onomatopoeia registration department 16 is a database that registers onomatopoeia corresponding to the onomatopoeia used when working at various work locations for the work object.

[0129] In the onomatopoeia registration section 16, onomatopoeia corresponding to the work sounds are registered as onomatopoeia for work sounds performed normally and onomatopoeia for work sounds not performed normally.

[0130] exist Figure 5 In the work assistance device 2 shown, the onomatopoeia registration unit 16 is registered with onomatopoeia corresponding to the work sounds generated during the work that can be performed on the work object. However, this is only one example; the onomatopoeia registration unit 16 may also register text information of work sounds when the work is performed normally and text information of work sounds when the work is not performed normally.

[0131] Figure 5 The work assistance device 2 shown has a built-in onomatopoeia registration unit 16. However, this is just one example, and the onomatopoeia registration unit 16 can also be provided outside the work assistance device 2.

[0132] The text information acquisition unit 17 includes a voice acquisition unit 17a and a text information output unit 13b.

[0133] The text information acquisition unit 17 acquires either the location data or the job identifier from the location data acquisition unit 11, and acquires the sound data from the sound data acquisition unit 15.

[0134] The text information acquisition unit 17 may also acquire, for example, the onomatopoeia corresponding to the work sound shown in the sound data from the onomatopoeia registration unit 16, as the onomatopoeia corresponding to the work sound of the work performed at the work position shown in the position data or the work shown in the work identifier.

[0135] The text information acquisition unit 17 converts the onomatopoeia into text information and outputs the text information to the display device 3.

[0136] For example, by the sound acquisition section 17a of the work. Figure 6 The operation sound acquisition circuit 37 shown is implemented.

[0137] The operation sound acquisition unit 17a acquires either position data or operation identifier from the position data acquisition unit 11, and acquires sound data from the sound data acquisition unit 15.

[0138] The work sound acquisition unit 17a acquires the onomatopoeia corresponding to the work sound shown in the sound data from the onomatopoeia registration unit 16, and uses it as the onomatopoeia corresponding to the work sound of the work performed at the work position shown in the position data or the work shown in the work identifier.

[0139] The sound acquisition unit 17a outputs the acquired onomatopoeic sound data to the text information output unit 13b.

[0140] exist Figure 5 In this context, it is assumed that the position data acquisition unit 11, sound data acquisition unit 15, onomatopoeia registration unit 16, work sound acquisition unit 17a, and text information output unit 13b, which are components of the work assistance device 2, are respectively transmitted through... Figure 6 This is implemented using dedicated hardware as shown. That is, it is assumed that the work assistance device 2 is implemented through a position data acquisition circuit 31, a sound data acquisition circuit 35, a sound recording circuit 36, a work sound acquisition circuit 37, and a text information output circuit 34.

[0141] Here, the onomatopoeic registration circuit 36 ​​corresponds, for example, to non-volatile or volatile semiconductor memories such as RAM, ROM, flash memory, EPROM, EEPROM, disk, floppy disk, optical disk, compact disk, mini disk, or DVD.

[0142] Furthermore, the location data acquisition circuit 31, the sound data acquisition circuit 35, the operation sound acquisition circuit 37, and the text information output circuit 34 each correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0143] The components of the work assistance device 2 are not limited to being implemented through dedicated hardware; the work assistance device 2 can also be implemented through software, firmware, or a combination of software and firmware.

[0144] When the operation assistance device 2 is implemented by software or firmware, the onomatopoeia registration unit 16 is configured to... Figure 3 The program for causing the computer to execute the processing steps of the location data acquisition unit 11, the sound data acquisition unit 15, the operation sound acquisition unit 17a, and the text information output unit 13b is stored in the memory 51 shown. Then, Figure 3 The processor 52 shown executes the program stored in the memory 51.

[0145] In addition, Figure 6 The diagram shows an example where the components of the work assistance device 2 are implemented using dedicated hardware. Figure 3 The example shown is an example of a work assistance device 2 implemented by software or firmware. However, this is just one example; it is also possible that some components of the work assistance device 2 are implemented by dedicated hardware, while the remaining components are implemented by software or firmware.

[0146] Next, for Figure 5The operation of the work assistance system shown will be explained. However, apart from microphone 4, sound data acquisition unit 15, onomatopoeia registration unit 16, and work sound acquisition unit 17a, the operation is similar to... Figure 1 The operation assistance system shown is the same. Therefore, here, only the operation of microphone 4, sound data acquisition unit 15, sound registration unit 16 and operation sound acquisition unit 17a will be described.

[0147] Microphone 4 collects the work sounds during the operation and outputs the sound data representing the work sounds to the work assistance device 2.

[0148] The sound data acquisition unit 15 of the work assistance device 2 acquires sound data representing the work sound from the microphone 4.

[0149] The sound data acquisition unit 15 outputs the sound data representing the operation sound to the operation sound acquisition unit 17a.

[0150] The onomatopoeia corresponding to the working sound are registered in the onomatopoeia registration section 16.

[0151] In the onomatopoeia registration section 16, onomatopoeia corresponding to the work sounds are registered as onomatopoeia for work sounds performed normally and onomatopoeia for work sounds not performed normally.

[0152] As an operation targeting an object, for example, if there are two operations (1) to (2), operation (1) is, for example, tightening a screw at position A of the object using an electric screwdriver, and operation (2) is, for example, embedding a connector at position B of the object.

[0153] In this case, the onomatopoeia corresponding to the onomatopoeia of each of the tasks (1) to (2) is registered in the onomatopoeia registration section 16 as shown below.

[0154] Homework (1)

[0155] →The onomatopoeic sound "da da" during normal operation.

[0156] The onomatopoeic sound "squeak" when work is not being performed normally.

[0157] Homework (2)

[0158] →The onomatopoeic word "click" during normal operation.

[0159] The onomatopoeic word "puff" when the operation is not performed normally.

[0160] The operation sound acquisition unit 17a acquires either position data or operation identifier from the position data acquisition unit 11, and acquires sound data from the sound data acquisition unit 15.

[0161] The work sound acquisition unit 17a acquires from the sound registration unit 16 the sound corresponding to the work performed at the work position shown in the position data or the work shown in the work identifier, and the sound corresponding to the work sound shown in the sound data, as the work sound when the work is performed by the worker.

[0162] Specifically, if the determined task is, for example, task (1), the task sound acquisition unit 17a compares the task sound shown in the sound data with the onomatopoeic sounds "da da" and "zhi ga".

[0163] If the working sound shown in the sound data is similar to the onomatopoeic sound "da da", the working sound acquisition unit 17a determines that the work is being performed normally. If the working sound shown in the sound data is similar to the onomatopoeic sound "zhi ga", the working sound acquisition unit 17a determines that the work is not being performed normally.

[0164] Therefore, if the determined task is, for example, task (1) and the task is performed normally, the task sound acquisition unit 17a acquires "da da" as an onomatopoeia from the onomatopoeia registration unit 16.

[0165] If the identified task is, for example, task (1), and the task is not performed normally, the task sound acquisition unit 17a acquires "squeak" as an onomatopoeia from the onomatopoeia registration unit 16.

[0166] If the determined task is, for example, task (2), the task sound acquisition unit 17a compares the task sound shown in the sound data with the onomatopoeic "click" and "puff".

[0167] If the working sound shown in the sound data is similar to the onomatopoeic sound "click", the working sound acquisition unit 17a determines that the operation is being performed normally. If the working sound shown in the sound data is similar to the onomatopoeic sound "plop", the working sound acquisition unit 17a determines that the operation is not being performed normally.

[0168] Therefore, if the identified task is, for example, task (2) and the task is performed normally, the task sound acquisition unit 17a acquires "click" as an onomatopoeia from the onomatopoeia registration unit 16.

[0169] If the identified task is, for example, task (2), and the task is not performed normally, the task sound acquisition unit 17a acquires “puff” as an onomatopoeia from the onomatopoeia registration unit 16.

[0170] The sound acquisition unit 17a outputs the acquired onomatopoeic sound data to the text information output unit 13b.

[0171] The text information output unit 13b obtains sound data representing onomatopoeia from the working sound acquisition unit 17a.

[0172] The text information output unit 13b converts the onomatopoeia shown in the sound data into text information and outputs the text information to the display device 3.

[0173] In cases where work is not being performed normally, the text information output unit 13b can also convert the text information to text information with a different form than the text information used in normal work situations in order to easily distinguish it from normal work situations. For example, text information with a different form may have a different color, a different font, or a different size than the text information used in normal work situations.

[0174] The text information output unit 13b can also display text information for the display device 3 in a comic style. For example, it can also use comic-style onomatopoeia or comic-style dialogue boxes to display text information in AR.

[0175] In Embodiment 2 described above, the onomatopoeia registration unit 16 registers onomatopoeia corresponding to the work sounds when the work is performed normally, and onomatopoeia corresponding to the work sounds when the work is not performed normally. The work assistance device 2 is configured to include a sound data acquisition unit 15, which acquires sound data representing the work sounds from a microphone 4 that collects work sounds during work. Furthermore, the text information acquisition unit 17 of the work assistance device 2 is configured to acquire onomatopoeia corresponding to the work sounds shown in the sound data acquired by the sound data acquisition unit 15 from the onomatopoeia registration unit 16, as onomatopoeia corresponding to the work sounds of a predetermined work. Therefore, in the work assistance device 2, the worker or the monitor of the work can confirm whether the work they are performing or the work being monitored is being performed correctly.

[0176] exist Figure 5 In the work assistance system shown, the onomatopoeia registration unit 16 registers onomatopoeia corresponding to the work sounds when the work is not performed normally. It can also register onomatopoeia that are completely different from the onomatopoeia corresponding to the work sounds when the work is performed normally, as onomatopoeia corresponding to the work sounds when the work is not performed normally. Examples of completely different onomatopoeia include the onomatopoeia for a tool falling ("clang"), the onomatopoeia for a part breaking ("click"), or the onomatopoeia for a trolley approaching the work position ("clunk clunk clunk").

[0177] Implementation method 3.

[0178] In Embodiment 3, a work assistance device 2 equipped with a sound data extraction unit 18 is described. If the sound data obtained by the sound data acquisition unit 15 includes sound data representing work sounds and sound data representing ambient sounds, the sound data extraction unit 18 extracts the sound data representing work sounds from the sound data obtained by the sound data acquisition unit 15.

[0179] Figure 7 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 3. Figure 7 In, with Figure 1 and Figure 5 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0180] Figure 8 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 3. Figure 8 In, with Figure 2 and Figure 6 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0181] Figure 7 The work assistance device 2 shown includes a position data acquisition unit 11, a sound data acquisition unit 15, a sound recording unit 16, a sound data extraction unit 18, an ambient sound determination unit 19, and a text information acquisition unit 20.

[0182] The sound data extraction unit 18, for example, through... Figure 8 The sound data extraction circuit 38 shown is implemented.

[0183] The sound data extraction unit 18 obtains sound data representing the working sound during the operation from the sound data acquisition unit 15.

[0184] If the sound data includes sound data representing the work sound and sound data representing the ambient sound, the sound data extraction unit 18 extracts the sound data representing the work sound from the sound data acquired by the sound data acquisition unit 15 and outputs the sound data representing the work sound to the text information acquisition unit 20.

[0185] Furthermore, if the system contains sound data representing operational sounds and sound data representing ambient sounds, the sound data extraction unit 18 extracts the sound data representing ambient sounds from the sound data acquired by the sound data acquisition unit 15 and outputs the sound data representing ambient sounds to the ambient sound determination unit 19.

[0186] Ambient sound determination unit 19, for example, by Figure 8 The ambient sound determination circuit 39 shown is implemented.

[0187] The ambient sound determination unit 19 obtains sound data representing the ambient sound from the sound data extraction unit 18.

[0188] The ambient sound determination unit 19 determines the simulated sound corresponding to the ambient sound shown in the sound data.

[0189] The ambient sound determination unit 19 outputs sound data representing the onomatopoeia corresponding to the ambient sound to the text information acquisition unit 20.

[0190] The text information acquisition unit 20 includes a voice acquisition unit 20a and a text information output unit 20b.

[0191] The text information acquisition unit 20 acquires either the location data or the job identifier from the location data acquisition unit 11, acquires the sound data representing the job sound from the sound data extraction unit 18, and acquires the sound data representing the onomatopoeia corresponding to the ambient sound from the ambient sound determination unit 19.

[0192] The text information acquisition unit 20 acquires the onomatopoeia corresponding to the working sound shown in the sound data extracted by the sound data extraction unit 18 from the onomatopoeia registration unit 16, and uses it as the onomatopoeia corresponding to the working sound of the predetermined work.

[0193] The text information acquisition unit 20 converts the onomatopoeia corresponding to the work sound into text information, and converts the onomatopoeia corresponding to the ambient sound into text information.

[0194] The text information acquisition unit 20 outputs each piece of text information to the display device 3.

[0195] For example, through the sound acquisition section 20a of the work. Figure 8 The operation sound acquisition circuit 40 shown is used to achieve this.

[0196] The operation sound acquisition unit 20a acquires either position data or operation identifier from the position data acquisition unit 11, and acquires sound data representing the operation sound from the sound data extraction unit 18.

[0197] The work sound acquisition unit 20a acquires the onomatopoeia corresponding to the work sound shown in the sound data extracted by the sound data extraction unit 18 from the onomatopoeia registration unit 16, and uses it as the onomatopoeia corresponding to the work sound of the work.

[0198] The sound acquisition unit 20a outputs the sound data representing the onomatopoeia corresponding to the sound to the text information output unit 20b.

[0199] Text information output unit 20b, for example, via Figure 8 The text information output circuit 41 shown is used to achieve this.

[0200] The text information output unit 20b obtains sound data representing onomatopoeia from the working sound acquisition unit 20a, and obtains sound data representing onomatopoeia corresponding to the ambient sound from the ambient sound determination unit 19.

[0201] The text information output unit 20b converts the onomatopoeia corresponding to the work sound into text information, and converts the onomatopoeia corresponding to the ambient sound into text information.

[0202] The text information output unit 20b outputs various text information to the display device 3.

[0203] exist Figure 7 In this context, it is assumed that the position data acquisition unit 11, sound data acquisition unit 15, onomatopoeia registration unit 16, sound data extraction unit 18, ambient sound determination unit 19, work sound acquisition unit 20a, and text information output unit 20b, which are components of the work assistance device 2, are respectively transmitted through... Figure 8 This is implemented using dedicated hardware as shown. That is, it is assumed that the work assistance device 2 is implemented through a position data acquisition circuit 31, a sound data acquisition circuit 35, a sound registration circuit 36, a sound data extraction circuit 38, an ambient sound determination circuit 39, a work sound acquisition circuit 40, and a text information output circuit 41.

[0204] The location data acquisition circuit 31, the sound data acquisition circuit 35, the sound data extraction circuit 38, the ambient sound determination circuit 39, the operation sound acquisition circuit 40, and the text information output circuit 41 each correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0205] The components of the work assistance device 2 are not limited to being implemented through dedicated hardware; the work assistance device 2 can also be implemented through software, firmware, or a combination of software and firmware.

[0206] When the operation assistance device 2 is implemented by software or firmware, the onomatopoeia registration unit 16 is configured to... Figure 3 The program for executing the processing steps of the location data acquisition unit 11, the sound data acquisition unit 15, the sound data extraction unit 18, the ambient sound determination unit 19, the operation sound acquisition unit 20a, and the text information output unit 20b is stored in the memory 51 shown. Then, Figure 3 The processor 52 shown executes the program stored in the memory 51.

[0207] In addition, Figure 8 The diagram shows an example where the components of the work assistance device 2 are implemented using dedicated hardware. Figure 3 The example shown is an example of a work assistance device 2 implemented by software or firmware. However, this is just one example; it is also possible that some components of the work assistance device 2 are implemented by dedicated hardware, while the remaining components are implemented by software or firmware.

[0208] Next, for Figure 7The operation of the work assistance system shown will be explained. However, apart from the sound data extraction unit 18, the ambient sound determination unit 19, and the text information acquisition unit 20, the operation of the system will be explained. Figure 5 The operation assistance system shown is the same. Therefore, the operation of the sound data extraction unit 18, the ambient sound determination unit 19, and the text information acquisition unit 20 will be mainly described here.

[0209] Microphone 4 collects the sounds of the operation during the process.

[0210] Additionally, if ambient noise is generated during the operation, microphone 4 will collect the ambient noise along with the operation noise. Ambient noise includes, for example, the sound of air conditioning equipment in the room where the operation is being performed, or operation noise accompanying other operations.

[0211] Microphone 4 outputs sound data, representing the sounds including work sounds and ambient sounds, to work assistance device 2.

[0212] The sound data acquisition unit 15 of the work assistance device 2 acquires sound data from the microphone 4 and outputs the sound data to the sound data extraction unit 18.

[0213] The sound data extraction unit 18 acquires sound data from the sound data acquisition unit 15.

[0214] If the acquired sound data includes sound data representing the work sound and sound data representing the ambient sound, the sound data extraction unit 18 extracts the sound data representing the work sound from the acquired sound data.

[0215] In addition, the sound data extraction unit 18 extracts sound data representing ambient sound from the acquired sound data.

[0216] The processing of extracting sound data representing the task sound and the processing of extracting sound data representing the ambient sound are well-known techniques, so detailed descriptions are omitted. However, for example, it is possible to extract sound data representing the task sound or sound data representing the ambient sound based on the difference between the direction of the task sound relative to the microphone 4 and the direction of the ambient sound relative to the microphone 4.

[0217] The sound data extraction unit 18 outputs sound data representing the operation sound to the operation sound acquisition unit 20a, and outputs sound data representing the ambient sound to the ambient sound determination unit 19.

[0218] The operation sound acquisition unit 20a acquires either position data or operation identifier from the position data acquisition unit 11, and acquires sound data representing the operation sound from the sound data extraction unit 18.

[0219] Work Sound Acquisition Unit 20a and Figure 5Similarly, the work sound acquisition unit 17a obtained from the sound registration unit 16 acquires the sound corresponding to the work performed at the work position shown in the position data or the work shown in the work identifier and the work sound shown in the sound data, and uses it as the sound corresponding to the work sound of the work.

[0220] The sound acquisition unit 20a outputs the sound data representing the onomatopoeia corresponding to the sound to the text information output unit 20b.

[0221] The ambient sound determination unit 19 obtains sound data representing the ambient sound from the sound data extraction unit 18.

[0222] The ambient sound determination unit 19 determines the simulated sound corresponding to the ambient sound shown in the sound data.

[0223] For example, if the ambient sound determination unit 19 stores various ambient sounds and corresponding analogues in its internal memory, the ambient sound determination unit 19 determines the ambient sound corresponding to the ambient sound shown by the sound data by comparing it with the various ambient sounds stored in the internal memory. Then, the ambient sound determination unit 19 retrieves the analogue corresponding to the determined ambient sound from the internal memory.

[0224] The ambient sound determination unit 19 outputs sound data representing the onomatopoeia corresponding to the ambient sound to the text information output unit 20b.

[0225] The text information output unit 20b obtains sound data representing onomatopoeia from at least one of the operation sound acquisition unit 20a and the ambient sound determination unit 19.

[0226] The text information output unit 20b converts the onomatopoeia shown in the acquired sound data into text information. The process of converting onomatopoeia into text information is a known technique, so a detailed explanation is omitted.

[0227] The text information output unit 20b outputs text information to the display device 3. The text information is then displayed on the display device 3.

[0228] In Embodiment 3 described above, the onomatopoeia registration unit 16 registers onomatopoeia corresponding to work sounds when work is performed normally and onomatopoeia corresponding to work sounds when work is not performed normally. The work assistance device 2 is configured to include: a sound data acquisition unit 15, which acquires sound data representing work sounds from a microphone 4 that collects work sounds during work; and a sound data extraction unit 18, which extracts sound data representing work sounds from the sound data acquired by the sound data acquisition unit 15 if the sound data acquired by the sound data acquisition unit 15 includes sound data representing work sounds during work and sound data representing ambient sounds. Furthermore, the text information acquisition unit 20 of the work assistance device 2 is configured to acquire onomatopoeia corresponding to the work sounds shown in the sound data extracted by the sound data extraction unit 18 from the onomatopoeia registration unit 16, as onomatopoeia corresponding to work sounds when work is performed at the work position indicated by the position data. Therefore, in the work assistance device 2, even when ambient sounds are generated in addition to work sounds, the worker or the monitor monitoring the work can confirm whether their work or the work being monitored is being performed correctly.

[0229] Implementation method 4.

[0230] In Embodiment 4, a work assistance device equipped with a text information acquisition unit 21 will be described. If the onomatopoeia corresponding to the work sound shown by the sound data acquired by the sound data acquisition unit 15 is not registered in the onomatopoeia registration unit 16, the text information acquisition unit 21 will determine the onomatopoeia corresponding to the sound data acquired by the sound data acquisition unit 15 as the work sound of the work performed at the work position shown by the position data or the work sound of the work indicated by the work identifier.

[0231] Figure 9 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 4. Figure 9 In, with Figure 1 , Figure 5 and Figure 7 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0232] Figure 10 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 4. Figure 10 In, with Figure 2 , Figure 6 and Figure 8 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0233] Figure 9 The work assistance device 2 shown includes a position data acquisition unit 11, a sound data acquisition unit 15, a sound recording unit 16, and a text information acquisition unit 21.

[0234] The text information acquisition unit 21 includes a voice acquisition unit 21a and a text information output unit 13b.

[0235] The text information acquisition unit 21 acquires either location data or job identifier from the location data acquisition unit 11, and acquires sound data from the sound data acquisition unit 15.

[0236] The text information acquisition unit 21 acquires the onomatopoeia corresponding to the work sound shown in the sound data from the onomatopoeia registration unit 16, and uses it as the onomatopoeia corresponding to the work sound of the work performed at the work position shown in the position data or the work shown in the work identifier.

[0237] If the onomatopoeia corresponding to the work sound shown in the sound data is not registered in the onomatopoeia registration unit 16, the text information acquisition unit 21 will determine the onomatopoeia corresponding to the sound data obtained by the sound data acquisition unit 15 as the onomatopoeia corresponding to the work sound of the work performed at the work position shown in the position data or the work indicated by the work identifier.

[0238] For example, the sound acquisition unit 21a of the operation is obtained by Figure 9 The operation sound acquisition circuit 42 shown is implemented.

[0239] The operation sound acquisition unit 21a acquires either position data or operation identifier from the position data acquisition unit 11, and acquires sound data from the sound data acquisition unit 15.

[0240] The work sound acquisition unit 21a acquires the onomatopoeia corresponding to the work sound shown in the sound data from the onomatopoeia registration unit 16, and uses it as the onomatopoeia corresponding to the work sound of the work performed at the work position shown in the position data or the work shown in the work identifier.

[0241] If the onomatopoeia corresponding to the work sound shown in the sound data is not registered in the onomatopoeia registration unit 16, the work sound acquisition unit 21a will determine the onomatopoeia corresponding to the sound data acquired by the sound data acquisition unit 15 as the work sound of the work performed at the work position shown in the position data or the work sound shown in the work identifier.

[0242] The sound acquisition unit 21a outputs sound data representing the onomatopoeia acquired from the onomatopoeia registration unit 16, or sound data representing the determined onomatopoeia, to the text information output unit 13b.

[0243] exist Figure 9 In this context, it is assumed that the position data acquisition unit 11, sound data acquisition unit 15, onomatopoeia registration unit 16, work sound acquisition unit 21a, and text information output unit 13b, which are components of the work assistance device 2, are respectively transmitted through... Figure 10This is implemented using dedicated hardware as shown. That is, it is assumed that the work assistance device 2 is implemented by a position data acquisition circuit 31, a sound data acquisition circuit 35, a sound recording circuit 36, a work sound acquisition circuit 42, and a text information output circuit 34.

[0244] The location data acquisition circuit 31, the sound data acquisition circuit 35, the operation sound acquisition circuit 42, and the text information output circuit 34 each correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0245] The components of the work assistance device 2 are not limited to being implemented through dedicated hardware; the work assistance device 2 can also be implemented through software, firmware, or a combination of software and firmware.

[0246] When the operation assistance device 2 is implemented by software or firmware, the onomatopoeia registration unit 16 is configured to... Figure 3 The program for causing the computer to execute the processing steps of the location data acquisition unit 11, the sound data acquisition unit 15, the operation sound acquisition unit 21a, and the text information output unit 13b is stored in the memory 51. Then, Figure 3 The processor 52 shown executes the program stored in the memory 51.

[0247] In addition, Figure 10 The diagram shows an example where the components of the work assistance device 2 are implemented using dedicated hardware. Figure 3 The example shown is an example of a work assistance device 2 implemented by software or firmware. However, this is just one example; it is also possible that some components of the work assistance device 2 are implemented by dedicated hardware, while the remaining components are implemented by software or firmware.

[0248] Next, for Figure 9 The operation of the work assistance system shown will be explained. However, apart from the work sound acquisition unit 21a, the operation of the system is similar to that of the work assistance system shown. Figure 5 The operation assistance system shown is the same. Therefore, only the operation of the operation sound acquisition unit 21a will be described here.

[0249] The operation sound acquisition unit 21a acquires either position data or operation identifier from the position data acquisition unit 11, and acquires sound data from the sound data acquisition unit 15.

[0250] The sound acquisition unit 21a and Figure 5 Similarly, the work sound acquisition unit 17a obtained from the onomatopoeia registration unit 16 acquires the onomatopoeia corresponding to the work sound of the work performed at the work position shown in the position data or the work shown in the work identifier and the work sound shown in the sound data, and uses it as the onomatopoeia corresponding to the work sound of the work.

[0251] If the onomatopoeia corresponding to the working tone shown in the sound data is not registered in the onomatopoeia registration unit 16, then the working tone acquisition unit 21a and Figure 5 Unlike the work sound acquisition unit 17a shown, the work sound acquisition unit 21a determines the onomatopoeia corresponding to the sound data and uses it as the onomatopoeia corresponding to the work sound.

[0252] Specifically, if various job sounds not registered in the onomatopoeia registration unit 16 and the onomatopoeia corresponding to each job sound are stored, for example, in the internal memory of the job sound acquisition unit 21a, the job sound acquisition unit 21a determines the job sound corresponding to the job sound shown in the sound data by comparing the job sound shown in the sound data with the various job sounds stored in the internal memory. Then, the job sound acquisition unit 21a retrieves the onomatopoeia corresponding to the determined job sound from the internal memory.

[0253] As various work sounds not registered in the onomatopoeia registration section 16, there are work sounds in cases where a box with screws is shaken and placed inside. As onomatopoeia corresponding to this work sound, there are, for example, "click-clack" or "splash-splash".

[0254] The sound acquisition unit 21a outputs sound data representing the onomatopoeia acquired from the onomatopoeia registration unit 16, or sound data representing the determined onomatopoeia, to the text information output unit 13b.

[0255] In the above implementation method 4, Figure 9 The work assistance device 2 shown is configured such that if the onomatopoeia corresponding to the work sound shown in the sound data acquired by the sound data acquisition unit 15 is not registered in the onomatopoeia registration unit 16, the text information acquisition unit 21 determines the onomatopoeia corresponding to the sound data acquired by the sound data acquisition unit 15 as the onomatopoeia corresponding to the work sound. Therefore, in Figure 9 In the work assistance device 2 shown, with Figure 1 Similarly, the work assistance device 2 shown allows the operator or the monitor of the work to confirm whether the work they are performing or the work being monitored is being performed correctly. In addition, even if the operator is performing a work involving a work sound that is not registered in the sound registration unit 16, the operator can confirm the work sound.

[0256] Implementation method 5.

[0257] In Embodiment 5, the work assistance device 2, which includes a progress determination unit 24 for determining the progress of a work, will be described.

[0258] Figure 11 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 5. Figure 11 In, with Figure 1 , Figure 5 , Figure 7 and Figure 9 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0259] Figure 12 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 5. Figure 12 In, with Figure 2 , Figure 6 , Figure 8 and Figure 10 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0260] Figure 11 The work assistance device 2 shown includes a position data acquisition unit 11, a sound registration unit 12, a text information acquisition unit 13, a work step data acquisition unit 23, a progress status determination unit 24, and a scheduled work determination unit 25.

[0261] Work step data acquisition section 23, for example, by Figure 12 The data acquisition circuit 43 is used to implement the operation steps shown.

[0262] The work step data acquisition unit 23 acquires work step data that indicates the sequence of work performed by the operator.

[0263] The work step data acquisition unit 23 outputs the work step data to the progress status determination unit 24 and the scheduled work determination unit 25 respectively.

[0264] Progress status assessment department 24, for example, by Figure 12 The progress status determination circuit 44 shown is implemented.

[0265] The progress status determination unit 24 obtains the work information determined by the work determination unit 10 and obtains the work step data from the work step data acquisition unit 23.

[0266] The progress assessment unit 24 determines the progress of the work based on the work information and work step data.

[0267] The progress status determination unit 24 outputs the progress status determination result to the scheduled operation determination unit 25.

[0268] Pre-determined operation determination unit 25, for example, by Figure 12 The predetermined task determination circuit 45 shown is implemented.

[0269] The scheduled operation determination unit 25 obtains operation step data from the operation step data acquisition unit 23 and obtains the progress status determination result from the progress status determination unit 24.

[0270] Based on the judgment results of the work step data and progress status, the scheduled work determination unit 25 determines the next scheduled work for the operator and displays the determined work content on the display device 3.

[0271] Figure 11 The work assistance device 2 shown applies the work step data acquisition unit 23, the progress status determination unit 24, and the predetermined work determination unit 25 to the work steps data acquisition unit 23, the progress status determination unit 24, and the predetermined work determination unit 25 respectively. Figure 1 The work assistance device 2 shown is used to obtain the work. However, this is only one example; the work step data acquisition unit 23, the progress status determination unit 24, and the predetermined work determination unit 25 can also be applied to the work. Figure 5 The work auxiliary device 2 shown Figure 7 The work assistance device 2 shown, or Figure 9 The work assistance device 2 shown is obtained.

[0272] exist Figure 11 In this context, it is assumed that the position data acquisition unit 11, the sound registration unit 12, the work sound acquisition unit 13a, the text information output unit 13b, the work step data acquisition unit 23, the progress status determination unit 24, and the predetermined work determination unit 25, which are components of the work assistance device 2, respectively pass through... Figure 12 This is implemented using dedicated hardware as shown. That is, it is assumed that the work assistance device 2 is implemented by a position data acquisition circuit 31, a sound registration circuit 32, a work sound acquisition circuit 33, a text information output circuit 34, a work step data acquisition circuit 43, a progress status determination circuit 44, and a predetermined work determination circuit 45.

[0273] The location data acquisition circuit 31, the operation sound acquisition circuit 33, the text information output circuit 34, the operation step data acquisition circuit 43, the progress status determination circuit 44, and the predetermined operation determination circuit 45 each correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0274] The components of the work assistance device 2 are not limited to being implemented through dedicated hardware; the work assistance device 2 can also be implemented through software, firmware, or a combination of software and firmware.

[0275] When the operation assistance device 2 is implemented by software or firmware, the onomatopoeia registration unit 12 is configured to... Figure 3 The program for executing the processing steps of the location data acquisition unit 11, the work sound acquisition unit 13a, the text information output unit 13b, the work step data acquisition unit 23, the progress status determination unit 24, and the predetermined work determination unit 25 is stored in the memory 51 shown. Then, Figure 3 The processor 52 shown executes the program stored in the memory 51.

[0276] In addition, Figure 12 The diagram shows an example where the components of the work assistance device 2 are implemented using dedicated hardware. Figure 3 The example shown is an example of a work assistance device 2 implemented by software or firmware. However, this is just one example; it is also possible that some components of the work assistance device 2 are implemented by dedicated hardware, while the remaining components are implemented by software or firmware.

[0277] Next, for Figure 11 The operation of the work assistance system shown will be explained. However, apart from the work step data acquisition unit 23, the progress status determination unit 24, and the scheduled work determination unit 25, the operation is similar to... Figure 1 The operation assistance system shown is the same. Therefore, only the operation of the operation step data acquisition unit 23, the progress status determination unit 24, and the scheduled operation determination unit 25 will be described here.

[0278] The work step data acquisition unit 23 acquires work step data that indicates the sequence of work performed by the operator.

[0279] As for the work performed by the operator, for example, if there is a work (1) to tighten a screw on the work object, a work (2) to embed a connector on the work object, or a work (3) to make a hole on the work object, then the order of these work (1) to (3) is recorded in the work step data as shown below.

[0280] Homework step data = {Homework (1) → Homework (2) → Homework (3)}

[0281] The work step data acquisition unit 23 outputs the work step data to the progress status determination unit 24 and the scheduled work determination unit 25 respectively.

[0282] As information about the operation determined by the operation determination unit 10, the progress status determination unit 24 obtains position data from the position data acquisition unit 11 and operation step data from the operation step data acquisition unit 23.

[0283] The progress assessment unit 24 determines the progress of the operation based on location data and operation step data.

[0284] The progress status determination unit 24 outputs the progress status determination result to the scheduled operation determination unit 25.

[0285] The following is a detailed explanation of the progress status determination process performed by the progress status determination unit 24.

[0286] The internal memory of the progress determination unit 24, for example, as shown below, records the correspondence between the job position shown by the position data and the job.

[0287] Work position (1) → Work of tightening screws on the work object (1)

[0288] Work location (2) → Work (2) of embedding connectors into the work object.

[0289] Work location (3) → Work involving drilling holes in the work object (3)

[0290] The progress determination unit 24 determines the operation corresponding to the operation location shown in the location data by referring to the above correspondence.

[0291] If the progress determination unit 24 determines the work corresponding to the work location, it determines the progress status of the work based on the work step data.

[0292] Specifically, if the task corresponding to the task position is, for example, task position (1), the progress status determination unit 24 determines that the task the operator is currently performing is task (1), and uses this as the progress status of the task.

[0293] If the task corresponding to the task position is, for example, task position (2), then the progress status determination unit 24 determines that the task the operator is currently performing is task (2), and uses this as the progress status of the task.

[0294] If the task corresponding to the task position is, for example, task position (3), then the progress status determination unit 24 determines that the task that the operator is currently performing is task (3), and uses this as the progress status of the task.

[0295] The scheduled operation determination unit 25 obtains operation step data from the operation step data acquisition unit 23 and obtains the progress status determination result from the progress status determination unit 24.

[0296] Based on the judgment results of the work step data and progress status, the scheduled work determination unit 25 determines the next scheduled work for the operator.

[0297] Specifically, if it is determined that the work that the operator is about to perform is work (1) as the progress status of the work, the scheduled work determination unit 25 determines that the next scheduled work of the operator is work (2) based on the work step data.

[0298] If the current task to be performed by the operator is determined to be task (2) as the progress of the task, the scheduled task determination unit 25 determines the next scheduled task to be performed by the operator based on the task step data (3).

[0299] The scheduled task determination unit 25 causes the display device 3 to display the content of the next scheduled task for the operator.

[0300] Specifically, if the operator's next scheduled task is, for example, task (2), the scheduled task determination unit 25 causes the display device 3 to display the content of a task such as "embedding a connector on the work object".

[0301] If the operator's next scheduled task is, for example, task (3), the scheduled task determination unit 25 causes the display device 3 to display the content of the task, such as "drilling a hole in the work object".

[0302] The contents of these jobs are stored, for example, in the internal memory of the job determination unit 25.

[0303] In the above implementation method 5, Figure 11 The work assistance device 2 shown is configured to include: a work step data acquisition unit 23, which acquires work step data representing the work content; and a progress status determination unit 24, which determines the progress status of the work based on the work information determined by the work determination unit 10 and the work step data acquired by the work step data acquisition unit 23. Therefore, in Figure 11 In the work assistance device 2 shown, with Figure 1 Similarly, the work assistance device 2 shown allows the operator or the monitor of the work to confirm whether the work they are performing or the work being monitored is being performed correctly. In addition, the operator or the monitor can confirm the progress of the work.

[0304] In addition, in implementation method 5, Figure 11 The work assistance device 2 shown is configured to include a predetermined work determination unit 25. This predetermined work determination unit 25 determines the next predetermined work for the operator based on work step data obtained by the work step data acquisition unit 23 and the progress status determination result of the progress status determination unit 24, and then displays the determined work content on the display device 3. Therefore, in Figure 11 In the work assistance device 2 shown, the operator or monitor can confirm the next scheduled work.

[0305] Implementation method 6.

[0306] In Embodiment 6, a work assistance device equipped with a text information acquisition unit 27 will be described. This text information acquisition unit 27, when the worker is performing a silent task, acquires text information of an effect sound corresponding to the task, text information of a mimetic word or the sound representing a mimetic word corresponding to the task, or any of the following based on position data acquired by the position data acquisition unit 11: the effect sound, the sound representing a mimetic word, or a symbol corresponding to the task. The effect sound and the sound representing a mimetic word are sounds added as part of the performance effect, and are not the task sound itself. However, as an effect sound, for example, when the worker performs a silent task, a sound associated with the performance of that task is used. Therefore, the effect sound is a concept included in the sound representing the task.

[0307] Mimetic words are words that describe the state of an object without actual sound, including words that express an atmosphere of silence or language that describes a state of silence. Additionally, the sound representing a mimetic word is the sound used to represent the mimetic word, including sounds that express an atmosphere of silence or a state of silence. Mimetic words and onomatopoeia are concepts included within onomatopoeia. Therefore, the sound representing a mimetic word corresponding to a task is a concept included in the sound representing the task.

[0308] Figure 13 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 6. Figure 13 In, with Figure 1 , Figure 5 , Figure 7 , Figure 9 and Figure 11 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0309] Figure 14 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 6. Figure 14 In, with Figure 2 , Figure 6 , Figure 8 , Figure 10 and Figure 12 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0310] Figure 13 The work assistance device 2 shown includes a position data acquisition unit 11, a sound recording unit 26, and a text information acquisition unit 27.

[0311] For example, by the Onomatopoeia Registration Department 26 Figure 14 The onomatopoeic registration circuit 46 shown is implemented.

[0312] The onomatopoeia registration department 26 is a database that registers onomatopoeia corresponding to the onomatopoeia used when work is carried out at multiple work locations for the work object.

[0313] In cases where the work performed at various work locations is not a sound-producing task, or where the sound produced during the work is difficult to discern, the sound effect registration unit 26 may be used to register sound effects, mimetic words, or sounds or symbols representing those mimetic words in place of the sound effect corresponding to the work sound.

[0314] Figure 13 The work assistance device 2 shown has a built-in onomatopoeia registration unit 26. However, this is just one example, and the onomatopoeia registration unit 26 can also be provided on the outside of the work assistance device 2.

[0315] The text information acquisition unit 27 includes a voice acquisition unit 27a and a text information output unit 27b.

[0316] The text information acquisition unit 27 acquires location data from the location data acquisition unit 11.

[0317] Text Information Acquisition Department 27 and Figure 1 Similarly, the text information acquisition unit 13 shown also acquires text information representing the sound of the operation, i.e., the operation sound.

[0318] The text information acquisition unit 27 may also acquire any one of the following as the text information of the operation sound: the text information of the effect sound corresponding to the operation, the text information of the mimetic word, the text information representing the sound of the mimetic word, or the information of the symbol corresponding to the operation, based on the position data acquired by the position data acquisition unit 11.

[0319] The text information acquisition unit 27 outputs the acquired text information of the operation sound to the display device 3.

[0320] For example, by the sound acquisition section 27a of the work. Figure 14 The operation sound acquisition circuit 47 shown is implemented.

[0321] The sound acquisition unit 27a acquires position data from the position data acquisition unit 11.

[0322] Work Sound Acquisition Section 27a and Figure 1 Similarly, the work sound acquisition unit 13a shown acquires the simulated sound corresponding to the work sound based on the position data.

[0323] Specifically, the work sound acquisition unit 27a acquires from the onomatopoeia registration unit 26 the onomatopoeia corresponding to the work sound of the work performed at the work position shown in the position data, and uses it as the onomatopoeia corresponding to the work sound of the work.

[0324] The sound acquisition unit 27a outputs the sound data representing the onomatopoeia to the text information output unit 27b.

[0325] The sound acquisition unit 27a can also be used, for example, when the operator is performing a task that does not produce sound. Figure 1 Unlike the work sound acquisition unit 13a shown, it acquires effect sounds and the like corresponding to work that does not produce sound, based on the position data acquired by the position data acquisition unit 11.

[0326] Specifically, the operation sound acquisition unit 27a acquires from the onomatopoeia registration unit 26 the effect sound corresponding to the operation position shown in the position data, the sound of the onomatopoeia word corresponding to the operation position shown in the position data, or the symbol corresponding to the operation position shown in the position data, as the operation sound when performing an operation that does not produce sound.

[0327] The sound acquisition unit 27a outputs any one of the sound data representing sound effects, sound data representing mimetic words, and text information of symbols to the text information output unit 27b.

[0328] In order to perform the same processing as the effect sound, the working sound acquisition unit 27a can temporarily acquire the mimetic word as sound data and output it to the text information output unit 27b. However, it can also replace acquiring the sound representing the mimetic word, acquire the text information of the mimetic word and output it directly to the text information output unit 27b.

[0329] Text information output unit 27b, for example, via Figure 14 The text information output circuit 48 shown is used to achieve this.

[0330] If sound data representing onomatopoeia is output from the sound acquisition unit 27a, the text information output unit 27b converts the onomatopoeia represented by the sound data into text information.

[0331] If sound data representing sound effects is output from the sound acquisition unit 27a, the text information output unit 27b converts the sound effects represented by the sound data into text information.

[0332] If the sound data representing a mimetic word is output from the sound acquisition unit 27a, the text information output unit 27b converts the mimetic word shown in the sound data into text information.

[0333] If the signal information is output from the sound acquisition unit 27a, the text information output unit 27b converts the signal information into data that the display device 3 can display, as needed. The text information output unit 27b can also convert the acquired signal information into either text information or image data. If the acquired signal information is already data that the display device 3 can display, the text information output unit 27b can omit the above conversion process.

[0334] The text information output unit 27b outputs any one of the following to the display device 3: onomatopoeic text information, sound effect text information, mimetic word text information, and symbols.

[0335] Figure 13 The work assistance device 2 shown applies the onomatopoeia registration unit 26 and the text information acquisition unit 27 to respectively Figure 1 This is obtained by using the work assistance device 2 shown. However, this is only one example; it is also possible to apply the onomatopoeia registration unit 26 and the text information acquisition unit 27 to the work assistance device 2, respectively. Figure 5 The work auxiliary device 2 shown Figure 7 The work auxiliary device 2 shown Figure 9 The work assistance device 2 shown or Figure 11 The work assistance device 2 shown is obtained.

[0336] exist Figure 13 In this context, it is assumed that the position data acquisition unit 11, the onomatopoeia registration unit 26, the work sound acquisition unit 27a, and the text information output unit 27b, which are components of the work assistance device 2, are respectively transmitted through... Figure 14 This is implemented using dedicated hardware as shown. That is, it is assumed that the work assistance device 2 is implemented through a position data acquisition circuit 31, a sound registration circuit 46, a work sound acquisition circuit 47, and a text information output circuit 48.

[0337] Here, the onomatopoeic registration circuit 46 corresponds, for example, to non-volatile or volatile semiconductor memories such as RAM, ROM, flash memory, EPROM, EEPROM, disk, floppy disk, optical disk, compact disk, mini disk, or DVD.

[0338] Furthermore, the location data acquisition circuit 31, the operation sound acquisition circuit 47, and the text information output circuit 48 each correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0339] The components of the work assistance device 2 are not limited to being implemented through dedicated hardware; the work assistance device 2 can also be implemented through software, firmware, or a combination of software and firmware.

[0340] When the operation assistance device 2 is implemented by software or firmware, the onomatopoeia registration unit 26 is configured to... Figure 3 The program for executing the processing steps of the position data acquisition unit 11, the operation sound acquisition unit 27a, and the text information output unit 27b is stored in the memory 51 shown. Then, Figure 3 The processor 52 shown executes the program stored in the memory 51.

[0341] In addition, Figure 14 The diagram shows an example where the components of the work assistance device 2 are implemented using dedicated hardware. Figure 3 The example shown is an example of a work assistance device 2 implemented by software or firmware. However, this is just one example; it is also possible that some components of the work assistance device 2 are implemented by dedicated hardware, while the remaining components are implemented by software or firmware.

[0342] Next, for Figure 13 The operation of the work assistance system shown will be explained. However, apart from the onomatopoeia registration unit 26, the work sound acquisition unit 27a, and the text information output unit 27b, the operation of the system is similar to that of the work assistance system. Figure 1 The operation assistance system shown is the same. Therefore, only the operation of the onomatopoeia registration unit 26, the operation sound acquisition unit 27a, and the text information output unit 27b will be described here.

[0343] The onomatopoeia registration department 26 contains onomatopoeia corresponding to the working sounds when working is carried out at various working positions for the work object.

[0344] As for the work location for the work object, there are, for example, 8 work locations (1) to (8).

[0345] In the case where the work at the three work positions (1) to (3) is a sound-generating work, work position (1) is, for example, the position where a screw is tightened into the work object using an electric screwdriver, work position (2) is, for example, the position where a connector is fitted, and work position (3) is, for example, the position where a hole is drilled in the work object using an electric drill.

[0346] In such a case, the onomatopoeia corresponding to the onomatopoeia of the work performed at the work positions (1) to (3) is, for example, registered in the onomatopoeia registration section 26 as shown below.

[0347] Work location (1) → Onomatopoeia "da da"

[0348] Work location (2) → Onomatopoeia "click"

[0349] Work location (3) → Onomatopoeia "Zzzzz"

[0350] In the case where the work at work positions (4) to (8) is a noiseless operation, work position (4) is, for example, the position where the moisture of the work object is wiped with a cloth.

[0351] In such a case, as an effect sound for the work sound when the work is being performed at the work position (4), for example, an effect sound as shown below is registered in the onomatopoeia registration unit 26.

[0352] Work location (4) → Sound effect "hiss"

[0353] The working position (5) is, for example, the position in which the part is aligned with the object being worked in the case of an operation. The working position (6) is, for example, the position of the dial for rotating the object being worked.

[0354] In such a case, the sound of the mimetic word corresponding to the sound of the work performed at work positions (5) to (6) is, for example, registered in the mimetic word registration section 26 as shown below.

[0355] Location (5) → The sound "pat" representing the mimetic word

[0356] Location (6) → The sound "click-clack" representing the mimetic word

[0357] The working position (7) is, for example, the position corresponding to the feature point of the component that requires attention to the installation direction, and the working position (8) is, for example, the position of the protrusion of the component installed on the object of the work.

[0358] In such cases, as a symbol corresponding to the working sound when working at the working positions (7) to (8), for example, a symbol as shown below is registered in the onomatopoeia registration section 26.

[0359] Work location (7) → Mark ""

[0360] Work location (8) → Mark "!"

[0361] The sound acquisition unit 27a acquires position data from the position data acquisition unit 11.

[0362] The work sound acquisition unit 27a acquires from the onomatopoeia registration unit 26 any one of the following: the onomatopoeia corresponding to the work position shown in the position data, the effect sound corresponding to the work position shown in the position data, the sound of the onomatopoeic word corresponding to the work position shown in the position data, or the symbol corresponding to the work position shown in the position data, as the work sound when the worker performs the work.

[0363] If the work location shown in the location data is, for example, work location (1), then the work sound acquisition unit 27a acquires "da da" as an onomatopoeia.

[0364] If the work location shown in the location data is, for example, work location (2), then the work sound acquisition unit 27a acquires "click" as an onomatopoeia.

[0365] If the work location shown in the location data is, for example, work location (3), then the work sound acquisition unit 27a acquires "zzzz" as an onomatopoeia.

[0366] If the work position shown in the location data is, for example, work position (4), then the work sound acquisition unit 27a acquires "hiss" as an effect sound.

[0367] If the work location shown in the location data is, for example, work location (5), then the work sound acquisition part 27a acquires "pata" as the sound representing the mimetic word.

[0368] If the work location shown in the location data is, for example, work location (6), then the work sound acquisition unit 27a acquires "click-click" as the sound representing the mimetic word.

[0369] If the work position shown in the location data is, for example, work position (7), then the work sound acquisition unit 27a acquires ““” as a mark.

[0370] If the work location shown in the location data is, for example, work location (8), then the work sound acquisition unit 27a acquires “!” as a mark.

[0371] The sound acquisition unit 27a outputs any one of the acquired sound data representing onomatopoeia, sound data representing sound effects, sound data representing mimetic words, and acquired symbols to the text information output unit 27b.

[0372] The text information output unit 27b acquires sound data representing onomatopoeia, sound data representing sound effects, sound data representing mimetic words, and any symbol from the working sound acquisition unit 27a.

[0373] If the text information output unit 27b acquires sound data representing onomatopoeia, it converts the onomatopoeia shown in the sound data into text information.

[0374] If the text information output unit 27b acquires sound data representing sound effects, it converts the sound effects represented by the sound data into text information.

[0375] If the text information output unit 27b acquires sound data representing a mimetic word, it converts the mimetic word shown in the sound data into text information.

[0376] The text information output unit 27b outputs either the converted text information or any of the acquired symbols to the display device 3.

[0377] Thus, text information or symbols are displayed on display device 3.

[0378] If the work location shown in the location data is, for example, work location (1), then the text information "da da" is displayed on display device 3; if it is work location (4), then the text information "hiss" is displayed on display device 3.

[0379] If the work location shown in the location data is, for example, work location (5), then the text information "pat" is displayed on display device 3; if it is work location (7), then the mark "" is displayed on display device 3.

[0380] For text information or symbols on display device 3, AR display can also be achieved using onomatopoeia from comics or dialogue boxes from comics.

[0381] In the above-described embodiment 6, the work assistance device 2 is configured such that the text information acquisition unit 27 acquires text information of the sound effect corresponding to the work, text information of the sound representing the mimetic word corresponding to the work, and any one of the symbols corresponding to the work, as text information of the work sound for the determined work. Therefore, in the work assistance device 2, regardless of whether the work performed by the worker is a sound-producing work or a soundless work, the worker or the monitor supervising the work can confirm whether the work they are performing or the work being monitored is being performed correctly.

[0382] Implementation method 7.

[0383] In Embodiment 7, the work assistance device 2, which includes a correctness determination unit 28 for determining the correctness of a work operation, will be described.

[0384] Figure 15 This is a structural diagram showing a work assistance system including the work assistance device 2 of embodiment 7. Figure 15 In, with Figure 1 , Figure 5 , Figure 7 , Figure 9 , Figure 11 and Figure 13 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0385] Figure 16 This is a hardware structure diagram showing the hardware of the work assistance device 2 according to embodiment 7. Figure 16 In, with Figure 2 , Figure 6 , Figure 8 , Figure 10, Figure 12 and Figure 14 The same labels indicate the same or equivalent parts, so detailed descriptions are omitted.

[0386] Figure 15 The work assistance device 2 shown includes a position data acquisition unit 11, a sound recording unit 12, a text information acquisition unit 13, a work step data acquisition unit 23, a correctness judgment unit 28, and a attention reminder unit 29.

[0387] Correctness / incorrectness determination section 28, for example, by Figure 16 The correctness determination circuit 49 shown is implemented.

[0388] The error determination unit 28 obtains position data from the position data acquisition unit 11 and obtains operation step data from the operation step data acquisition unit 23.

[0389] The error determination unit 28 determines the correctness of the operation based on the location data and operation step data.

[0390] The correctness determination unit 28 outputs the correctness determination result to the attention reminder unit 29.

[0391] Note: Reminder 29, for example, by... Figure 16 The attention reminder circuit 50 shown is implemented.

[0392] Note that the reminder unit 29 obtains the correctness judgment result from the correctness judgment unit 28.

[0393] If the error determination unit 28 determines that there is an error in the operation, the attention reminder unit 29 will remind the operator to pay attention.

[0394] Figure 15 The work assistance device 2 shown applies the work step data acquisition unit 23, the error judgment unit 28, and the attention reminder unit 29 to the work steps respectively. Figure 1 This is obtained by using the work assistance device 2 shown. However, this is only one example; the work step data acquisition unit 23, the error judgment unit 28, and the attention reminder unit 29 could also be applied to... Figure 5 The work auxiliary device 2 shown Figure 7 The work auxiliary device 2 shown Figure 9 The work auxiliary device 2 shown Figure 11 The work assistance device 2 shown, or Figure 13 The work assistance device 2 shown is obtained.

[0395] exist Figure 15 In this context, it is assumed that the position data acquisition unit 11, the onomatopoeia registration unit 12, the work sound acquisition unit 13a, the text information output unit 13b, the work step data acquisition unit 23, the error judgment unit 28, and the attention reminder unit 29, which are components of the work assistance device 2, are respectively transmitted through... Figure 16 The dedicated hardware implementation is shown. That is, it is assumed that the work assistance device 2 is implemented by a position data acquisition circuit 31, a sound registration circuit 32, a work sound acquisition circuit 33, a text information output circuit 34, a work step data acquisition circuit 43, a correctness judgment circuit 49, and a attention reminder circuit 50.

[0396] The location data acquisition circuit 31, the operation sound acquisition circuit 33, the text information output circuit 34, the operation step data acquisition circuit 43, the correctness judgment circuit 49, and the attention reminder circuit 50 each correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0397] The components of the work assistance device 2 are not limited to being implemented through dedicated hardware; the work assistance device 2 can also be implemented through software, firmware, or a combination of software and firmware.

[0398] When the operation assistance device 2 is implemented by software or firmware, the onomatopoeia registration unit 12 is configured to... Figure 3 The program for executing the processing steps of the position data acquisition unit 11, the work sound acquisition unit 13a, the text information output unit 13b, the work step data acquisition unit 23, the error judgment unit 28, and the attention reminder unit 29 is stored in the memory 51 of the computer shown. Then, Figure 3 The processor 52 shown executes the program stored in the memory 51.

[0399] In addition, Figure 16 The diagram shows an example where the components of the work assistance device 2 are implemented using dedicated hardware. Figure 3 The example shown is an example of a work assistance device 2 implemented by software or firmware. However, this is just one example; it is also possible that some components of the work assistance device 2 are implemented by dedicated hardware, while the remaining components are implemented by software or firmware.

[0400] Next, for Figure 15 The operation of the illustrated work assistance system will be explained. This includes, except for the error judgment unit 28 and the attention reminder unit 29, which are related to… Figure 11 The operation assistance system shown is the same. Therefore, only the operation of the error judgment unit 28 and the attention reminder unit 29 will be described here.

[0401] The error determination unit 28 obtains position data from the position data acquisition unit 11 and obtains operation step data from the operation step data acquisition unit 23.

[0402] The error determination unit 28 determines the correctness of the operation performed by the operator based on the location data and operation step data.

[0403] The following is a detailed explanation of the correctness determination process performed by the correctness determination unit 28.

[0404] In the internal memory of the error determination unit 28, for example, as shown below, the correspondence between the job position shown by the position data and the job is recorded.

[0405] Work position (1) → Work of tightening screws on the work object (1)

[0406] Work location (2) → Work (2) of embedding connectors into the work object.

[0407] Work location (3) → Work (3) to open the first hole on the work object.

[0408] Work location (4) → Work (4) to open the second hole on the work object.

[0409] In addition, the operator's past work history is recorded in the internal memory of the error determination unit 28.

[0410] If the operator has performed, for example, the operation of fastening screws to the work object (1) and the operation of fitting connectors to the work object (2), and if the operation (1) and the operation (2) were performed normally, then the operation history indicating that the operation (1) and the operation (2) were performed is recorded.

[0411] The error determination unit 28 determines the operation that the operator is about to perform based on the location data obtained from the location data acquisition unit 11.

[0412] If the work position shown in the location data is, for example, work position (3), then the error determination unit 28 determines that the work that the operator is about to perform is to open the first hole on the work object (3).

[0413] If the work position shown in the location data is, for example, work position (4), then the error determination unit 28 determines that the work that the operator is about to perform is to open the second hole on the work object (4).

[0414] In addition, the error determination unit 28 determines the actual work that the operator should perform by comparing the order of the work shown in the work step data with the work history recorded in the internal memory, etc.

[0415] If the job history recorded in the internal memory or the like indicates that jobs (1) and (2) have been performed, then the error determination unit 28 determines that the real job that the operator should perform is job (3).

[0416] The error determination unit 28 can also determine whether to transfer to the next operation based on the text information of the operation sound output by the text information acquisition unit 13 to the display device 3.

[0417] Specifically, the error determination unit 28, for example, if it stores text information such as the work sounds displayed up to this point as work history, can also determine whether the worker can proceed to the next task by judging whether the pre-determined text information such as the work sounds is displayed correctly. For example, if the specification is to display the sound produced when a screw is finally tightened as the work sound, by judging whether the work sound is displayed, subsequent processing can easily determine whether the screw tightening process was performed correctly. In this way, the presence or absence of text information such as work sounds can also be used in the completion judgment of each task.

[0418] If the work that the operator is about to perform is consistent with the actual work that the operator should be performing, then the error determination unit 28 determines that the work performed by the operator is the correct work.

[0419] If the work that the operator is currently performing is inconsistent with the actual work that the operator should be performing, then the correctness determination unit 28 determines that the work performed by the operator is an incorrect work.

[0420] The correctness determination unit 28 outputs the correctness determination result to the attention reminder unit 29.

[0421] Note that the reminder unit 29 obtains the correctness judgment result from the correctness judgment unit 28.

[0422] If the error determination unit 28 determines that the operation the operator is about to perform is incorrect, the attention reminder unit 29 reminds the operator, for example, by displaying a message on the display device 3 indicating that an incorrect operation is about to be performed.

[0423] In the above implementation method 7, Figure 15 The work assistance device 2 shown is configured to include: a work step data acquisition unit 23, which acquires work step data representing the work content; a correctness determination unit 28, which determines the correctness of the work based on the position data acquired by the position data acquisition unit 11 and the work step data acquired by the work step data acquisition unit 23; and a attention reminder unit 29, which reminds the operator to pay attention if the determination result of the correctness determination unit 28 indicates that there is an error in the work. Therefore, in Figure 15 The work assistance device 2 shown can prevent errors in the work performed by the operator.

[0424] In embodiments 1 to 7, when the operator performs the work using their own hands, the work assistance device 2 outputs textual information of the work audio to the display device 3. However, this is only one example; the work assistance device 2 may also output textual information of the work audio to the display device 3 when the operator performs the work remotely, for example, by using a robot arm. In this case, the display device 3 is positioned where the operator can see it.

[0425] Therefore, even if the operator is located at a remote location far from the work site, they can confirm whether the work has been performed correctly by observing the text information displayed on the display device 3.

[0426] Furthermore, this disclosure allows for free combination of various embodiments, or arbitrary modification of the constituent elements of each embodiment, or omission of any constituent element in each embodiment.

[0427] Industrial availability

[0428] This disclosure applies to work assistance devices, work assistance methods, and work assistance systems.

[0429] Label Explanation

[0430] 1. Sensor; 2. Work Aid Device; 3. Display Device; 4. Microphone; 10. Work Determination Unit; 11. Position Data Acquisition Unit; 12. Sound Registration Unit; 13. Text Information Acquisition Unit; 13a. Work Sound Acquisition Unit; 13b. Text Information Output Unit; 15. Sound Data Acquisition Unit; 16. Sound Registration Unit; 17. Text Information Acquisition Unit; 17a. Work Sound Acquisition Unit; 18. Sound Data Extraction Unit; 19. Ambient Sound Determination Unit; 20. Text Information Acquisition Unit; 20a. Work Sound Acquisition Unit; 20b. Text Information Output Unit; 21. Text Information Acquisition Unit; 21a. Work Sound Acquisition Unit; 23. Work Step Data Acquisition Unit; 24. Progress Status Determination Unit; 25. Scheduled Work Determination Unit; 26. Sound Registration Unit; 27. Text Information Acquisition Unit; 27a. Work Sound Acquisition Unit 27b Text Information Output Unit, 28 Correct / Error Judgment Unit, 29 Attention Reminder Unit, 31 Location Data Acquisition Circuit, 32 Onomatopoeia Registration Circuit, 33 Work Sound Acquisition Circuit, 34 Text Information Output Circuit, 35 Sound Data Acquisition Circuit, 36 Onomatopoeia Registration Circuit, 37 Work Sound Acquisition Circuit, 38 Sound Data Extraction Circuit, 39 Ambient Sound Determination Circuit, 40 Work Sound Acquisition Circuit, 41 Text Information Output Circuit, 42 Work Sound Acquisition Circuit, 43 Work Step Data Acquisition Circuit, 44 Progress Status Judgment Circuit, 45 Scheduled Work Determination Circuit, 46 Onomatopoeia Registration Circuit, 47 Work Sound Acquisition Circuit, 48 Text Information Output Circuit, 49 Correct / Error Judgment Circuit, 50 Attention Reminder Circuit, 51 Memory, 52 Processor.

Claims

1. A work assistance device, comprising: The task determination department determines the tasks that serve as auxiliary objects; and The text information acquisition unit acquires text information of the operation sound, which is the sound representing the operation determined by the operation determination unit.

2. The work assistance device according to claim 1, characterized in that, The text information acquisition unit acquires the onomatopoeia information corresponding to the determined work sound from the onomatopoeia registration unit, which registers onomatopoeia information corresponding to the work sound that can be used as an auxiliary object, and outputs the onomatopoeia text information to the display device based on the acquired onomatopoeia information.

3. The work assistance device according to claim 2, characterized in that, The sound data of the mimetic sound is registered as mimetic sound information in the mimetic sound registration unit. The text information acquisition unit converts the acquired onomatopoeic sound data into text information and outputs the converted text information to the display device.

4. The work assistance device according to claim 2 or 3, characterized in that, The onomatopoeia registration department registers onomatopoeia corresponding to work sounds during normal operation and onomatopoeia corresponding to work sounds during abnormal operation. The work assistance device includes a sound data acquisition unit that acquires sound data representing the work sounds from a microphone that collects work sounds during the work being assisted. The text information acquisition unit obtains from the onomatopoeia registration unit the onomatopoeia corresponding to the work sound shown in the sound data obtained by the sound data acquisition unit, and uses it as the onomatopoeia corresponding to the work sound of the determined work.

5. The work assistance device according to claim 4, characterized in that, The work assistance device includes a sound data extraction unit. When the sound data acquired by the sound data acquisition unit includes sound data representing work sounds and sound data representing ambient sounds, the sound data extraction unit extracts the sound data representing work sounds from the acquired sound data. The text information acquisition unit obtains from the onomatopoeia registration unit the onomatopoeia corresponding to the work sound shown in the sound data extracted by the sound data extraction unit, and uses it as the onomatopoeia corresponding to the work sound of the determined work.

6. The work assistance device according to claim 5, characterized in that, When the sound data acquired by the sound data acquisition unit includes sound data representing work sounds and sound data representing ambient sounds, the sound data extraction unit extracts the sound data representing ambient sounds from the acquired sound data. The work assistance device includes an ambient sound determination unit, which obtains a simulated sound corresponding to the ambient sound shown by the sound data extracted by the sound data extraction unit.

7. The work assistance device according to claim 5, characterized in that, If the onomatopoeia corresponding to the work sound shown in the sound data obtained by the sound data acquisition unit is not registered in the onomatopoeia registration unit, then the text information acquisition unit will determine the onomatopoeia corresponding to the sound data obtained by the sound data acquisition unit as the onomatopoeia corresponding to the work sound of the determined work.

8. The work assistance device according to any one of claims 1 to 7, characterized in that, The job determination unit determines the job as an auxiliary object based on job instruction data indicating the job to be performed next.

9. The work assistance device according to any one of claims 1 to 8, characterized in that, The job determination unit includes a position data acquisition unit, which acquires position data representing the job position from a sensor that detects the position of the job to be performed, i.e., the job position. The job determination unit determines the job as an auxiliary object based on the location data obtained by the location data acquisition unit.

10. The work assistance device according to claim 9, characterized in that, The sensor detects the work location based on work instruction data indicating the work to be performed next. The location data acquisition unit acquires location data from the sensor, representing the work location detected based on the work instruction data.

11. The work assistance device according to any one of claims 1 to 10, characterized in that, The work assistance device includes: The work step data acquisition unit acquires work step data representing the order of the work; and The progress status determination unit determines the progress status of the operation based on the operation information determined by the operation determination unit and the operation step data obtained by the operation step data acquisition unit.

12. The work assistance device according to claim 11, characterized in that, The work assistance device includes a predetermined work determination unit, which determines the next predetermined work based on the work step data obtained by the work step data acquisition unit and the progress status determination result of the progress status determination unit, and causes the display device to display the content of the determined work.

13. The work assistance device according to any one of claims 1 to 12, characterized in that, The text information acquisition unit acquires text information of the sound effect corresponding to the task, text information of the sound of the mimetic word corresponding to the task, and any one of the symbols corresponding to the task, as the text information of the task sound determined.

14. The work assistance device according to any one of claims 1 to 13, characterized in that, The work assistance device includes: The error determination unit determines the correctness of the operation based on the text information of the operation audio output to the display device by the text information acquisition unit; and If the error determination unit determines that the operation is incorrect, the attention reminder unit will remind the operator to pay attention.

15. The work assistance device according to claim 14, characterized in that, The error determination unit determines whether to transfer to the next operation based on the text information of the operation sound output to the display device by the text information acquisition unit.

16. A method for assisting with a task, wherein, The task determination department determines the tasks that will be used as auxiliary objects. The text information acquisition unit acquires text information of the operation sound, which is the sound indicating the operation determined by the operation determination unit.

17. A job assistance system, comprising: The task determination department determines the tasks that serve as auxiliary objects. The text information acquisition unit acquires text information of the operation sound, which is the sound representing the operation determined by the operation determination unit; and The display device displays text representing the operation sound according to the text information output from the text information acquisition unit.

18. The work assistance system according to claim 17, characterized in that, The work assistance system also includes a sensor that detects the position of the object being assisted, i.e., the work position. The job determination unit includes a location data acquisition unit that acquires location data representing the job location from the sensor. The job determination unit determines the job as an auxiliary object based on the location data obtained by the location data acquisition unit.

19. The work assistance system according to claim 18, characterized in that, The sensor detects the work location based on work instruction data indicating the next task to be performed by the operator. The location data acquisition unit acquires location data from the sensor, representing the work location detected based on the work instruction data.

Images