Work location identification system, work location identification method, and work location identification program
The system uses sound or vibration sensors to identify work locations by estimating hand distances and sound source directions, addressing the limitations of imaging-based systems and ensuring accurate work position determination.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- TOYOTA JIDOSHA KK
- Filing Date
- 2023-09-27
- Publication Date
- 2026-06-23
AI Technical Summary
Existing work position identification systems fail to accurately identify the work location due to variations in operator imaging position or posture, necessitating a solution that does not rely on imaging means.
A system utilizing sound or vibration sensors attached to both hands of the operator to estimate the distance between hands, determine the direction and distance of the sound source, and identify the work location based on pre-configured candidates using AI.
Enables accurate work location identification without imaging means, leveraging sound or vibration sensors to determine the work position by estimating hand distances and sound source directions.
Smart Images
Figure 0007878241000001 
Figure 0007878241000002 
Figure 0007878241000003
Abstract
Description
Technical Field
[0001] The present invention relates to a work position identification system, a work position identification method, and a work position identification program.
Background Art
[0002] A system for grasping the state of work has been developed. In Patent Document 1, a control device for grasping the state of work including a communication unit and a processing unit is disclosed. The communication unit can receive a captured image captured by an imaging device of a work location where work for tightening a fastening part is performed from a tool provided with the imaging device. The processing unit determines whether the fastening part is correctly tightened based on the comparison result between a target image indicating a state where the fastening part is normally tightened at the work location and the captured image.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, depending on the imaging position or the posture of the operator, the system of Patent Document 1 may not be able to identify the work position from the captured image. Therefore, an object of the present disclosure is to provide a work position identification system capable of identifying a work position using a vibration or sound sensor.
Means for Solving the Problems
[0005] One aspect of the work position identification system of the present disclosure is imaging means for imaging an operator, a sound or vibration sensor for sensing work sounds worn on both hands of the operator, A means for estimating the distance between the hands of the worker from images of both hands captured by the imaging means, A sound source estimation means that estimates the direction of the sound source of the work sound and the distance to the sound source of the work sound based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, The work location identification system comprises a work location identification means that identifies a work location based on information obtained from the sound source estimation means and a set of pre-configured work location candidates.
[0006] With the above configuration, a work location identification system can be provided that can identify the work location using vibration or sound sensors.
[0007] Another aspect of the work location identification system of this disclosure is: Glove sensors attached to both hands of the worker, Sound or vibration sensors that detect work sounds are attached to both hands of the worker, A means for estimating the distance between the hands of the worker from the information acquired by the glove sensor, A sound source estimation means that estimates the direction of the sound source of the work sound and the distance to the sound source of the work sound based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, The work location identification system comprises a work location identification means that identifies a work location based on information obtained from the sound source estimation means and a set of pre-configured work location candidates.
[0008] The above configuration provides a work location identification system that can determine the work location using vibration or sound sensors. In particular, it makes it possible to determine the work location without using imaging means.
[0009] The work location identification system of this disclosure is The imaging means captures images of the workspace and sets candidate work positions from the images of the workspace.
[0010] With the above configuration, potential work positions can be set in advance.
[0011] The method for identifying the work location in this disclosure is: The imaging device captures images of the worker, Sound or vibration sensors attached to both hands of the worker detect the work sounds, The distance between the hands is estimated from the images of the worker's hands captured by the imaging means. Based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, the direction of the sound source of the work sound and the distance to the sound source of the work sound are estimated. This is a method for identifying a work location, which identifies the work location based on the estimated direction and distance of the sound source and a set of pre-configured candidate work locations.
[0012] The above configuration provides a work location identification method that can identify the work location using vibration or sound sensors.
[0013] The work location identification program in this disclosure is The imaging device captures images of the worker, Sound or vibration sensors attached to both hands of the worker detect the work sounds, The distance between the hands is estimated from the images of the worker's hands captured by the imaging means. Based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, the direction of the sound source of the work sound and the distance to the sound source of the work sound are estimated. This is a work location identification program that causes an information processing device to identify a work location based on the estimated direction and distance of the sound source and a set of pre-configured work location candidates.
[0014] With the above configuration, a work location identification program can be provided that can identify the work location using vibration or sound sensors. [Effects of the Invention]
[0015] This disclosure provides a work location identification system that can identify the work location using vibration or sound sensors.
Brief Description of the Drawings
[0016] [Figure 1] It is a diagram showing the operating principle of the work position identification system according to the embodiment. [Figure 2] It is a schematic diagram showing the configuration of the work position identification system according to the embodiment. [Figure 3] It is a flowchart of the work position identification method according to the embodiment.
Modes for Carrying Out the Invention
[0017] Embodiment Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the invention according to the claims is not limited to the following embodiments. Also, not all of the configurations described in the embodiments are essential as means for solving the problems. For the sake of clarity of explanation, the following description and drawings have been appropriately omitted and simplified. In each drawing, the same elements are denoted by the same reference numerals, and duplicate explanations are omitted as necessary.
[0018] (Explanation of the work position identification system according to the embodiment) FIG. 1 is a diagram showing the operating principle of the work position identification system according to the embodiment. FIG. 2 is a schematic diagram showing the configuration of the work position identification system according to the embodiment. The work position identification system 200 will be described with reference to FIGS. 1 and 2. The work position identification system 200 is, for example, a system that identifies the position where work is being performed from a plurality of work position candidates.
[0019] As shown in Figure 1, the work position identification system according to this embodiment uses an imaging means to capture images of the work area and pre-set candidate work positions from the images of the work area. The work position identification system 200 searches for and sets candidate work positions from within the work area using AI (Artificial Intelligence) or the like. In this case, there are multiple candidate work positions, A, B, and C. When sound or vibration sensors set on both hands detect a value exceeding a threshold, they record a waveform. For example, suppose the waveforms of the right hand and the left hand are obtained as shown in the lower part of Figure 1.
[0020] If the distance between the right and left hands is known from video or a glove sensor, the approximate distance to the sound source can be estimated by comparing the distance between both hands with the data acquired from sound or vibration sensors attached to both hands. By multiplying the estimated distance by the data of candidate work positions, the work position can be identified. For example, in this case, the work position can be identified as C.
[0021] As shown in Figure 2, the work position identification system 200 according to this embodiment includes an imaging means 201, a sound or vibration sensor 202, a means for estimating the distance between both hands 203, a sound source estimation means 204, and a work position identification means 205.
[0022] The imaging means 201 pre-images the workplace as described above. The imaging means 201 also images the worker and both of their hands. The imaging means 201 is, for example, an RGB camera. The imaging means for imaging the workplace and the imaging means for imaging the worker may be the same or different. The imaging means for imaging the workplace may capture still images. It is preferable that the imaging means for imaging the worker captures video.
[0023] The sound or vibration sensor 202 is attached to both hands of the worker. The sound or vibration sensor 202 detects work sounds. By detecting values that exceed a threshold, the sound or vibration sensor 202 can acquire work sounds without picking up background noise in the workplace.
[0024] The two-hand distance estimation means 203 receives video data from the imaging means 201. The two-hand distance estimation means 203 estimates the distance between the hands from the images of the worker's hands captured by the imaging means 201. The two-hand distance estimation means 203 estimates the distance between the hands using AI. Alternatively, a glove sensor may be used instead of the imaging means 201. The two-hand distance estimation means 203 may also estimate the distance between the hands by acquiring data from the glove sensor.
[0025] The sound source estimation means 204 receives sound data from a sound or vibration sensor. Based on the information of the work sound obtained from the sound or vibration sensor and the distance between both hands, the sound source estimation means 204 estimates the direction of the work sound source and the distance to the work sound source. The sound source estimation means 204 may use AI.
[0026] The work position identification means 205 identifies the work position based on information obtained from the sound source estimation means 204 and pre-set work position candidates. The work position identification means 205 identifies the work position using AI. The information processing device comprises a hands-to-hand distance estimation means 203, a sound source estimation means 204, and a work position identification means 205. The information processing device comprises a processor such as a CPU (Central Processing Unit) that processes programs to execute processing, and memory that stores programs. The information processing device may be a single device or may consist of multiple devices. The information processing device may be a cloud server that distributes and executes some or all of its functions.
[0027] With the above configuration, a work position identification system can be provided that can identify the work position using a vibration or sound sensor. Furthermore, when a globe sensor is used, the work position can be identified without using the imaging means 201 for anything other than setting candidate work positions.
[0028] (Description of the method for determining the work position according to the embodiment) Figure 3 is a flowchart of the work location identification method according to the embodiment. The work location identification method according to the embodiment will be explained with reference to Figure 3. The work location identification method is, for example, a method for identifying the location where work is being performed from among multiple candidate work locations.
[0029] As shown in Figure 3, first the imaging means 201 images the candidate work location (step S301). The imaging means 201 images the work area. Next, the work location identification system 200 sets the candidate work location in advance (step S302). Up to this point, it can be performed in the pre-processing stage before actually performing the work.
[0030] Next, the imaging means 201 captures a video of the worker's hands (step S303). The hands-to-hand distance estimation means 203 estimates the distance between the hands (step S305). Alternatively, a glove sensor may be used to measure the hands-to-hand distance instead of the imaging means 201. The work position identification means 205 also acquires the video data (step S304).
[0031] The sound sensor 1 on the right hand detects the work sound (step S306). Similarly, the sound sensor 2 on the left hand detects the work sound (step S307).
[0032] Next, the sound source estimation means 204 estimates the direction and distance from the sound source based on the distance between both hands and the difference in sound intensity between sound sensor 1 and sound sensor 2 (step S308). That is, the sound source estimation means 204 estimates the distance and direction of the sound source by acquiring information from steps S305, S306, and S307.
[0033] Finally, the work position identification means 205 identifies a position from among the designated work position candidates (step S309). The work position identification means 205 identifies the work position based on the work position candidates set in step S302 and the distance and direction of the sound source. The work position identification means 205 may also use the video data acquired in step S305 as a work position candidate, or it may set work position candidates using only video data without pre-setting any work position candidates.
[0034] The above configuration provides a work position identification method that can identify the work position using a vibration or sound sensor. Furthermore, when a glove sensor is used, the work position can be identified without using an imaging means other than preprocessing. In addition, a work position identification program can be provided by having an information processing device execute the above work position identification method.
[0035] Furthermore, some or all of the processing in the information processing device described above can be implemented as a computer program. Such programs can be stored using various types of non-temporary computer-readable media and supplied to a computer. Non-temporary computer-readable media include various types of tangible recording media. Examples of non-temporary computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Programs may also be supplied to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.
[0036] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. [Explanation of Symbols]
[0037] 200 Work position identification system, 201 Imaging means, 202 Sound or vibration sensor, 203 Hand-to-hand distance estimation means, 204 Sound source estimation means, 205 Work position identification means
Claims
1. An imaging device for imaging workers, Sound or vibration sensors that detect work sounds are attached to both hands of the worker, A means for estimating the distance between the hands of the worker from images of both hands captured by the imaging means, A sound source estimation means that estimates the direction of the sound source of the work sound and the distance to the sound source of the work sound based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, A work location identification system comprising: a work location identification means that identifies a work location based on information obtained from the sound source estimation means and a set of pre-configured work location candidates.
2. Glove sensors attached to both hands of the worker, Sound or vibration sensors that detect work sounds are attached to both hands of the worker, A means for estimating the distance between the hands of the worker from the information acquired by the glove sensor, A sound source estimation means that estimates the direction of the sound source of the work sound and the distance to the sound source of the work sound based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, A work location identification system comprising: a work location identification means that identifies a work location based on information obtained from the sound source estimation means and a set of pre-configured work location candidates.
3. A work location identification system according to claim 1 or 2, wherein an imaging means captures an image of the work area and sets candidate work locations from the image of the work area.
4. The imaging device captures images of the worker, Sound or vibration sensors attached to both hands of the worker detect the work sounds, The distance between the hands is estimated from the images of the worker's hands captured by the imaging means. Based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, the direction of the sound source of the work sound and the distance to the sound source of the work sound are estimated. A method for identifying a work location, which identifies a work location based on the estimated direction and distance of the sound source and a set of pre-configured candidate work locations.
5. The imaging device captures images of the worker, Sound or vibration sensors attached to both hands of the worker detect the work sounds, The distance between the hands is estimated from the images of the worker's hands captured by the imaging means. Based on the information of the work sound obtained from the sound or vibration sensor and the distance between the hands, the direction of the sound source of the work sound and the distance to the sound source of the work sound are estimated. A work location identification program that causes an information processing device to identify a work location based on the estimated direction and distance of the sound source and a set of pre-configured work location candidates.