Operation Analysis Device

Abstract

To improve the accuracy of work analysis.SOLUTION: A work analysis apparatus using machine learning includes: a movement detection unit which detects movement of a worker; a contact detection unit which detects a contact between the worker and an object; and a determination unit which determines whether the worker is working or not, on the basis of a result detected by the movement detection unit and a result detected by the contact detection unit.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates to a work analysis device.

Background Art

[0002] Techniques for analyzing the operations of workers are known. For example, Patent Document 1 discloses an invention that has a gyro sensor, an acceleration sensor, an azimuth sensor, and a magnetic force sensor, and determines which of the movement direction estimation by the gyro sensor or the movement direction estimation by the azimuth sensor can be more accurately estimated according to the value of the magnetic force measured by the magnetic force sensor in the estimation of the movement direction in the horizontal direction.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, the movement of the worker is analyzed without distinguishing whether the worker is in the middle of work. For example, when working on a large object such as an automobile, the worker repeats walking and transportation within one process, so even for the same work, an increase or decrease in the number of steps may affect the analysis result.

[0005] One aspect of this disclosure aims to improve the accuracy of work analysis in view of the above technical problems.

Means for Solving the Problems

[0006] A work analysis device according to one aspect of this disclosure includes a movement detection unit that detects the movement of a worker, a contact detection unit that detects the contact between the worker and an object, and a determination unit that determines whether the worker is in the middle of work based on the detection result by the movement detection unit and the detection result by the contact detection unit. [Effects of the Invention]

[0007] According to one aspect of this disclosure, the accuracy of work analysis is improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram illustrating the overview of the work analysis. [Figure 2] This is a block diagram showing an example of a work analysis device. [Figure 3] This flowchart shows an example of a work analysis method. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments of this disclosure will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0010] [Embodiment] In recent years, there has been a growing need to analyze the movements of workers on production lines from the perspective of quality assurance or improving production efficiency. For example, when working with large objects (workpieces) such as automobiles, workers repeatedly walk and work within a single process. Conventional technologies perform work analysis without separating walking and working, so even with the same net work, differences in walking distance cause variations in work information, which is one reason why the accuracy of work analysis decreases. In addition, when large movements occur due to the worker's walking, errors in sensor values ​​such as acceleration accumulate, causing a drift phenomenon where the analyzed position deviates from the actual position.

[0011] One embodiment of the present disclosure is a work analysis device for analyzing work performed by a worker. In this embodiment, the work analysis device determines whether or not a worker is performing work based on the detection results of the worker's movement and the detection results of contact between the worker and an object. This makes it possible to separate the worker into movement and work, improving the accuracy of the work analysis.

[0012] <Overview of work analysis> Figure 1 is a diagram illustrating the overview of work analysis. Figure 1(A) shows an example of the relationship between the estimated work procedure and the worker's movement history. In conventional work analysis, sensor data of a predetermined time length (e.g., 0.6 seconds) was used as the processing unit, skeletal information at each time point was detected from the sensor data, and the worker's work procedure at each time point was estimated based on the time change of the skeletal information. In addition, in conventional work analysis, the worker's position was detected from the sensor data, and the worker's movement history was calculated based on the time change of the worker's position. At this time, since the correct label did not include information indicating "movement," it was not possible to distinguish between movement and work.

[0013] In the work analysis of this embodiment, the worker's movement history is synchronized with the estimated work procedure, enabling the separation of movement and work. This allows for work analysis using only information during work, thus enabling work analysis without depending on the number of steps taken during movement. Furthermore, since sensor data such as acceleration can be reset for each task, the accumulation of errors can be suppressed, and drift phenomena can be avoided.

[0014] Figure 1(B) shows a specific example of a detailed work analysis. Figure 1(B) shows a worker taking out a part (image a), transporting the taken part (image b), tightening the part at the destination (image c), completing the tightening (image d), and moving it back to its original position (image e).

[0015] In the work analysis of this embodiment, a more detailed distinction is made by combining the detection results of worker movement and the detection results of contact between the worker and an object. When contact between the worker and an object is detected, it becomes possible to distinguish between simple "movement" and "transportation" while holding a workpiece. It also becomes possible to distinguish between fine movements such as picking up a part from a shelf and actual work.

[0016] For example, in image a, it is detected that there is no movement and an object is in contact with the hand. In this case, it can be determined that the operator is performing the "operation" of part removal. In image b, it is detected that there is movement and an object is in contact with the hand. In this case, it can be determined that the operator is performing the "transportation" of the part. In image c, it is detected that there is no movement and an object is in contact with the hand. In this case, it can be determined that the operator is performing the "operation" of tightening. In image d, it is detected that there is no movement and there is no object in contact with the hand. In this case, it can be determined that the operator has completed the "operation" of tightening. In image e, it is detected that there is movement and there is no object in contact with the hand. In this case, it can be determined that the operator is performing a simple "movement".

[0017] <Work analysis device> The work analysis device in this embodiment will be described while referring to FIG. 2. FIG. 2 is a block diagram showing an example of the work analysis device.

[0018] The work analysis device 100 in this embodiment is realized by an information processing device such as a computer. The information processing device includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), an input device, an output device, an external I / F (interface), and a communication I / F, etc., and each is interconnected by a system bus.

[0019] As shown in FIG. 2, the work analysis device 100 includes an acquisition unit 101, a position and orientation estimation unit 102, a movement detection unit 103, a contact detection unit 104, a work procedure estimation unit 105, and a determination unit 106. The work analysis device 100 is connected to the sensor C through various wired or wireless interfaces.

[0020] Sensor C is a variety of sensors that observe information about the operator. Sensor C may be installed at the work site or worn by the operator. As an example, Sensor C may be a process camera, a viewpoint camera, an acceleration sensor, a pressure sensor, a GPS module, or the like. The process camera is a camera that captures an overhead view of the work site. The viewpoint camera is a camera that is worn by the operator to capture the operator's field of vision. The acceleration sensor, the pressure sensor, or the GPS module is a wearable device worn by the operator.

[0021] The acquisition unit 101 acquires sensor data obtained by observing the operator from Sensor C. The acquisition unit 101 acquires sensor data at a predetermined time interval.

[0022] The position and orientation estimation unit 102 estimates the position and orientation of the operator based on the sensor data acquired by the acquisition unit 101. For example, the position and orientation estimation unit 102 may estimate the position and orientation of the operator by detecting the operator's skeleton from the image information captured by the process camera. The detection of the skeleton can be performed using a learned machine learning model. The position of the operator may be estimated using an acceleration sensor or a GPS module.

[0023] The movement detection unit 103 detects the movement of the operator based on the estimation result of the position and orientation by the position and orientation estimation unit 102. For example, the movement detection unit 103 may determine whether the operator is moving based on the temporal change in the position of the operator.

[0024] The contact detection unit 104 detects the contact between the operator and an object based on the sensor data acquired by the acquisition unit 101. For example, the contact detection unit 104 may recognize the operator and the object from the image information captured by the process camera and determine whether the operator's fingertips overlap with the object. The object may be, for example, a part to be worked on or a tool used in the work. The recognition of the operator and the object can be performed using a learned object detection model or a segmentation model.

[0025] The work procedure estimation unit 105 estimates the work procedure based on the position and orientation estimation results from the position and orientation estimation unit 102. For example, the work procedure estimation unit 105 may estimate the work procedure based on the time changes in the skeleton of the worker's upper body. The work procedure estimation can be performed using a trained machine learning model.

[0026] The determination unit 106 determines whether the worker is performing work or not based on the detection results from the movement detection unit 103 (movement information), the detection results from the contact detection unit 104 (hand information), and the estimation results from the work procedure estimation unit 105 (work information). For example, the determination unit 106 may differentiate between movement and work. Alternatively, for example, the determination unit 106 may differentiate between work, movement, or transportation.

[0027] <Work analysis method> The work analysis method performed by the work analysis device 100 in this embodiment will be described with reference to Figure 3. Figure 3 is a flowchart of an example of the work analysis method. The work analysis method is repeatedly performed on one unit of sensor data.

[0028] In step S1, sensor C observes the worker and generates sensor data. The acquisition unit 101 acquires the sensor data from sensor C. The acquisition unit 101 sends the acquired sensor data to the position and attitude estimation unit 102 and the contact detection unit 104.

[0029] In step S2, the position and orientation estimation unit 102 estimates the worker's position and orientation based on the sensor data received from the acquisition unit 101. The position and orientation estimation unit 102 sends the position and orientation estimation result to the movement detection unit 103 and the work procedure estimation unit 105.

[0030] In step S3, the movement detection unit 103 receives the position and orientation estimation result from the position and orientation estimation unit 102. The movement detection unit 103 detects the worker's movement based on the time change of the worker's position. The movement detection unit 103 sends movement information indicating the detection result to the determination unit 106.

[0031] In step S4, the contact detection unit 104 recognizes the worker's hand and the object based on the sensor data received from the acquisition unit 101, and detects contact between the worker's hand and the object. The contact detection unit 104 sends hand information indicating the detection result to the determination unit 106.

[0032] In step S5, the work procedure estimation unit 105 estimates the work procedure based on the sensor data received from the acquisition unit 101. The work procedure estimation unit 105 sends work information indicating the estimation result to the determination unit 106.

[0033] In step S6, the determination unit 106 determines whether the first determination condition is met based on the movement information, end-effector information, and work information. The first determination condition is that the movement information indicates no movement, the work information indicates work is being performed, and the end-effector information indicates contact with an object.

[0034] If the first determination condition is met (YES), the determination unit 106 proceeds to step S7. On the other hand, if the first determination condition is not met (NO), the determination unit 106 proceeds to step S8.

[0035] In step S7, the determination unit 106 determines that it is "work". After that, the determination unit 106 returns to step S1.

[0036] In step S8, the determination unit 106 determines whether the second determination condition is met based on the movement information, end-effector information, and work information. The second determination condition is that the movement information indicates movement, the work information indicates no work, and the end-effector information indicates contact with an object.

[0037] If the second determination condition is met (YES), the determination unit 106 proceeds to step S9. On the other hand, if the second determination condition is not met (NO), the determination unit 106 proceeds to step S10.

[0038] In step S9, the determination unit 106 determines that "transportation" is required. The determination unit 106 then returns to step S1.

[0039] In step S10, the determination unit 106 determines whether the third determination condition is met based on the movement information, end-effector information, and work information. The third determination condition is that the movement information indicates movement, the work information indicates no work, and the end-effector information indicates no contact with an object.

[0040] If the third determination condition is met (YES), the determination unit 106 proceeds to step S11. On the other hand, if the third determination condition is not met (NO), the determination unit 106 proceeds to step S12.

[0041] In step S11, the determination unit 106 determines that "movement" has occurred. After that, the determination unit 106 returns to step S1.

[0042] In step S12, the determination unit 106 of the work analysis device 100 determines that it is a "dangerous operation." This is because it is considered that the worker is moving while performing the work. After that, the determination unit 106 returns to step S1.

[0043] <Application Examples> This embodiment can be applied to the following processes, for example: Firstly, all general work performed outside the vehicle. Secondly, the process of taking parts from shelves and assembling them. Thirdly, the process of transporting large workpieces using locker moldings and assembling them into the vehicle. Fourthly, the process of taking out and transporting workpieces after injection molding. Fifthly, the detection of multitasking work.

[0044] <Effects> The work analysis device in this embodiment determines whether a worker is performing work or not based on the detection results of worker movement and contact between the worker and an object. Therefore, according to this embodiment, movement and work can be separated. In one aspect, according to this embodiment, the accuracy of work analysis is improved.

[0045] Although embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications or changes are possible within the scope of the gist of the present invention as described in the claims. [Explanation of symbols]

[0046] 100: Work analysis device 101: Acquisition unit 102: Position and orientation estimation unit 103: Movement detection unit 104: Contact detection unit 105: Work procedure estimation unit 106: Judgment unit

Claims

[Claim 1] A movement detection unit that detects the movement of the worker, A contact detection unit that detects contact between the worker and an object, A determination unit determines whether the worker is performing work or not based on the detection results from the movement detection unit and the detection results from the contact detection unit, A work analysis device equipped with the following features.

Images