Teacher Data Collection Device

Abstract

To provide a training data collection apparatus capable of collecting a large amount of training data.SOLUTION: A training data collection apparatus includes a processor configured to: acquire video obtained by imaging a worker assembling a component to a workpiece so that the worker, the component, and the workpiece may be located in a predetermined region at start of the work; extract position information of the hand of the worker, the component, and the workpiece; determine whether the work has been conducted in a predetermined procedure, on the basis of the position information; and store the video and a result determined on the video, in association with each other, as training data for use in machine learning.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a teacher data collection device.

Background Art

[0002] Patent Document 1 discloses a technique for automatically generating teacher data used in machine learning for generating a learned model for detecting abnormalities in a factory. In this technique, information (such as weight, etc.) necessary for determining the abnormality of an object is acquired together with a video, and the video and the determination result are linked to be used as teacher data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in an actual factory, the probability of occurrence of defective products is extremely low, and it has not been possible to collect a sufficient amount of teacher data.

[0005] The present disclosure has been made in view of the above, and an object thereof is to provide a teacher data collection device capable of collecting a large amount of teacher data.

Means for Solving the Problems

[0006] The teacher data collection device according to the present disclosure acquires a video of an operator assembling parts onto a workpiece, the video being captured with the operator, the parts, and the workpiece arranged and captured so as to be within a predetermined area at the start of the operation. The device extracts the position information of the operator's hand, the parts, and the workpiece from the video, determines whether the operation was performed in a predetermined procedure based on the position information, and stores, in association with the video and the determination result for the video, the data as teacher data for use in machine learning, and is provided with a processor configured to perform the above.

Effect of the Invention

[0007] According to the present disclosure, it is possible to realize a teacher data collection device capable of collecting a large amount of teacher data.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Mode for Carrying Out the Invention

[0009] The teacher data collection device according to the embodiment of the present disclosure will be described with reference to the drawings. Note that the components in the following embodiments include those that can be replaced by those skilled in the art and are easy to replace, or those that are substantially the same.

[0010] (Teacher Data Collection Device) The teacher data collection device is a device that collects teacher data for machine learning to generate a learned model for determining whether an operator assembles parts to a workpiece in an appropriate procedure in a factory or the like. The teacher data is a data group that associates a video in which the state of work is imaged with a determination result as to whether the work in the video is performed in an appropriate procedure. By using this teacher data for AI to perform machine learning, a learned model for determining whether the work in the imaged video is performed in an appropriate procedure can be generated.

[0011] The teacher data collection device can collect a large amount of teacher data by collecting videos in which the state of work is imaged from a number of factories.

[0012] Various specific configurations of the teacher data collection device are conceivable. Examples of the teacher data collection device include a server system, a personal computer, a workstation, a mobile phone, a smartphone, a tablet terminal device, a vehicle, or other machine tools. Further, the teacher data collection device may be equipped with a learned model composed of, for example, a learning result by deep learning or other machine learning inside the configuration listed above.

[0013] As shown in FIG. 1, the teacher data collection device 1 includes a control unit 11, a storage unit 12, an acquisition unit 13, and a display unit 14.

[0014] The control unit 11 is realized by a processor such as a CPU (Central Processing Unit) and a memory (main storage unit) such as a RAM (Random Access Memory) and a ROM (Read Only Memory).

[0015] The control unit 11 loads a program into the working area of the main memory unit and executes it, and controls each component unit and the like through the execution of the program, thereby realizing a function that meets a predetermined purpose. The control unit 11 functions as the extraction unit 111 and the determination unit 112 through the execution of the above-described program.

[0016] Note that in FIG. 1, an example is shown in which the functions of each part of the control unit 11 are realized by one device (for example, a computer or the like). However, for example, the functions of each part may be realized by a plurality of devices. That is, among the control unit 11, the extraction unit 111 and the determination unit 112 may be realized by different devices, and a configuration in which these devices are connected by a network may be employed.

[0017] The extraction unit 111 extracts the position information of the hand, parts, and workpiece of the operator from a video obtained by imaging the operation of the operator assembling the parts onto the workpiece.

[0018] The determination unit 112 determines whether the operation has been performed in a predetermined procedure based on the position information of the hand, parts, and workpiece of the operator extracted by the extraction unit 111.

[0019] The storage unit 12 is realized by a recording medium such as an EPROM (Erasable Programmable ROM), a hard disk drive (HDD), and a removable medium. Examples of the removable medium include disk recording media such as a USB (Universal Serial Bus) memory, a CD (Compact Disc), a DVD (Digital Versatile Disc), and a BD (Blu-ray (registered trademark) Disc).

[0020] The storage unit 12 can store an operating system (OS), various programs, various tables, various databases, etc. The storage unit 12 stores teacher data 121. The teacher data is data that associates a video with the determination result of the determination unit 112 for that video. In addition to these, the storage unit 12 may store the processing results in the control unit 11, etc., as necessary. Further, the storage unit 12 may store a learned model generated using the teacher data.

[0021] The acquisition unit 13 is composed of, for example, a communication module capable of receiving various information. The acquisition unit 13 communicates with external devices, etc. through, for example, a network (not shown) and receives various information. The acquisition unit 13 acquires a video of an operator assembling parts onto a workpiece, where the video is taken with the operator, the parts, and the workpiece arranged to be reflected in a predetermined area at the start of the operation.

[0022] The display unit 14 is composed of, for example, a liquid crystal display or an organic EL display. The display unit 14 displays, for example, the processing results in the control unit 11 and other various information.

[0023] (Teacher Data Collection Method) An example of the processing procedure of the teacher data collection method executed by the teacher data collection device according to the embodiment will be described with reference to FIG. 2. FIG. 3 is a diagram for explaining the state of the operation. As shown in FIG. 3, in the image Im taken at the start of the operation, the operator O, parts P1 to P4, and the workpiece W are arranged to be reflected in a predetermined area of the image Im. In other words, the viewing angle of the camera, the position of the operator O, parts P1 to P4, and the workpiece W at the start of the operation are adjusted so that the image Im shown in FIG. 3 is taken. And it is assumed that the procedure of the operator O assembling parts P1 to P4 onto the workpiece W is defined in the order of part P1, part P2, part P3, part P4.

[0024] The image Im is, for example, an image captured by an overhead camera mounted on the ceiling of a factory. However, as long as the areas in the image where the worker, the workpiece, and the parts are reflected are defined, the direction and the angle of view for capturing the image are not particularly limited. However, when collecting videos from multiple factories or multiple production lines, it is required that the worker O, the parts P1 to P4, and the workpiece W be arranged so as to be reflected in a predetermined area of the image Im in each factory and each production line.

[0025] The parts P1 to P4 and the workpiece W are not particularly limited. That is, the parts P1 to P4 and the workpiece W may be different for each video capturing the operation, and the finished products may also be different.

[0026] First, the acquisition unit 13 acquires a video capturing the operation of the worker O assembling the parts P1 to P4 onto the workpiece W (step S1). In this video, as shown in FIG. 3, the worker O, the parts P1 to P4, and the workpiece W are arranged in advance so as to be reflected in a predetermined area at the start of the operation.

[0027] Subsequently, the extraction unit 111 extracts the position information of the hand of the worker O, the parts P1 to P4, and the workpiece W from the video capturing the operation of the worker O assembling the parts P1 to P4 onto the workpiece W (step S2).

[0028] Then, the determination unit 112 determines whether the operation has been performed in a predetermined procedure based on the position information of the hand of the worker O, the parts P1 to P4, and the workpiece W extracted by the extraction unit 111 (step S3). Specifically, the determination unit 112 determines whether the worker O has assembled the parts P1, P2, P3, and P4 onto the workpiece W in this order based on the temporal change of each position information in the video. At this time, if the assembly order is incorrect or if the process of assembling any of the parts P1 to P4 is missing, the determination unit 112 determines that the operation has not been performed in a predetermined procedure.

[0029] Thereafter, the storage unit 12 stores the training data associating the video with the determination result of the determination unit 112 for that video (step S4).

[0030] In the teacher data collection device according to the embodiment described above, the operator O, parts P1 to P4, and the workpiece W shown in the video are determined in advance in terms of the arrangement and work procedure at the start of work, and a video of the work is captured. As a result, even for videos capturing different operations of parts, workpieces, or finished products, it can be used as teaching data for machine learning to generate a learned model for determining whether the work was performed in an appropriate procedure. And according to the teacher data collection device 1, by collecting a large number of videos from a large number of factories using different parts, workpieces, or finished products, a larger amount of teaching data can be collected than when collecting videos from factories using the same parts, workpieces, and finished products.

[0031] Then, by performing machine learning using the large amount of teaching data collected by the teacher data collection device 1, a learned model for determining whether the work was performed in an appropriate procedure can be generated. When this learned model is used, it is also possible to notify the operator of an alert when the operator makes a mistake in the work order or skips a process.

[0032] Further effects and modifications can be easily derived by those skilled in the art. Therefore, a broader aspect of the present invention is not limited to the specific details and representative embodiments expressed and described as above. Accordingly, various changes can be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

Explanation of Reference Numerals

[0033] 1 Teacher data collection device 11 Control unit 111 Extraction unit 112 Determination unit 12 Storage unit 121 Teacher data 13 Acquisition unit 14 Display unit

Claims

Claim 1 A video obtained by imaging an operation in which an operator assembles a component to a workpiece, the video being captured with the operator, the component, and the workpiece arranged so as to be reflected in a predetermined area at the start of the operation, extracting position information of the hand of the operator, the component, and the workpiece from the video, determining whether the operation has been performed in a predetermined procedure based on the position information, a processor configured to store, in association with the video and a determination result for the video, as teaching data for machine learning A teaching data collection device comprising.

Images