Information processing apparatus

Abstract

The information processing apparatus includes: an acquisition unit configured to acquire a plurality of captured images of a work object moving on a conveyance path over time; and a control unit that derives, from the plurality of captured images, a work time that includes a worker in a work section around the work object during a period in which the work object is located at a reference position on the conveyance path, and outputs information indicating the work time of the worker.

Description

Technical Field

[0001] The present disclosure relates to an information processing device. Background Art

[0002] During the assembly or repair of automobiles and other equipment, objects are moved between work processes in an assembly line along conveyor paths, such as belt conveyors. Workers assigned to each work process perform the work on the objects corresponding to the work process. Various technologies have been proposed to assist in the management of these workers' work. For example, Japanese Patent Application Laid-Open No. 2021-125183 discloses an invention for determining when a worker's work has begun and ended.

[0003] There is room for improvement in the efficiency of operator management of work. Summary of the Invention

[0004] Hereinafter, an information processing device and the like are disclosed that can improve the efficiency of work management of workers in an assembly line operation.

[0005] The information processing device disclosed in the present invention includes: an acquisition unit that acquires multiple captured images of a work object moving on a conveying path over time; and a control unit that derives, from the multiple captured images, a work time in which a work partition around the work object includes an operator while the work object is located at a reference position on the conveying path, and outputs information representing the work time of the operator.

[0006] According to the information processing device and the like in the present disclosure, it is possible to improve the efficiency of management of work performed by workers in a production line operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, wherein like reference numerals denote like elements, and wherein:

[0008] Figure 1 This is a diagram showing a configuration example of a management system.

[0009] Figure 2 This is a flowchart showing an example of the operation procedure of the information processing device.

[0010] Figure 3A It is a diagram showing an example of a captured image.

[0011] Figure 3B It is a diagram showing an example of a captured image.

[0012] Figure 4 It is a diagram showing a display example of work time information. DETAILED DESCRIPTION

[0013] Hereinafter, embodiments will be described.

[0014] Figure 1 This diagram illustrates an example configuration of a management system for an assembly line operation in one embodiment, wherein the management system includes an information processing device. Management system 1 assists operators 16 in managing operations on one or more work objects 14, wherein the one or more work objects 14 move along a conveyor path 13 within a factory, shown in a generally top-down view. Conveyor path 13 is provided with multiple reference positions 15 corresponding to the processes of the assembly line operation (hereinafter, the two reference positions shown as an example are subdivided into numbers 15-1 and 15-2). Work objects 14 move along conveyor path 13 in a sequential order, one at each reference position 15-1 and one at each reference position 15-2, along movement directions 17 indicated by arrows. At each reference position 15, when work objects 14 move to the conveyor path, one or more operators 16 perform predetermined operations on the work objects 14 corresponding to the respective processes. Conveyor path 13 includes, for example, a conveying mechanism such as a belt conveyor. Work objects 14 are, for example, automobile bodies or chassis. The work performed by the operator 16 is the assembly and connection of various parts. The management system 1 has one or more information processing devices 10, terminal devices 11 and camera devices 12, wherein the information processing device 10, terminal device 11 and camera device 12 are connected to each other through a wired network or wireless network such as a LAN (Local Area Network) within the factory so as to be able to communicate information with each other. The camera device 12 is set at a position where it can capture the work object 14 and the reference position 15 (either 15-1 or 15-2) moving on the conveying path 13. The information processing device 10 sends information for managing the work of the operator 16 to the terminal device 11 based on the captured image sent from the camera device 12. The terminal device 11 outputs the information sent from the information processing device 10 and presents it to the operator, so that the operator can understand the status of the work of the operator 16.

[0015] The information processing device 10 includes a communication unit 101, serving as an "acquisition unit," which acquires multiple captured images of a work object 14 moving along a conveyor path 13 over time. A control unit 103 derives, from the multiple captured images, the working time of a work section surrounding the work object 14 that includes a worker 16 while the work object 14 is at a reference position 15 on the conveyor path 13, and outputs information indicating the working time of the worker 16 (hereinafter referred to as "working time information"). The state in which a portion of the working section, including its ends, overlaps with the reference position 15 corresponds to the state in which the work object 14 is at the reference position 15. According to this embodiment, the terminal device 11 receives the working time information output by the information processing device 10 and outputs the working time information. This allows an operator to, for example, grasp the working time associated with each of the multiple workers 16 and finely manage the work of each worker 16. Consequently, the efficiency of managing the work of workers in an assembly line operation can be improved.

[0016] Next, the configurations of the information processing device 10 , the terminal device 11 , and the imaging device 12 will be described.

[0017] The information processing device 10 includes a communication unit 101, a storage unit 102, and a control unit 103. The information processing device 10 is, for example, one or more personal computers. If the information processing device 10 is configured to include multiple devices capable of communicating with each other, the communication unit 101, the storage unit 102, or the control unit 103 is appropriately configured in the multiple devices.

[0018] The communication unit 101 includes one or more wired or wireless communication interfaces. Examples of communication interfaces include a LAN interface and a dedicated line interface. The communication unit 101 receives information used for the operation of the information processing device 10 and transmits information generated by the operation of the information processing device 10. The information processing device 10 communicates information with the terminal device 11 and the imaging device 12 via the communication unit 101.

[0019] The storage unit 102 includes, for example, a main storage device, an auxiliary storage device, or one or more semiconductor memories functioning as cache memory, one or more magnetic memories, one or more optical memories, or a combination of at least two of these memories. Semiconductor memories are, for example, RAM (Random Access Memory) or ROM (Read Only Memory). RAM is, for example, SRAM (Static RAM) or DRAM (Dynamic RAM). ROM is, for example, EEPROM (Electrically Erasable Programmable ROM). The storage unit 102 stores information used for the operation of the information processing device 10 and information obtained through the operation of the information processing device 10.

[0020] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. Examples of processors include general-purpose processors such as CPUs (Central Processing Units) or specialized processors such as GPUs (Graphics Processing Units) dedicated to specific processing. Examples of dedicated circuits include FPGAs (Field-Programmable Gate Arrays) and ASICs (Application-Specific Integrated Circuits). The control unit 103 controls the various components of the information processing device 10 while executing information processing related to the operation of the information processing device 10.

[0021] The functions of the information processing device 10 are realized by the processor included in the control unit 103 executing a control program. The control program is a program for causing the computer to perform the steps included in the operation of the information processing device 10, thereby causing the computer to realize the functions corresponding to the steps. In other words, the control program is a program for causing the computer to function as the information processing device 10. Furthermore, some or all of the functions of the information processing device 10 may be realized by dedicated circuits included in the control unit 103. Alternatively, the control program may be stored on a non-transitory recording / storage medium readable by the information processing device 10, and the information processing device 10 may read the control program from the medium.

[0022] The terminal device 11 is, for example, a personal computer. It has the same configuration as the communication unit 101, storage unit 102, and control unit 103 of the information processing device 10, and includes an input interface for receiving information input based on operator operations and an output interface such as a display and speaker for outputting various information to the operator.

[0023] The imaging device 12 includes a camera that captures images using visible light and its control / communication circuit. The camera captures images at a rate of, for example, several dozen frames per second over time, and transmits the captured images to the information processing device 10. The camera, for example, includes a wide-angle lens and is positioned to overlook the work object 14 and reference position 15 on the conveyance path 13.

[0024] use Figure 2 、 Figure 3A 、 Figure 3B , the operation of the information processing device 10 is explained. Figure 2 This is a flowchart showing an example of the operation procedure of the information processing device. Figure 2 Each step is executed by the control unit 103. Figure 2 The process is executed at an arbitrary cycle, for example, a cycle of several tens of milliseconds to several seconds, when operating on the conveying path 13 . Figure 3A 、 Figure 3B An example of an image captured over time by the imaging device 12 is shown.

[0025] exist Figure 2 In step S100, the control unit 103 acquires captured images over time from the imaging device 12. The control unit 103 acquires, for example, captured images of an arbitrary number of frames, two or more.

[0026] In step S102, the control unit 103 determines whether the front end of the work partition has reached the reference position. The control unit 103 performs arbitrary image recognition processing on the time-lapsed captured image to detect the work object 14, and delineates the work partition including the work object 14. In addition, the control unit 103 delineates the reference position 15 in the captured image based on the information on the positional relationship between the imaging device 12 and the conveying path 13 pre-stored in the storage unit 102. Alternatively, a target object representing the reference position 15 is pre-set on the conveying path 13, and the control unit 103 delineates the reference position 15 by detecting the target object. Moreover, the control unit 103 detects the movement of the work object 14 based on the change in the position of the work object 14 in the time-lapsed captured image, thereby determining whether the front end of the work partition has reached the reference position. For example, Figure 3AAs shown in captured image 30A, when a work object 14 moving along conveyor path 13 in moving direction 17 is detected, a work area 31 is defined that includes work object 14. Work area 31 is a generally rectangular area that encompasses a size range that increases the likelihood that worker 16 is working on work object 14 when located inside work area 31. For example, it encompasses a range of several tens to 200 centimeters around work object 14. The control unit 103 uses information such as the distance from imaging device 12 to conveyor path 13 and the field of view of imaging device 12 to derive the size of work object 14 detected from the captured image and defines work area 31 for work object 14 in captured image 30A. The control unit 103 then determines whether the leading end of work area 31, that is, the end of work area 31 closest to moving direction 17, has reached reference position 15. If the leading end of the work section 31 has reached the reference position 15 (Yes), the control unit 103 proceeds to step S104 to determine the start of the work process at the reference position 15. On the other hand, if the leading end of the work section 31 has not reached the reference position 15 (No), the control unit 103 continues the determination in step S102.

[0027] In step S106, the control unit 103 detects the worker 16 within the work zone 31 and measures time. The control unit 103 detects the worker 16 within the work zone 31 by detecting a characteristic image representing the worker 16. An example of a characteristic image is the helmet worn by the worker 16. When multiple workers 16 perform work in a single operation, each worker 16 wears a helmet of a different color, so that it does not overlap with other workers 16. This allows the control unit 103 to detect and distinguish each worker 16 using the helmet and its color as a characteristic image. It should be noted that the characteristic image may also be, for example, unique text or symbols marked on work clothes. Furthermore, the control unit 103 measures the time each worker 16 spends within the work zone 31. This measured time corresponds to the working time of each worker 16. It should be noted that if a worker 16 is not within the work zone 31, the control unit 103 does not measure time for each worker 16.

[0028] In step S108, the control unit 103 determines whether the rear end of the work section 31 has reached the reference position 15. In step S108, if Figure 3BAs shown in the captured image 30B, if the rear end of the work area 31, that is, the side of the work object 14 in the direction opposite to the moving direction 17, has reached the reference position 15 (Yes), the control unit 103 proceeds to step S110. If not (No), steps S106 and S108 are executed again. In this repetitive processing loop, the work time measured when step S106 is executed is accumulated for each worker 16 and stored in the storage unit 102.

[0029] In step S110 , the control unit 103 determines that the work process at the reference position 15 is completed, and ends the timing for each worker 16 started in step S106 .

[0030] In step S112, the control unit 103 generates work time information indicating the work time of the worker 16. The work time information includes the accumulated work time of each worker 16 in the work process at the reference position 15.

[0031] In step S114, the control unit 103 outputs the work time information. For example, the control unit 103 transmits the work time information to the terminal device 11. In this way, the terminal device 11 can display the work time information to present it to the operator.

[0032] The information processing device 10 can execute the operation for each of the work steps corresponding to the plurality of reference positions 15. Figure 2 The control unit 103 may also generate operation time information including the operation time of each operator 16 for each operation step for a plurality of operation steps.

[0033] Figure 4 This is an example of the operation time information displayed by the terminal device 11. The vertical axis represents the operation time, the horizontal axis represents the operation process and the operator 16 in each operation process, and the cumulative value of the operation time of each operator 16 in the operation process "first cycle", "second cycle", "third cycle" and "fourth cycle" is represented by a chart with different shades. By accepting the presentation of such operation time information, the operator can grasp the operation time related to each of the multiple operators 16 and can manage the operation of each operator 16 in detail. Therefore, the management efficiency of the operation of the operators in the assembly line operation can be improved. It should be noted that, Figure 4 The display mode of the work time information shown is an example, and the work time information may be displayed in other types of graphs, tables, etc.

[0034] In the above, the control unit 103 is pre-configured to be able to perform, for example, recognition of the shape of the work object 14, recognition that the front end of the work partition 31 has reached the reference position 15, recognition that the rear end of the work partition 31 has reached the reference position 15, and detection of the operator 16 in the work partition 31 through machine learning using mathematical models such as CNN (Convolutional Neural Network) or RNN (Recurrent Neural Network).

[0035] In step S106 of the modified example, the control unit 103 may also detect the operator 16's motion pattern over time, such as the pattern of hand movements, from the captured images over time, and use the consistency of the operator 16's motion pattern with a high probability of being in the work as a further condition to measure the working time. By predetermining the motion pattern with a high probability of being in the work and storing it in the storage unit 102, and by having the control unit 103 compare the motion pattern, it is possible to determine whether the operator is in the work. Alternatively, the control unit 103 may use a model generated by machine learning that determines whether the operator is in the work based on the motion pattern to determine whether the operator is in the work. In this way, the working time can be measured more accurately.

[0036] According to the above embodiment, the operator can grasp the status of the work of the worker 16, and the management efficiency of the work of the worker in the production line operation can be improved.

[0037] While the embodiments have been described above based on the accompanying drawings and examples, it should be noted that those skilled in the art can easily make various variations and modifications based on the present disclosure. Therefore, it should be noted that such variations and modifications are within the scope of the present disclosure. For example, the functions included in each component, step, etc. may be reconfigured in a manner that does not logically conflict, and multiple components, steps, etc. may be combined into one, or they may be divided.

Claims

1. An information processing device comprising: an acquisition unit that acquires a plurality of captured images of a work object moving on a conveyance path over time; and The control unit derives, from the plurality of captured images, a working time during which a worker is included in a working section around the work object while the work object is located at a reference position on the transport path, and outputs information indicating the working time of the worker.

2. The information processing device according to claim 1, wherein The working time is the time when the control unit detects a characteristic image representing the worker from the working section in the captured image.

3. The information processing device according to claim 1, wherein When the work section overlaps with the reference position, the control unit determines that the work object is located at the reference position.

4. The information processing device according to claim 1, wherein: The control unit derives the working time for each of the plurality of workers and outputs information indicating the working time for each worker. The information processing apparatus according to claim 1 , wherein: The control unit derives the working time on the condition that a predetermined movement pattern of the worker is detected from the plurality of captured images.

Images