Information processing device

Abstract

To improve the efficiency of work management of a worker in assembly line production.SOLUTION: An information processing device includes: an acquisition unit configured to acquire multiple captured images of a work object moving along a transport path over time; and a control unit configured to derive, from the multiple captured images, working time during which a worker is in a work area around the work object while the work object is positioned at a reference position on the transport path and output information indicating the working time of the worker.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device. [Background technology]

[0002] In the assembly or repair of automobiles and other products, work objects are moved between work processes on an assembly line along a conveyor belt or other transport route, and workers assigned to each work process perform the work corresponding to the work process on the work objects. Various technologies have been proposed to support the management of such worker work. For example, Patent Document 1 discloses an invention that determines when a worker starts and finishes their work. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-125183 Summary of the Invention [Problem to be solved by the invention]

[0004] There is room for improvement in the efficiency of managing worker work.

[0005] The following describes a ride-on processing device and the like that can improve the management efficiency of workers' work in assembly line production. [Means for solving the problem]

[0006] The information processing device of the present disclosure has an acquisition unit that acquires multiple captured images of a work object moving on a transport path over time, and a control unit that derives, from the multiple captured images, the work time that a worker is included in the work area surrounding the work object while the work object is located at a reference position on the transport path, and outputs information indicating the work time of the worker. [Effects of the Invention]

[0007] According to the information processing device and the like of the present disclosure, it is possible to improve the efficiency of managing the work of workers in assembly line production. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an example of the configuration of a management system. [Figure 2] FIG. 10 is a flowchart illustrating an example of an operation procedure of the information processing device. [Figure 3A] FIG. 10 is a diagram illustrating an example of a captured image. [Figure 3B] FIG. 10 is a diagram illustrating an example of a captured image. [Figure 4] FIG. 10 is a diagram illustrating a display example of work time information. DETAILED DESCRIPTION OF THE INVENTION

[0009] The following describes the embodiments.

[0010] FIG. 1 is a diagram illustrating an example of the configuration of an assembly line production management system including an information processing device according to an embodiment. The management system 1 supports management of work by workers 16 on one or more work objects 14 moving along a conveyance path 13 in a factory, as shown in a schematic plan view. Multiple reference positions 15 (hereinafter, the two illustrated reference positions are distinguished by sub-numbers 15-1 and 15-2) corresponding to the assembly line processes are set along the conveyance path 13. The work objects 14 move along the conveyance path 13 sequentially from the reference positions 15-1 and 15-2 along a movement direction 17 indicated by an arrow. At each reference position 15, as the work objects 14 move, one or more workers 16 perform predetermined work on the work objects 14 corresponding to each process. The conveyance path 13 has a conveyance mechanism such as a belt conveyor. The work objects 14 are, for example, automobile bodies, chassis, etc. The work performed by the workers 16 includes assembling and connecting various parts. The management system 1 includes one or more information processing devices 10, terminal devices 11, and imaging devices 12, which are interconnected via a wired or wireless network such as an in-house local area network (LAN) so that they can communicate information with each other. The imaging device 12 is installed in a position where it can capture images of a work object 14 moving on a conveyance path 13 and a reference position 15 (either 15-1 or 15-2). The information processing device 10 sends information for managing the work of a worker 16 to the terminal device 11 based on the captured images sent from the imaging device 12. The terminal device 11 outputs the information sent from the information processing device 10 and presents it to the operator, allowing the operator to understand the status of the work of the worker 16.

[0011] The information processing device 10 includes a communication unit 101, which serves as an "acquisition unit" and acquires multiple captured images of the work object 14 moving on the transport path 13 over time. The control unit 103 derives, from the captured images, the work time during which the worker 16 is present in the work section surrounding the work object 14 while the work object 14 is positioned at a reference position 15 on the transport path 13, and outputs information indicating the work time of the worker 16 (hereinafter referred to as "work time information"). The state in which a portion of the work section, including the edge, overlaps with the reference position 15 corresponds to the state in which the work object 14 is positioned at the reference position 15. According to this embodiment, the terminal device 11 receives the work time information output by the information processing device 10 and outputs the work time information. This allows an operator to grasp the work time of each of the multiple workers 16, for example, and manage the work of each worker 16 in detail. This improves the efficiency of managing the work of workers in an assembly line production system.

[0012] Next, the configurations of the information processing device 10, the terminal device 11, and the imaging device 12 will be described.

[0013] The information processing device 10 has a communication unit 101, a storage unit 102, and a control unit 103. The information processing device 10 is, for example, one or more personal computers. When the information processing device 10 is configured to include multiple devices that can communicate with each other, the communication unit 101, the storage unit 102, or the control unit 103 are appropriately arranged in the multiple devices.

[0014] The communication unit 101 includes one or more interfaces for wired or wireless communication. The communication interface is, for example, a LAN interface, a dedicated line interface, etc. The communication unit 101 receives information used in the operation of the information processing device 10, and transmits information obtained by the operation of the information processing device 10. The information processing device 10 communicates information with the terminal device 11 and the imaging device 12 via the communication unit 101.

[0015] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these, that function as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable read only memory (EEPROM). The storage unit 102 stores information used in the operation of the information processing device 10 and information obtained by the operation of the information processing device 10.

[0016] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for specific processing. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the information processing device 10 while controlling each unit of the information processing device 10.

[0017] The functions of the information processing device 10 are realized by executing a control program on a processor included in the control unit 103. The control program is a program that causes a computer to execute processing of steps included in the operation of the information processing device 10, thereby causing the computer to realize functions corresponding to the processing of those steps. In other words, the control program is a program that causes a computer to function as the information processing device 10. Also, some or all of the functions of the information processing device 10 may be realized by a dedicated circuit included in the control unit 103. Also, the control program may be stored in a non-transitory recording / storage medium readable by the information processing device 10, and read by the information processing device 10 from the medium.

[0018] The terminal device 11 is, for example, a personal computer. The terminal device 11 has a configuration equivalent to the communication unit 101, the storage unit 102, and the control unit 103 of the information processing device 10, and also has an input interface that accepts information input by an operator's operation, and an output interface such as a display and a speaker that outputs various information to the operator.

[0019] The imaging device 12 has a camera that captures images using visible light and its control and communication circuits. The camera captures images over time, for example, at several tens of frames per second, and the captured images over time are sent to the information processing device 10. The camera has, for example, a wide-angle lens and is installed so as to overlook the work object 14 and the reference position 15 on the conveyance path 13.

[0020] The operation of the information processing device 10 will be described with reference to Figures 2, 3A, and 3B. Figure 2 is a flowchart showing an example of the operation procedure of the information processing device. Each step in Figure 2 is executed by the control unit 103. The procedure in Figure 2 is executed at an arbitrary period, for example, a period of several tens of microseconds to several seconds, when the transport path 13 is in operation. Figures 3A and 3B show examples of images captured over time by the imaging device 12.

[0021] 2, the control unit 103 acquires captured images over time from the imaging device 12. The control unit 103 acquires captured images of any number of frames, for example, two or more.

[0022] In step S102, the control unit 103 determines whether the leading edge of the work section has reached the reference position. The control unit 103 performs an arbitrary image recognition process on the captured images over time to detect the work object 14 and define a work section including the work object 14. The control unit 103 also defines a reference position 15 in the captured images based on information about the positional relationship between the imaging device 12 and the conveyance path 13, which is pre-stored in the memory unit 102. A landmark indicating the reference position 15 may be provided on the conveyance path 13 in advance, and the control unit 103 may detect the landmark to define the reference position 15. The control unit 103 also detects the movement of the work object 14 from changes in the position of the work object 14 in the captured images over time and determines whether the leading edge of the work section has reached the reference position. For example, as shown in captured image 30A in FIG. 3A, when the work object 14 moving along the conveyance path 13 in the movement direction 17 is detected, a work section 31 including the work object 14 is defined. The work section 31 is a generally rectangular section that includes a range of several tens to 200 centimeters around the work object 14 and has dimensions such that when a worker 16 is located inside the work section 31, there is a high probability that the worker 16 is performing work on the work object 14. The control unit 103 derives the dimensions of the work object 14 detected from the captured image using information such as the distance from the imaging device 12 to the conveyance path 13 and the angle of view of the imaging device 12, and determines the work section 31 relative to the work object 14 in the captured image 30A. The control unit 103 then determines whether the leading edge of the work section 31, i.e., the end of the work section 31 closer to the movement direction 17, has reached the reference position 15. If the work section 31 has reached the reference position 15 (Yes), the control unit 103 proceeds to step S104 and determines the start of the work process at the reference position 15. On the other hand, if the work section 31 has not reached the reference position 15 (No), the control unit 103 continues the determination in step S102.

[0023] In step S106, the control unit 103 detects and times the workers 16 within the work area 31. The worker detection unit 18 detects the workers 16 located within the work area 31 by detecting a characteristic image representing the workers 16. The characteristic image is, for example, a helmet worn by the workers 16. When multiple workers 16 are performing work in a single work process, each worker 16 wears a helmet of a different color in advance so as not to overlap with the other workers 16. In this way, the control unit 103 can detect and identify each worker 16 using the helmet and its color as a characteristic image. The characteristic image may be, for example, unique letters, symbols, etc. attached to the work uniform. The control unit 103 then times the time that each worker 16 is located within the work area 31. The time thus measured corresponds to the work time of each worker 16. If the worker 16 is not located within the work area 31, the control unit 103 does not time each worker 16.

[0024] In step S108, the control unit 103 determines whether the rear end of the work section 31 has reached the reference position 15. In step S108, as shown in the captured image 30B in FIG. 3B, if the rear end of the work section 31, that is, the side closer to the opposite direction of the movement direction 17 of the work target 14, has reached the reference position 15 (Yes), the control unit 103 proceeds to step S110, and if not (No), the control unit 103 executes steps S106 and S108 again. In the repeatedly executed processing cycle, the work time measured when step S106 is executed is accumulated for each worker 16 and stored in the memory unit 102.

[0025] In step S110, the control unit 103 determines that the work process at the reference position 15 has ended, and ends the timing for each worker 16 that began in step S106.

[0026] In step S112, the control unit 103 generates work time information indicating the work time of the worker 16. The work time information includes the cumulative work time of each worker 16 in the work process at the reference position 15.

[0027] In step S114, the control unit 103 outputs the work time information. For example, the control unit 103 sends the work time information to the terminal device 11. This allows the terminal device 11 to, for example, display the work time information and present it to the operator.

[0028] 2 for each of the work processes corresponding to the plurality of reference positions 15. The control unit 103 may generate work time information including the work time of each worker 16 for each of the work processes for the plurality of work processes.

[0029] FIG. 4 shows an example of work time information displayed by the terminal device 11. The axis represents work time, and the horizontal axis represents work processes and the workers 16 in each work process. The cumulative values ​​of work time for each worker 16 in each of the work processes "1st Cycle," "2nd Cycle," "3rd Cycle," and "4th Cycle" are shown as graphs with different hatching. By receiving such work time information, the operator can grasp the work time for each of the multiple workers 16 and manage the work of each worker 16 in detail. This improves the efficiency of managing the work of workers in assembly line production. Note that the display format of the work time information shown here is just one example, and the work time information may be displayed in other types of graphs, table formats, etc.

[0030] As described above, the control unit 103 is pre-configured to be able to recognize the shape of the work object 14, recognize that the leading end of the work area 31 has reached the reference position 15, recognize that the trailing end of the work area 31 has reached the reference position 15, and detect the worker 16 in the work area 31, by machine learning using a mathematical model such as a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network).

[0031] In step S106 in the modified example, the control unit 103 may detect a movement pattern of the worker 16 over time, for example, a hand movement pattern, from images captured over time, and measure the work time with the additional condition that the movement pattern of the worker 16 matches a pattern that is highly likely to indicate that the worker is working. Such movement patterns that are highly likely to indicate that the worker is working may be identified in advance and stored in the storage unit 102, and the control unit 103 may determine that the worker is working by comparing the movement pattern. Alternatively, the control unit 103 may determine that the worker is working using a model generated by machine learning that determines that the worker is working based on the movement pattern. This makes it possible to measure the work time more precisely.

[0032] According to the above-described embodiment, the operator can grasp the state of work of the worker 16, and the efficiency of managing the work of the worker in an assembly line production can be improved.

[0033] Although the embodiments have been described above based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means, step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined or divided into one. [Explanation of symbols]

[0034] 1. Work management system 10. Information processing equipment 11 Terminal equipment 12 Imaging device 13 Transport Route 14 Work Objects 15 Reference position 16 Workers 30A, 30B Captured images 31 Work Area 101 Communications Department 102 Storage section 103 Control Unit

Claims

1. an acquisition unit that acquires a plurality of images of a work object moving on a conveyance path over time; a control unit that derives, from the plurality of captured images, a working time during which the worker is in a working section around the work object while the work object is positioned at a reference position on the transport path, and outputs information indicating the working time of the worker; An information processing device having the above.

2. In claim 1, the work time is a time during which the control unit detects a characteristic image representing the worker from the work section in the captured image; Information processing device.

3. In claim 1, The control unit determines that the work object is located at the reference position when the work section overlaps with the reference position. Information processing device.

4. In claim 1, the control unit derives the work time for each of the plurality of workers and outputs information indicating the work time for each worker. Information processing device.

5. In claim 1, the control unit derives the work time on the condition that a predetermined movement pattern of the worker is detected from the plurality of captured images. Information processing device.

Images