Information processing device, luminaire, and information processing system

Abstract

To analyze the movement of a worker in working with high accuracy.SOLUTION: An information processing device includes: a storage unit that stores an image captured by an image capturing unit; a portion detection unit that detects a position of a specific portion of a body of a worker from the image stored in the storage unit; an object detection unit that detects an object from the image; and a work analysis unit that analyzes work information of the worker based on the detected object and the position of the specific portion.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device, a lighting device, and an information processing system. [Background technology]

[0002] Conventionally, a technology has been proposed in which an image captured by an imaging unit is acquired by an information processing device, the position of a specific body part of a worker is detected, and if the position of the specific body part is located within a work area, it is determined that the worker is performing useful work, and the working time of the useful work is recorded, and the working time when the worker completes one unit task is recorded. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-156279 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has room for further improvement in terms of analyzing the movements of a worker while working with higher accuracy.

[0005] An object of the present invention is to provide an information processing device, a lighting device, and an information processing system that are capable of analyzing the movements of a worker while working with high accuracy. [Means for solving the problem]

[0006] An information processing device according to an embodiment has a memory unit that stores images captured by a photographing unit, a body part detection unit that detects the position of a specific body part of a worker from the image stored in the memory unit, an object detection unit that detects an object from the image, and a work analysis unit that analyzes work information of the worker based on the detected object and the position of the specific body part. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a perspective view showing an example of the appearance of a lighting device included in an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an overview of an information processing system according to the embodiment. [Diagram 3] FIG. 3 is a block diagram showing the configuration of the information processing device according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing the processing procedure of the processing executed in the information processing device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The information processing device 20 described below has a memory unit 22 that stores images captured by an imaging unit (camera 16), a body part detection unit (first detection unit 232) that detects the position of a specific body part of the worker from the image stored in the memory unit 22, an object detection unit (second detection unit 233) that detects an object from the image, and a work analysis unit 234 that analyzes work information of the worker based on the detected object and the position of the specific body part.

[0009] The object detection unit described below detects the amount of objects. The task analysis unit 234 analyzes the task information based on the amount of objects.

[0010] The object detection unit described below detects the movement trajectory of the object. The task analysis unit 234 analyzes the task information based on the movement trajectory of the object.

[0011] The work analysis unit 234 described below analyzes the work information based on the state in which the worker is holding the object.

[0012] The object detection unit described below detects products and parts used in the products as objects. The work analysis unit 234 analyzes the work information based on the products and parts.

[0013] The body part detection unit described below detects the position of a specific body part based on the result of detection of the worker's skeleton.

[0014] The information processing method described below includes a storage process for storing the image captured by the imaging unit in the memory unit 22, a part detection process for detecting the positions of specific parts of the worker's body from the image stored in the memory unit 22, an object detection process for detecting an object from the image, and a work analysis process for analyzing the work information of the worker based on the detected object and the positions of the specific parts.

[0015] The information processing system 1 described below comprises a lighting device 100 having an imaging unit that captures images, a memory unit 22 that stores the images captured by the imaging unit, a body part detection unit that detects the positions of specific body parts of the worker from the images stored in the memory unit 22, an object detection unit that detects objects from the images, and a work analysis unit that analyzes work information of the worker based on the detected objects and the positions of the specific body parts.

[0016] Hereinafter, an information processing device, a lighting device, and an information processing system according to embodiments will be described with reference to the drawings. In the embodiments, the same components are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0017] [Lighting equipment] First, the configuration of a lighting device 10 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a perspective view showing an example of the appearance of the lighting device 10 included in an information processing system 1 according to an embodiment.

[0018] 1, the lighting device 10 includes a lighting section 11, an imaging unit 12, and a main body section 13. The lighting device 10 is, for example, a ceiling-mounted lighting device in which the main body section 13 is installed on a ceiling surface or the like, and light output from the lighting section 11 is irradiated onto a floor surface. The lighting device 10 is mainly used indoors, for example, for purposes such as monitoring a production line in a factory or monitoring the status of employees in an office.

[0019] For ease of understanding, Fig. 1 illustrates a three-dimensional Cartesian coordinate system including a Z axis with a positive direction being vertically downward and a negative direction being vertically upward. The X axis is illustrated along the length direction of the lighting device 10, and the Y axis is illustrated along the width direction of the lighting device 10.

[0020] The lighting unit 11 is an illumination bar having a plurality of light-emitting elements (not shown) arranged at predetermined intervals on a long chassis or substrate (not shown) arranged along the Y-axis direction, and a diffusion cover 14 is provided on the floor side, i.e., the positive Z-axis side, so that the light-emitting elements are housed between the chassis and the diffusion cover 14.

[0021] The diffusion cover 14 is made of a light-transmitting material such as acrylic or polycarbonate. The diffusion cover 14 is frosted to diffuse the light emitted from the multiple light-emitting elements. The diffusion cover 14 may contain a suitable diffusing material or coloring agent.

[0022] The imaging unit 12 has a light-shielding cover 15 and a camera 16. The camera 16 is an example of an imaging section. The imaging units 12 are arranged next to each other on the negative side of the X-axis of the illumination section 11.

[0023] The main body 13 holds the illumination section 11 and the imaging unit 12. The main body 13 also serves as a mounting member for mounting the illumination device 10 at a predetermined position such as a ceiling.

[0024] The light-shielding cover 15 is disposed on the positive Z-axis direction side of the main body 13 so as to cover the camera 16 between the light-shielding cover 15 and the main body 13. The light-shielding cover 15 also has a through-hole provided at a position facing the lens of the camera 16. By disposing such a light-shielding cover 15, it is possible to make it difficult for the light irradiated from the diffusion cover 14 of the illumination unit 11 to enter the lens of the camera 16.

[0025] In this way, the lighting device 10 of the present disclosure includes the imaging unit 12, and thus can automatically set, for example, brightness and color conditions that are most suitable for the imaging conditions of the imaging unit 12. In addition, as in the present disclosure, the imaging unit 12 and the illumination section 11 can be disposed in a relatively close positional relationship, so that when photographing with the imaging unit 12, the reflection of a shadow of an object caused by the light of the illumination section 11 can be minimized.

[0026] [Information Processing System] Next, the configuration of the information processing system 1 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an overview of the information processing system 1 according to the embodiment. Note that, in the following, a case where the information processing system 1 is introduced into a factory will be described as an example.

[0027] 2, the information processing system 1 according to the embodiment includes a plurality of lighting devices 10 and an information processing device 20. In the example shown in Fig. 2, each lighting device 10 is installed at a work site (for example, a production line) of a factory.

[0028] The lighting devices 10 are provided, for example, above the work spaces 40 (see FIG. 4) of the workers 50 in the production line, and the cameras 16 capture images of the work spaces 40 from above.

[0029] The information processing device 20 is a device that manages each lighting device 10 installed at a work site. For example, the information processing device 20 remotely controls the lighting mode of the lighting unit 11 of each lighting device 10, the imaging process of the camera 16, and the like via the network N.

[0030] Furthermore, the information processing device 20 collects and stores captured images captured by the camera 16 of each lighting device 10. For example, the captured images collected by the information processing device 20 are used for monitoring a work site. In the present disclosure, the information processing device 20 may acquire captured images as still images or as moving images.

[0031] In the present disclosure, the information processing system 1 detects the position of a specific body part of a worker while working and objects related to the work (products, parts, etc.), and performs an analysis of the worker's work based on the detected positions of the specific body parts and objects.

[0032] In this way, by detecting the worker's specific body parts and objects and analyzing the work, it is possible to perform work analysis based on the relationship between the object and the position of the specific body part, thereby enabling the worker's movements while working to be analyzed with high accuracy.

[0033] [Information processing device] Next, the configuration of the information processing device 20 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 20 according to the embodiment. As shown in Fig. 3, the information processing device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0034] The communication unit 21 is realized by, for example, a network interface card (NIC), etc. The communication unit 21 transmits and receives information to and from the plurality of lighting devices 10 and the administrator terminal 30 via the network N, for example.

[0035] The storage unit 22 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0036] 3, the storage unit 22 includes an image storage unit 221, a work time storage unit 222, a work process storage unit 223, and a work information storage unit 224. The image storage unit 221 stores images captured by each lighting device 10.

[0037] The work time memory unit 222 stores the time during which a worker 50 (see FIG. 4) performs valid work (hereinafter also referred to as valid work time) and the time during which a worker performs invalid work (hereinafter also referred to as invalid work time) in one unit task or one work process described below. The work time memory unit 222 also stores the takt time of the worker. The takt time is the time required for a worker to complete a work process for one product. The takt time may be stored separately for each unit task in the work process.

[0038] In this disclosure, useful work is work that adds value to an object of the work, such as attaching a part to an object.

[0039] In addition, in this disclosure, an invalid operation is an operation that does not add value to the object of the operation, such as removing a part or a tool for a work. In addition, in this disclosure, a unit operation is, for example, an operation performed on a single object.

[0040] The work process memory unit 223 stores pre-registered work processes that each worker 50 should perform in the production line. In the present disclosure, a work process is information in which each process is a unit task that the worker 50 should perform in the work space 40. Specifically, a work process includes the type of unit task to be performed and the order in which the unit tasks are performed (work sequence). Note that the work sequence does not have to be included in the work process.

[0041] The work information storage unit 224 stores the work performed by each worker 50. For example, the work information storage unit 224 stores the type of unit work performed by the worker 50 and the start time and end time of each unit work (or only one of the times may be stored). In other words, the work information storage unit 224 stores the history of work performed by each worker 50.

[0042] The control unit 23 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 20 using a RAM as a working area. The control unit 23 is also, for example, a controller, and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0043] 3, the control unit 23 includes an acquisition unit 231, a first detection unit 232, a second detection unit 233, an operation analysis unit 234, and an output unit 235, and realizes or executes the functions and actions of the control processing described below. Note that the internal configuration of the control unit 23 is not limited to the configuration shown in FIG. 3, and may be other configurations as long as they perform the control processing described below.

[0044] The acquisition unit 231 acquires captured images captured by the camera 16 of the lighting device 10. For example, the acquisition unit 231 acquires captured images from the camera 16 of each lighting device 10 in real time, and stores the acquired captured images in the video storage unit 221 for each work space 40 (or for each worker 50).

[0045] The first detection unit 232 performs various detections. For example, the first detection unit 232 functions as a part detection unit that detects the positions of both hands 51 (see FIG. 4) of the worker 50 from the captured image acquired by the acquisition unit 231 and stored in the video storage unit 221. Specifically, the first detection unit 232 detects the skeleton of the worker and detects the positions of both hands 51 of the worker 50 based on the detection result. Both hands 51 of the worker 50 are an example of a specific part of the body of the worker 50. Note that the first detection unit 232 may detect only one hand as the specific part of the body of the worker 50, or may detect parts other than the hands, such as the head or feet. In other words, the specific part may be any part of the body that the worker 50 uses in the work.

[0046] The second detection unit 233 functions as an object detection unit that detects the position of an object (target object) to be worked on in the work space 40 from the captured image acquired by the acquisition unit 231. Specifically, the second detection unit 233 detects the position of objects such as products (including objects in the middle of production) that the worker is manufacturing and parts used in the products. The second detection unit 233 also detects the amount of the detected products and parts. For example, the second detection unit 233 detects the number of completed products, the number of products in the middle of production, and the remaining amount of parts (the number of screws and the remaining amount of tape). The second detection unit 233 also detects the movement trajectory of an object that follows the detected object. For example, the second detection unit 233 detects the movement trajectory from the start of production of one product to the end of production.

[0047] The work analysis unit 234 analyzes the work information of the worker based on the positions of both hands 51 of the worker detected by the first detection unit 232 and the object detected by the second detection unit 233. Specifically, the work analysis unit 234 determines whether the worker is performing valid work or invalid work based on the detected positions of both hands 51 of the worker and the position of the object. Specifically, the work analysis unit 234 determines whether the work is valid or invalid based on the state in which the object is held by both hands 51 of the worker and whether both hands 51 are present within the work area.

[0048] Then, the work analysis unit 234 measures the time during which the worker performed valid work (valid work time) and the time during which the worker performed invalid work (invalid work time) in one unit work, and stores them in the work time memory unit 222 of the memory unit 22.

[0049] Furthermore, the work analysis unit 234 detects the takt time of the worker based on the detected positions of the specific body parts of the worker and the movement trajectory of the product. Specifically, the work analysis unit 234 detects the time required for the worker to complete the work process for each product as the takt time. Note that the takt time may be the time for each unit task in the work process.

[0050] The work analysis unit 234 also compares the takt time for each detected product, calculates the comparison of takt times and the variation for each worker, and stores the results as work information. This makes it possible to compare information about takt times for each worker. The work analysis unit 234 also compares the movement trajectories of each product to detect missing work processes, work mistakes, work interruptions, and the occurrence of exceptional work, and stores the results as work information.

[0051] The output unit 235 outputs various information. For example, the output unit 235 outputs the work information generated by the work analysis unit 234 to an administrator terminal of a manager who manages the workers. This allows the workers and managers to analyze the movements of the workers during work.

[0052] Next, a processing procedure of the processing executed by the information processing device 20 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the processing procedure of the processing executed by the information processing device 20 according to the embodiment. Note that, although the information processing system 1 will be described below as the processing subject, the processing shown in Fig. 4 may be performed by any of the devices such as the lighting device 10 and the information processing device 20 depending on the device configuration included in the information processing system 1.

[0053] 4, first, the information processing system 1 acquires a video captured by the camera 16 (step S101). Note that the information processing system 1 is not limited to acquiring the video directly from the lighting device 10, and may acquire the video stored in the video storage unit 221.

[0054] Next, the information processing system 1 analyzes the acquired video and detects the positions of both hands of the worker shown in the video (step S102).

[0055] Next, the information processing system 1 analyzes the acquired video and detects objects (manufactured products, parts, etc.) related to the work that are shown in the video (step S103).

[0056] Next, the information processing system 1 analyzes the work information based on the positions of the worker's hands and the position of the object (step S104), stores the analyzed work information in the work information storage unit 224 (step S105), and ends the process.

[0057] As described above, the information processing device 20 according to the embodiment includes the storage unit 22 that stores the video captured by the imaging unit (camera 16), a body part detection unit (first detection unit 232) that detects the position of a specific body part of the worker from the video stored in the storage unit 22, an object detection unit (second detection unit 233) that detects an object from the video, and a work analysis unit 234 that analyzes the work information of the worker based on the detected object and the position of the specific body part. This makes it possible to analyze the movement of the worker while working with high accuracy.

[0058] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. This embodiment and its modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and gist of the invention. [Explanation of symbols]

[0059] 1. Information Processing Systems 10. Lighting Equipment 11 Lighting Section 12 Imaging unit 13 Main body 14 Diffusion Cover 15 Light-shielding cover 16 Cameras 20 Information processing device 21 Communications Department 22 Memory section 23 Control Unit 30 Administrator terminal 40 Working Space 41 Workbench 50 workers 51 Both hands 221 Video storage unit 222 Working Time Memory Unit 223 Work process memory section 224 Work information storage unit 231 Acquisition Department 232 First detection unit 233 Second detection unit 234 Work Analysis Department 235 Output section N Network

Claims

1. A storage unit that stores the image captured by the imaging unit; a body part detection unit that detects a position of a specific body part of the worker from the image stored in the storage unit; an object detection unit that detects an object from the image; a work analysis unit that analyzes work information of the worker based on the detected object and the position of the specific part; An information processing device having the above configuration.

2. The object detection unit is Detecting the amount of the object; The work analysis unit, Analyzing the work information based on the amount of the object The information processing device according to claim 1 .

3. The object detection unit is Detecting a movement trajectory of the object; The work analysis unit, The work information is analyzed based on the movement trajectory of the object. The information processing device according to claim 1 .

4. The work analysis unit, The work information is analyzed based on a holding state of the object by the worker. The information processing device according to claim 1 .

5. The object detection unit is Detecting a product and a part used in the product as the object; The work analysis unit, Analyzing the work information based on the product and the part. The information processing device according to claim 1 .

6. The part detection unit Detecting the position of the specific part based on the result of the skeleton detection of the worker The information processing device according to claim 1 .

7. A storage step of storing the image captured by the image capturing unit in a storage unit; a part detection step of detecting a position of a specific part of the worker's body from the image stored in the storage unit; an object detection step of detecting an object from the image; a work analysis step of analyzing work information of the worker based on the detected object and the position of the specific part; An information processing method comprising:

8. A lighting device having a shooting unit for shooting an image; A storage unit that stores the image captured by the imaging unit; a body part detection unit that detects a position of a specific body part of the worker from the image stored in the storage unit; an object detection unit that detects an object from the image; a work analysis unit that analyzes work information of the worker based on the detected object and the position of the specific part; An information processing system having the above configuration.

Images