Information processing device and lighting system

Abstract

To provide an information processing device and an illumination system for accurately setting an analysis range.SOLUTION: An information processing device according to an embodiment includes an acquisition unit and a setting unit. The acquisition unit acquires photographed images photographed by a camera. The setting unit sets an analysis range for analyzing an action regarding a prescribed work by using, as a reference, a marker contained in the acquired photographed image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD An embodiment of the present invention relates to an information processing device and a lighting system. [Background technology]

[0002] A technique has been proposed for detecting the work being performed by a worker based on an image captured by an imaging unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-120577 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, there is room for improvement in the method of setting the analysis range for checking the work of the worker.

[0005] The problem to be solved by the present invention is to provide an information processing device and a lighting system that can set an analysis range with high accuracy. [Means for solving the problem]

[0006] The information processing device according to the embodiment includes an acquisition unit and a setting unit. The acquisition unit acquires a captured image captured by a camera. The setting unit sets an analysis range for analyzing movements related to a predetermined task based on markers included in the captured image. The setting unit sets, as the analysis range, a range including locations where the worker's hand movements are greater than a preset threshold, based on the markers and a heat map according to the worker's movements. [Effects of the Invention]

[0007] According to the present invention, the analysis range can be set with high precision. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a perspective view showing an example of the appearance of a lighting device included in a lighting system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an overview of a lighting system according to an embodiment. [Figure 3] FIG. 3 is a block diagram showing the configuration of the information processing device according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of setting an analysis range in a captured image. [Figure 5] FIG. 5 is a diagram showing an example of setting an analysis range in a captured image. [Figure 6] FIG. 6 is a flowchart illustrating the process of setting the analysis range according to the embodiment. [Figure 7] FIG. 7 is a diagram showing an example of setting an analysis range in a captured image according to a modified example. [Figure 8] FIG. 8 is a diagram showing an example of setting an analysis range in a captured image according to a modified example. [Figure 9] FIG. 9 is a diagram showing an example of setting an analysis range in a captured image according to a modified example. [Figure 10] FIG. 10 is a diagram showing an example of setting an analysis range in a captured image according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] The information processing device 20 according to the embodiment described below includes an acquisition unit 231 and a setting unit 234. The acquisition unit 231 acquires a captured image P captured by the camera 16. The setting unit 234 sets an analysis range R for analyzing movements related to a predetermined task, based on a marker M included in the acquired captured image P.

[0010] Moreover, the setting unit 234 according to the embodiment described below sets the analysis range R based on the coordinates of the marker M.

[0011] Furthermore, the setting unit 234 according to the embodiment described below sets the analysis range R based on the marker M placed on the workbench.

[0012] Furthermore, the setting unit 234 according to the embodiment described below sets the analysis range R based on the marker M, the installation location of which on the workbench can be changed.

[0013] Furthermore, the setting unit 234 according to the embodiment described below sets the analysis range R based on a plurality of markers M.

[0014] Furthermore, the setting unit 234 according to the embodiment described below sets an analysis range R for each of the multiple markers M.

[0015] Furthermore, the setting unit 234 according to the embodiment described below sets the analysis range R2 based on the markers and a heat map according to the worker's movements.

[0016] Furthermore, the setting unit 234 according to the embodiment described below can switch the setting of the analysis range between automatic setting based on the marker and manual setting.

[0017] Moreover, the lighting system 1 according to the embodiment described below includes an information processing device 20 and a lighting device. The lighting device has a camera 16 and a lighting device.

[0018] [Lighting equipment] First, the configuration of a lighting device 10 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a perspective view showing an example of the appearance of the lighting device 10 included in a lighting system 1 according to an embodiment.

[0019] 1, lighting device 10 includes lighting unit 11, imaging unit 12, and main body 13. Lighting device 10 is a ceiling-mounted lighting device in which main body 13 is installed on a ceiling surface or the like, and light output from lighting unit 11 is irradiated onto a floor surface. Lighting device 10 is primarily used indoors, for example, for monitoring production lines in a factory or the status of employees in an office.

[0020] For ease of understanding, Fig. 1 illustrates a three-dimensional Cartesian coordinate system including a Z axis, with the positive direction being vertically downward and the negative direction being vertically upward. The X axis is illustrated as extending along the length direction of the lighting device 10, and the Y axis is illustrated as extending along the width direction of the lighting device 10.

[0021] The lighting unit 11 is an illumination bar having a plurality of light-emitting elements (not shown) arranged at predetermined intervals on a long chassis or substrate (not shown) arranged along the Y-axis direction, and a diffusion cover 14 is provided on the floor side, i.e., on the positive Z-axis side, so that the light-emitting elements are housed between the chassis and the light-emitting elements.

[0022] The diffusion cover 14 is made of a light-transmitting material such as acrylic or polycarbonate. The diffusion cover 14 is frosted to diffuse the light emitted from the multiple light-emitting elements. Note that the diffusion cover 14 may contain a suitable diffusing material or colorant.

[0023] The imaging unit 12 has a light-shielding cover 15 and a camera 16. The imaging units 12 are arranged side by side adjacent to the illumination unit 11 on the negative side of the X axis.

[0024] The main body 13 holds the lighting section 11 and the imaging unit 12. The main body 13 also serves as a mounting member for mounting the lighting device 10 to a predetermined position such as a ceiling.

[0025] The light-shielding cover 15 is disposed on the positive side of the Z axis of the main body 13 so as to cover the camera 16 between the main body 13 and the light-shielding cover 15. The light-shielding cover 15 also has a through-hole provided in a position facing the lens of the camera 16. By disposing such a light-shielding cover 15, it is possible to make it difficult for light emitted from the diffusion cover 14 of the illumination unit 11 to enter the lens of the camera 16.

[0026] [Lighting System] Next, the configuration of a lighting system 1 according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an overview of the lighting system 1 according to an embodiment. Note that the following description will be given taking as an example a case where the lighting system 1 is installed in a factory.

[0027] 2, the lighting system 1 according to the embodiment includes a plurality of lighting devices 10 and an information processing device 20. In the example shown in FIG. 2, each lighting device 10 is installed in a work site (for example, a production line) of a factory.

[0028] The lighting devices 10 are provided, for example, above the work tables 40 (see FIG. 4) of the workers 50 in the production line, and the cameras 16 (see FIG. 1) capture images of the work tables 40 from above.

[0029] The information processing device 20 is a device that manages each lighting device 10 installed at a work site. For example, the information processing device 20 remotely controls the lighting mode of the lighting unit 11 of each lighting device 10, the imaging process of the camera 16, etc. via the network N.

[0030] The information processing device 20 also collects and stores captured images P (see FIG. 4) captured by the camera 16 of each lighting device 10. For example, the captured images P collected by the information processing device 20 are used to check and analyze the work content, monitor the work site, etc. In the present disclosure, the information processing device 20 may acquire the captured images P as still images or as moving images.

[0031] [Information processing device] Next, the configuration of the information processing device 20 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 20 according to the embodiment. As shown in Fig. 3, the information processing device 20 includes a communication unit 21, a storage unit 22, and a control unit 23. The information processing device 20 is, for example, a cloud server.

[0032] The communication unit 21 is realized by, for example, a network interface card (NIC), etc. The communication unit 21 transmits and receives information to and from a plurality of lighting devices 10 via the network N, for example.

[0033] The storage unit 22 functions as a storage unit in the cloud server and is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0034] For example, the storage unit 22 stores a captured image P captured by the camera 16. The storage unit 22 stores an analysis range R in the captured image P.

[0035] The control unit 23 functions as a control unit in the cloud server. The control unit 23 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 20 using RAM as a work area. The control unit 23 is also, for example, a controller, and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0036] 3, control unit 23 includes acquisition unit 231, detection unit 232, determination unit 233, and setting unit 234, and realizes or executes the functions and actions of the control processing described below. Note that the internal configuration of control unit 23 is not limited to the configuration shown in FIG. 3, and other configurations may be used as long as they perform the control processing described below.

[0037] The acquisition unit 231 acquires the captured image P captured by the camera 16 (see FIG. 1). The acquisition unit 231 acquires the captured image P from each camera 16 via the communication unit 21.

[0038] The detection unit 232 performs various detections. For example, the detection unit 232 detects a marker M from the captured image P. The detection unit 232 detects the coordinates of the marker M in the captured image P. The marker M is placed on the workbench 40. The placement location of the marker M with respect to the workbench 40 can be changed. The marker M is placed by, for example, an administrator. The marker M may also be placed by a worker 50 who performs work on the workbench 40.

[0039] The marker M is, for example, a symbol written on a card. For example, the marker M is a QR code (registered trademark) written on a card. The symbol may be a circle, a square, a cross, or the like. The marker M may be written on an object other than a card. The marker M may be written on an object with a three-dimensional shape. The marker M may be the shape of the card itself or the three-dimensional shape of the object itself.

[0040] The marker M is placed on the work bench 40, for example, via a member that can be attached to and detached from the work bench 40. It is desirable that the marker M is made of a member and has a shape that makes it difficult for it to fall off the work bench 40.

[0041] The detection unit 232 performs various detections by performing predetermined image processing on the acquired captured image P. The predetermined image processing is performed, for example, by using a detection model. The detection model is obtained by learning using a machine learning algorithm such as a deep neural network (DNN). The detection model may be obtained by supervised learning, for example. The detection model may be configured to use a machine learning algorithm such as a support vector machine (SVM) that uses a histogram of gradient (HOG) feature. The detection unit 232 may also be configured to perform various detections using, for example, template matching, without using a detection model that has undergone machine learning. The detection unit 232 detects the movement of the worker 50 from the captured image P. The detection unit 232 detects, for example, the positions of both hands.

[0042] The determination unit 233 determines whether or not the marker M has been detected. The determination unit 233 determines whether or not the marker M has been detected for each captured image P that has been acquired, for example.

[0043] The setting unit 234 sets an analysis range R for analyzing movements related to a predetermined task from the captured image P. The predetermined task is a task that is set in advance. For example, the predetermined task is a task performed on the workbench 40. The predetermined task is, for example, a task of attaching parts to an object, a task of packaging an object, or a task of picking up parts or work tools.

[0044] The setting unit 234 sets the analysis range R based on a marker M included in the captured image P. The setting unit 234 sets the analysis range R based on a marker M placed on the workbench 40. The setting unit 234 sets the analysis range R based on the coordinates of the marker M in the captured image P. For example, the setting unit 234 sets the coordinates of the analysis range R in the captured image P.

[0045] The setting unit 234 sets the analysis range R according to the marker M detected by the detection unit 232. The setting unit 234 sets the analysis range R according to the type of marker M. The marker M has information for setting the analysis range R. The information for distinguishing the method for setting the analysis range R includes the type of marker M, the shape of the marker M, the pattern of the marker M, etc.

[0046] For example, if the marker M detected by the detection unit 232 is a QR code, the setting unit 234 sets the analysis range R to have a shape and a size specified by the QR code.

[0047] For example, if the marker M is a QR code and the QR code indicates an analysis range R that extends above the captured image P with respect to the QR code and to the right of the captured image P with respect to the QR code, the analysis range R is set for the captured image P as shown in Fig. 4. Fig. 4 is a diagram showing an example of setting the analysis range R in the captured image P.

[0048] For example, if the marker M is a circle symbol and indicates an analysis range R having a predetermined radius relative to the circle symbol, the analysis range R is set for the captured image P as shown in FIG. 5. FIG. 5 is a diagram showing an example of setting the analysis range R for the captured image P. Note that the method for setting the analysis range R is not limited to the above method. For example, the analysis range R may be rectangular with the marker M at its center. For example, the analysis range R is set relative to the marker M in terms of the distance from the marker M and the extension direction relative to the marker M.

[0049] The information processing device 20, for example, collects work information of the workers 50 in the set analysis range R and performs an analysis to determine whether the work has been performed according to a pre-registered work procedure. The information processing device 20 also measures the takt time of the work, etc.

[0050] [Analysis range setting process] Next, the process of setting the analysis range R according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart illustrating the process of setting the analysis range R according to the embodiment.

[0051] The information processing device 20 acquires from the camera 16 a captured image P captured by the camera 16 (S100). The information processing device 20 executes predetermined image processing for detecting the marker M on the acquired captured image P (S101).

[0052] The information processing device 20 determines whether or not the marker M is present in the acquired captured image P (S102). If the marker M is not present in the captured image P (S102: No), the information processing device 20 ends the current process.

[0053] If the captured image P has a marker M (S102: Yes), the information processing device 20 sets an analysis range R according to the marker M (S103).

[0054] [effect] The information processing device 20 includes an acquisition unit 231 and a setting unit 234. The acquisition unit 231 acquires a captured image P captured by the camera 16. The setting unit 234 sets an analysis range R for analyzing movements related to a predetermined task, based on a marker M included in the acquired captured image P.

[0055] This allows the information processing device 20 to accurately set the analysis range R for the captured image P. For example, even if the work content on the workbench 40 is changed, or even if the layout of the workbench 40 is changed, the information processing device 20 can accurately set the analysis range R. Furthermore, the information processing device 20 can shorten the time required to set the analysis range R. For example, compared to when the analysis range R is set manually by an administrator or the like, the information processing device 20 can shorten the time required to set the analysis range R and can easily set the analysis range R.

[0056] The setting unit 234 sets the analysis range R based on the coordinates of the marker M.

[0057] This allows the information processing device 20 to accurately set the analysis range R for the marker M. The information processing device 20 can suppress deviations and variations in the analysis range R. As a result, the information processing device 20 can accurately detect the analysis results in the analysis range R, such as the movements and work content of the worker 50, and can improve the accuracy of the analysis results of the movements and work content of the worker 50.

[0058] The setting unit 234 sets the analysis range R based on the marker M set on the workbench 40 .

[0059] This allows the information processing device 20 to accurately set the analysis range R for the workbench 40. Therefore, for example, even if the work content on the workbench 40 is changed or the layout of the workbench 40 is changed, the information processing device 20 can accurately set the analysis range R for the workbench 40.

[0060] The setting unit 234 sets the analysis range R based on a marker M whose installation location on the workbench 40 can be changed.

[0061] As a result, even if the work content on the workbench 40 is changed, the marker M is moved according to the work content, etc., and the information processing device 20 can accurately set the analysis range R. Even if the work content on the workbench 40 is changed, the information processing device 20 can easily set an appropriate analysis range R.

[0062] [Variations] The information processing device 20 according to the modified example may automatically set the analysis range R for the captured image P after the analysis range R has been set once, until a reset operation for the analysis range R is performed. For example, even if the analysis range R is set and the marker M is removed from the workbench 40, the information processing device 20 may set the same analysis range R for the captured image P. The reset operation is performed, for example, by an administrator. The reset operation may also be performed at a predetermined timing. The predetermined timing is, for example, the timing when work begins. The predetermined timing includes a predetermined time.

[0063] The information processing device 20 according to the modified example may set the analysis range R based on a plurality of markers M. The information processing device 20 according to the modified example may set the analysis range R according to the type of the plurality of markers M. For example, as shown in FIG. 7, the information processing device 20 may set the analysis range R based on the coordinates of a plurality of related markers M. That is, the information processing device 20 may set the analysis range R according to the positional relationship of the plurality of markers M. FIG. 7 is a diagram showing an example of setting the analysis range R in a captured image P according to the modified example. The plurality of markers M in FIG. 7 are markers M in which the range of a polygon having the plurality of markers M as vertices is set as the analysis range R.

[0064] This allows the information processing device 20 to set the analysis range R with high accuracy based on the multiple markers M. Furthermore, the information processing device 20 can set an analysis range R having a complex shape.

[0065] An information processing device 20 according to the modified example may set an analysis range R for each of a plurality of markers M, as shown in Fig. 8. Fig. 8 is a diagram showing an example of setting the analysis range R in a captured image P according to the modified example.

[0066] This allows the information processing device 20 to set multiple analysis ranges R. For example, the information processing device 20 can analyze work for each analysis range R. For example, if the work procedures on the workbench 40 are determined, the information processing device 20 can easily check the work procedures by setting the analysis range R for each range where the hands of the worker 50 move according to the work procedures. Furthermore, for example, the information processing device 20 can easily measure the takt time for each work procedure.

[0067] 9, the information processing device 20 according to the modified example may set an area including the worker 50 as the analysis area R. FIG. 9 is a diagram showing an example of setting the analysis area R in the captured image P according to the modified example. That is, the marker M may include the worker 50.

[0068] The marker M may also include a box containing parts to be used in the work. For example, the information processing device 20 may detect information about the parts placed on the workbench 40 from the captured image P. For example, the information processing device 20 may detect part information from a part model number written on a box containing the part, a QR code written on the box, or the like. The information processing device 20 may also detect the type of part based on the captured image P.

[0069] The information processing device 20 according to the modified example may be able to adjust the analysis range R that is set based on the marker M. For example, when the worker 50 is performing work with his / her hands outside the analysis range R that is set based on the marker M, the information processing device 20 may set the analysis range R to include the range in which the worker 50's hands move. For example, the information processing device 20 according to the modified example may set the analysis range R based on a heat map.

[0070] The information processing device 20 according to the modified example may detect hand movements of the worker 50 from the captured image P and create a heat map based on the hand movements of the worker 50. The heat map may be created by another information processing device. The heat map is created based on the number of hand movements of the worker 50 within the angle of view of the captured image P captured by the camera 16, etc.

[0071] For example, in the heat map, areas where the worker 50 moves a lot and areas where the worker 50 moves less are distinguished by different colors or the like. The information processing device 20 according to the modified example may set the analysis range R based on the marker and the heat map. The information processing device 20 according to the modified example sets, as the analysis range R, a range that includes a predetermined range that includes areas in the heat map where the hand movements of the worker 50 are more than a preset threshold. For example, the information processing device 20 according to the modified example sets the analysis range R2 by adding an analysis range based on the heat map to the analysis range R that is set based on the marker M.

[0072] For example, as shown in Fig. 10, the information processing device 20 sets an analysis range R2 by adding a range including a predetermined range R1 that includes locations in the heat map where the hand movements of the worker 50 are greater than a preset threshold to the analysis range R that is set based on the marker M. Fig. 10 is a diagram showing an example of setting the analysis range R2 in a captured image P of a modified example. In Fig. 10, the analysis range R2 to which the range including the predetermined range R1 has been added is indicated by a dashed line.

[0073] As a result, the information processing device 20 according to the modified example can set the analysis range R2 based on a heat map according to the movement of the worker 50, thereby enabling the information processing device 20 to accurately set the analysis range R2 for the work of the worker 50. As a result, the information processing device 20 can improve the accuracy of the analysis results in the analysis range R2.

[0074] The information processing device 20 according to the modified example may be able to switch the setting of the analysis range R between automatic setting based on the marker M and manual setting. This allows the information processing device 20 to switch the setting method of the analysis range R depending on, for example, the purpose of use.

[0075] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as the inventions described in the claims and their equivalents. [Explanation of symbols]

[0076] 1. Lighting system 10. Lighting equipment 11 Lighting Department 12 Imaging unit 16 Camera 20 Information processing equipment 21 Communications Department 22 Memory section 23 Control Unit 40 Workbench 50 workers 231 Acquisition Department 232 Detector 234 Settings M marker P Captured image R Study Area R2 Study Area

Claims

1. an acquisition unit that acquires an image captured by the camera; a setting unit that sets an analysis range for analyzing a movement related to a predetermined task based on a marker included in the acquired captured image; Equipped with The setting unit is an information processing device that sets an analysis range to a range that includes locations where the worker's hand movements are greater than a predetermined threshold based on the markers and a heat map corresponding to the worker's movements.

2. The setting unit sets the analysis range based on the coordinates of the marker. The information processing device according to claim 1 .

3. The setting unit sets the analysis range based on the marker placed on the workbench.

3. The information processing device according to claim 1.

4. The setting unit sets the analysis range based on the marker, the installation location of which can be changed relative to the workbench. The information processing device according to claim 3 .

5. The setting unit sets the analysis range based on a plurality of markers.

5. The information processing device according to claim 1.

6. The setting unit sets the analysis range for each of a plurality of markers.

5. The information processing device according to claim 1.

7. The setting unit is capable of switching the setting of the analysis range between automatic setting based on the marker and manual setting.

7. The information processing device according to claim 1.

8. An information processing device according to any one of claims 1 to 7; an illumination device having the camera and an illumination unit; A lighting system comprising:

Images