Work information processing device

Abstract

To obtain a work information processing device that can facilitate improvements in workers' work processes. [Solution] The work information processing device comprises an acquisition unit that acquires captured images of an operator, and a processing unit that, based on the captured images, presents improvement measures related to the work being performed by the operator included in the captured images.

Description

Technical Field

[0001] The present invention relates to a work information processing device.

Background Art

[0002] Conventionally, there is a technique for calculating a load value indicating the physical load of a worker performing work. As this type of technique, for example, there is one that presents a work target and a load value in association with each other. There is also one that calculates a load value based on the skeletal points of a worker, generates evaluation information from the load value, and displays the evaluation information.

Prior Art Documents

Patent Documents

[0003] [[ID=2�]]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, it is unclear how the prior art should improve the work, and there is room for improvement.

[0005] The present invention has been made in view of the above, and one of the problems is to obtain a work information processing device that can easily improve the work of a worker.

Means for Solving the Problems

[0006] The work information processing device according to the present invention includes an acquisition unit that acquires a captured image of a worker, and a processing unit that presents improvement measures related to the work being performed by the worker included in the captured image based on the captured image. [[ID=S2]]

Effects of the Invention

[0007] The work information processing device according to the present invention provides a work information processing device that can facilitate improvements in workers' work processes. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a system configuration diagram showing an example of a work information processing system according to the embodiment. [Figure 2] Figure 2 shows an example of a work procedure manual according to this embodiment. [Figure 3] Figure 3 shows an example of a rough score list for work posture according to the embodiment. [Figure 4] Figure 4 shows an example of a workload value table according to the embodiment. [Figure 5] Figure 5 is a block diagram showing an example of the functional configuration of the work information processing device according to the embodiment. [Figure 6] Figure 6 is a diagram illustrating the skeletal coordinate estimation process according to the embodiment. [Figure 7] Figure 7 shows an example of the configuration of the analysis results screen according to this embodiment. [Figure 8] Figure 8 shows a portion of the analysis results screen according to this embodiment. [Figure 9] Figure 9 shows a portion of the analysis results screen according to this embodiment. [Figure 10] Figure 10 is a diagram illustrating the work information processing according to the embodiment. [Figure 11] Figure 11 is a diagram illustrating the work information processing according to the embodiment. [Figure 12] Figure 12 is a diagram illustrating the work information processing according to the embodiment. [Figure 13] Figure 13 is a diagram illustrating the work information processing according to the embodiment. [Figure 14] Figure 14 is a diagram illustrating the work information processing according to the embodiment. [Figure 15] Figure 15 is a diagram illustrating the work information processing according to the embodiment. [Figure 16] FIG. 16 is a diagram for explaining work information processing according to an embodiment. [Figure 17] FIG. 17 is a diagram for explaining work information processing according to a first modification of the embodiment. [Figure 18] FIG. 18 is a block diagram showing an example of the functional configuration of a work information processing apparatus according to a second modification of the embodiment. [Figure 19] FIG. 19 is a diagram for explaining work information processing according to a second modification of the embodiment. [Figure 20] FIG. 20 is a diagram for explaining work information processing according to a second modification of the embodiment. [Figure 21] FIG. 21 is a diagram for explaining work information processing according to a second modification of the embodiment. MODE FOR CARRYING OUT THE INVENTION

[0009] Hereinafter, embodiments will be described with reference to the drawings. In this specification, the components according to the embodiments and the descriptions of the components may be described in a plurality of expressions. The components and their descriptions are examples and are not limited by the expressions in this specification. The components may be specified by different names from those in this specification. Also, the components may be described by expressions different from those in this specification.

[0010] <Work information processing system> FIG. 1 is a system configuration diagram showing an example of a work information processing system according to an embodiment. The work information processing system 1 shown in FIG. 1 executes processing related to improving the work of workers who work in a facility such as a factory. The work information processing system 1 includes an imaging device 10, a work information processing device 20, and a terminal device 30. The imaging device 10, the work information processing device 20, and the terminal device 30 are connected to each other via a communication network. The communication network may perform communication by at least one of wired and wireless. The work information processing device 20 is also referred to as a work evaluation device.

[0011] <Imaging device> The imaging device 10 is installed in a location where it can capture images of workers performing tasks in a factory or similar facility. The imaging device 10 does not necessarily have to be installed in a predetermined location and may be handheld. The imaging device 10 is configured as an area camera (area sensor) capable of outputting a color two-dimensional image (captured image) by capturing an object. The imaging device 10 can capture, for example, at least one of still images and video. In this embodiment, the imaging device 10 captures at least video. Furthermore, the imaging device 10 includes a communication device, which transmits the captured image (captured data) captured by the imaging device 10 to the work information processing device 20.

[0012] <Terminal device 30> The terminal device 30 is, for example, a mobile terminal such as a smartphone or tablet device, or a personal computer. The terminal device 30 comprises an operating device 301, a display device 302, a communication device, and a control device.

[0013] The display device 302 is, for example, a display device such as a liquid crystal or organic EL display. The input device 303 is an input device such as a touch panel (including non-contact types) or hard keys superimposed on the display device 302. The display device 302 displays various screens. The communication device communicates with other devices. The control device controls each part of the terminal device 30.

[0014] <Work Information Processing Device> The work information processing device 20 comprises a processing device 201, a communication device 202, a storage device 203, an operating device 204, and a display device 205.

[0015] The processing unit 201 includes a CPU 211, ROM 212, RAM 213, etc. The CPU 211 performs various controls and calculations on the work information processing unit 20 by, for example, executing a program stored in the ROM 212. The processing unit 201 may also include a GPU and VRAM.

[0016] The communication device 202 is an element that realizes communication functions for communicating with external devices. In the work information processing device 20 of this embodiment, the communication device 202 establishes communication with the imaging device 10 and the terminal device 30.

[0017] The storage device 203 is composed of, for example, an HDD (Hard Disk Drive). The storage device 203 stores various information and data. For example, the storage device 203 stores the work procedure manual 231, the work posture raw score list 232, the workload value table 233, the improvement content table 234, and the learned model 235.

[0018] Figure 2 shows an example of a work procedure manual according to an embodiment. As shown in Figure 2, the work procedure manual 231 shows the content of each element work item (work item). The work procedure manual 231 is, for example, an electronic file.

[0019] Figure 3 shows an example of a work posture raw score list according to the embodiment. As shown in Figure 3, the work posture raw score list 232 shows the work load value (score in Figure 3) for each work posture, which is the posture of the worker. Note that the work load value is not limited to those shown in Figure 3.

[0020] Figure 4 shows an example of a work load value table according to the embodiment. As shown in Figure 4, the work load value table 233 shows the conditions and detailed items of the conditions for each work load value (value in Figure 4) for each work posture. The detailed items of the conditions include the angles of each part (joint) of the worker. The angles of each part of the worker include, for example, the angles of the waist, knees, arms, elbows, etc., and the inclination angle and twisting angle of the upper body relative to the lower body of the worker.

[0021] The operating device 204 shown in Figure 1 is composed of, for example, a keyboard, a mouse, and touch panel switches formed on the display screen of the display device 205.

[0022] The display device 205 is composed of, for example, a liquid crystal display.

[0023] Figure 5 is a block diagram showing an example of the functional configuration of the work information processing device according to this embodiment. As shown in Figure 5, the processing device 201 has an acquisition unit 221 and a processing unit 222 as functional units. The processing unit 222 also includes a posture estimation unit 223, a work load value calculation unit 224, a statistical processing unit 225, a work improvement suggestion unit 226, and a display data generation unit 227. Details of these functional units will be described later.

[0024] The program executed by the work information processing device 20 in this embodiment is provided pre-installed in a ROM 212 or the like.

[0025] The program executed by the work information processing device 20 of this embodiment may be configured to be provided as an installable or executable file recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk), or USB memory.

[0026] Furthermore, the program executed by the work information processing device 20 of this embodiment may be stored on a computer, such as the work information processing device 20, connected to a network such as the Internet, and provided by downloading it via the network. Alternatively, the program executed by the work information processing device 20 of this embodiment may be provided or distributed via a network such as the Internet.

[0027] The program executed by the work information processing device 20 of this embodiment has a modular configuration including the above-described parts (acquisition unit 221, processing unit 222). In actual hardware, the CPU 211 reads the program from the ROM 212 and executes it, loading the above-described parts onto main memory such as RAM 213, and generating the acquisition unit 221 and processing unit 222 on the main memory such as RAM 213. Alternatively, the GPU may read the program from the ROM 212 and execute it, loading the above-described parts onto main memory such as VRAM, and generating the acquisition unit 221 and processing unit 222 on the main memory such as VRAM.

[0028] Alternatively, the functions of the acquisition unit 221 and the processing unit 222 may be implemented using hardware.

[0029] The acquisition unit 221 acquires the captured image G (see Figure 6) of the worker from the imaging device 10.

[0030] The processing unit 222 presents improvement measures related to the work being performed by the worker included in the captured image G, based on the captured image G. For example, the processing unit 222 presents improvement measures by adding them to the work procedure manual 231 related to the work. At this time, the processing unit 222 calculates a workload value indicating the physical burden on the worker based on the captured image G and presents improvement measures based on the workload value. The processing unit 222 also outputs data for an analysis result screen M (see Figures 7 to 9) that can be displayed by the display devices 205 and 302, which includes information corresponding to the workload value and the captured image G.

[0031] The following describes each part of the processing unit 222 in detail.

[0032] The posture estimation unit 223 estimates the worker's posture based on the captured image G. Here, Figure 6 is a diagram illustrating the skeletal coordinate estimation process according to the embodiment. For example, as shown in Figure 6, the posture estimation unit 223 estimates the worker's two-dimensional skeletal coordinates K1 (also referred to as the two-dimensional skeleton) based on the worker image H, which is an image of the worker in the captured image G, and estimates the worker's three-dimensional skeletal coordinates K2 (also referred to as the three-dimensional skeleton) from the worker's two-dimensional skeletal coordinates K1. Hereafter, the two-dimensional skeletal coordinates K1 and the three-dimensional skeletal coordinates K2 will be collectively referred to as skeletal coordinates K (skeletal model). The posture estimation unit 223 may also directly estimate the three-dimensional skeletal coordinates K2 from the captured image G.

[0033] The posture estimation unit 223 estimates the skeletal coordinates K of the worker's posture using a known method. For example, the posture estimation unit 223 estimates the skeletal coordinates K using a trained model 235. The trained model 235 includes, for example, a two-dimensional skeletal coordinate estimation model and a three-dimensional skeletal coordinate estimation model. The two-dimensional skeletal coordinate estimation model is trained to detect a worker (person) in the captured image G when the captured image G is input, and to estimate and output the two-dimensional skeletal coordinates K1 of the detected worker. The three-dimensional skeletal coordinate estimation model is trained to estimate and output the three-dimensional skeletal coordinates K2 of the worker when the two-dimensional skeletal coordinates K1 of the worker are input.

[0034] The work load value calculation unit 224 calculates a work load value indicating the physical load on the worker based on the worker's posture (the worker's three-dimensional skeletal coordinates K2, etc.) estimated by the posture estimation unit 223. At this time, the work load value calculation unit 224 calculates the angles of each part of the worker from the worker's posture (the worker's three-dimensional skeletal coordinates K2, etc.) estimated by the posture estimation unit 223. The angles of each part of the worker include, for example, the angles of the waist, knees, arms, elbows, etc., as well as the inclination angle and twist angle of the upper body relative to the lower body of the worker. The work load value calculation unit 224 compares the calculated angles of each part of the worker with the work load value table 233 to calculate the worker's work load value. The work load value calculation unit 224 outputs the calculated work load value along with the time of work (the time of acquisition of the captured image).

[0035] The statistical processing unit 225 generates and outputs statistical data related to the workload value calculated by the workload value calculation unit 224. For example, the statistical processing unit 225 calculates and outputs the workload value for each predetermined period during the work period, which is the period of work performed by the worker. For example, the statistical processing unit 225 calculates the final score according to the calculation formula. Note that the statistical processing unit 225 is not required to be provided.

[0036] The work improvement suggestion unit 226 suggests improvement measures for the work being performed by the worker included in the captured image G, based on the workload value output from the workload value calculation unit 224 or the statistical processing unit 225. For example, the work improvement suggestion unit 226 extracts improvement measures corresponding to the workload value output from the workload value calculation unit 224 or the statistical processing unit 225 from the improvement content table 234. The work improvement suggestion unit 226 adds the extracted improvement measures to the work procedure manual 231, or modifies the work procedure manual 231 with the extracted improvement measures. The improvement measures are those that reduce the workload value.

[0037] The display data generation unit 227 generates and outputs data for the analysis results screen based on the data output from each of the above units. The analysis results screen is an example of a screen that can be displayed by the display devices 205 and 302.

[0038] Figure 7 shows an example of the configuration of the analysis results screen according to the embodiment. Figure 8 shows a part of the analysis results screen according to the embodiment. Figures 7 to 9 show parts of the analysis results screen according to the embodiment. As shown in Figure 7, the analysis results screen M includes a graph Ma, a graph explanation section Mb, an image display section Mc, and a workload value display section Gd. The analysis results screen M is an example of a screen.

[0039] As shown in Figure 8, Graph Ma shows the relationship between the workload value, the angles of each part of the worker's body, and the time (elapsed time). The left vertical axis of Graph Ma represents the workload value, the right vertical axis represents the angles, and the horizontal axis represents the time (elapsed time). The graph explanation section Mb provides explanations regarding the contents of Graph Ma.

[0040] As shown in Figure 9, the workload value display section Gd shows a list of workload values, and a specific display indicates the workload value at a particular time. This specific display is, for example, indicated by a background color different from other areas. In the example in Figure 9, the workload value "8" is specifically displayed. This specific time is indicated by the movable line L1 in graph Ma. The movable line L1 moves along the horizontal axis of graph Ma.

[0041] As shown in Figure 9, the image display unit Mc displays the captured image G and information Me. Information Me is displayed overlaid on the captured image G. Warning Mf corresponds to the work performed at the time indicated by the movable line L1. Information Me includes warning Mf and the angles of each part of the worker. Warning Mf is, for example, "There is a load on your waist." Warning Mf is pre-set according to the workload value. For example, if the workload value is "4 points," the warning Mf is "There is a load on your knees," and if the workload value is "5 points," it is "There is a load on both your waist and knees." Warning Mf may be highlighted with color or bold text. Warning Mf is not limited to text; it may also be an icon, or a combination of text and an icon. Warning Mf may also be displayed outside the captured image G.

[0042] Next, the processes executed by the work information processing device 20 will be described with reference to Figures 10 to 16. Figures 10 to 16 are diagrams illustrating the work information processing according to the embodiment.

[0043] As shown in Figure 10, the acquisition unit 221 acquires the captured image G from the imaging device 10. The acquisition unit 221 also acquires the start and end times of each elemental operation within the video of the captured image G. The start and end times of each elemental operation are, for example, entered by the user via the operating device 204, and the acquisition unit 221 acquires the entered start and end times.

[0044] Next, as shown in Figure 11, the posture estimation unit 223 calculates the worker's posture (skeletal coordinates K) based on the captured image G, and the work load value calculation unit 224 calculates the angles of each part of the worker based on the worker's posture (skeletal coordinates K) and generates angle information D1 including these angles of each part. Alternatively, the posture estimation unit 223 may calculate the angles of each part of the worker and generate angle information D1 including these angles of each part.

[0045] Next, as shown in Figure 12, the workload value calculation unit 224 compares the angles of each part of the worker included in the angle information D1 with the workload value table 233 to calculate the worker's workload value. The workload value calculation unit 224 performs the above process for each frame of the captured image G. The workload value calculation unit 224 outputs the calculated workload value along with the time of the work (the time the captured image was taken).

[0046] Next, as shown in Figure 13, the work improvement suggestion unit 226 compares the time period for each elemental task with the calculated workload value to identify elemental tasks with relatively high workload values. In the example in Figure 3, the workload value is relatively high in time period Maa, and this time period Maa corresponds to the second elemental task. A time period Maa with a relatively high workload value may be, for example, a time period in which the workload value remains above a predetermined threshold (a value greater than the minimum workload value) for a predetermined period. Note that statistical data calculated by the statistical processing unit 225 may be used as the workload value for this time period.

[0047] Next, as shown in Figure 14, the work improvement suggestion unit 226 obtains the details of elemental tasks with relatively high workload values ​​from the work procedure manual 231.

[0048] Next, as shown in Figure 15, the work improvement suggestion unit 226 extracts improvement measures corresponding to the angle information D1 and workload value in the identified element work from the improvement content table 234 and generates the improvement content. Figure 15 is an example in which, for the second element work item "assemble XX while bending over to YY", the improvement content "while sitting in a chair" is extracted from the improvement content table 234, and the improvement measure "assemble XX while sitting in a chair to YY" is generated.

[0049] Next, as shown in Figure 16, the work improvement suggestion unit 226 adds the generated improvement measures to the work procedure manual 231. Specifically, the work improvement suggestion unit 226 rewrites the second element work item, "assemble XX while bending over at YY," to "assemble XX while sitting on a chair at YY." The work procedure manual 231 (improvement measures) can be displayed, for example, on the display devices 205 and 302.

[0050] <Summary> As described above, the work information processing device 20 of this embodiment comprises an acquisition unit 221 and a processing unit 222. The acquisition unit 221 acquires an image G of a worker. The processing unit 222, based on the image G, presents improvement measures related to the work being performed by the worker included in the image G.

[0051] With this configuration, the processing unit 222 presents improvement measures related to the work being performed by the worker, as contained in the captured image G, making it easier for the worker to improve their work. Furthermore, with this configuration, users such as workers can efficiently understand and interpret the improvement measures.

[0052] Furthermore, the processing unit 222 calculates a workload value indicating the physical burden on the worker based on the captured image G, suggests improvement measures based on the workload value, and outputs data for an analysis result screen M (screen) that can be displayed by the display devices 205 and 302, which includes information corresponding to the workload value (e.g., warning Mf) and the captured image G.

[0053] With this configuration, an analysis result screen M (screen) including information corresponding to the workload value and the captured image G can be displayed on the display devices 205 and 302, thereby improving the efficiency of users such as workers in understanding and interpreting the workload situation on them during work.

[0054] Furthermore, the processing unit 222 presents improvement measures by adding them to the work procedure manual 231 related to the work.

[0055] This configuration makes it easier to communicate improvement measures to users such as workers.

[0056] Next, a first modified example will be described. Figure 17 is a diagram illustrating the work information processing related to the first modified example of the embodiment.

[0057] As shown in Figure 17, in this modified example, the work improvement suggestion unit 226 generates improvement measures corresponding to the angle information D1 and workload value of the identified elemental work using the trained model 235. The trained model 235 is trained to generate and output the best solution when, for example, the angle information D1 and workload value of the identified elemental work are input.

[0058] Next, a second modification will be described. Figure 18 is a block diagram showing an example of the functional configuration of a work information processing device according to the second modification of the embodiment. Figure 19 is a diagram illustrating the work information processing according to the second modification of the embodiment. Figure 20 is a diagram illustrating the work information processing according to the second modification of the embodiment. Figure 21 is a diagram illustrating the work information processing according to the second modification of the embodiment.

[0059] In this modified example, the processing unit 222 estimates the worker's posture based on the captured image G, identifies the worker's actions based on the captured image G, and calculates a work load value indicating the worker's physical burden based on the worker's posture and actions.

[0060] As an example, as shown in Figure 18, the processing unit 222 further includes an action estimation unit 228. The action estimation unit 228 identifies the worker's actions based on the captured image G. As shown in Figure 19, the action estimation unit 228 calculates skeletal coordinates K from the captured image G, for example, similar to the posture estimation unit 223. The action estimation unit 228 may also obtain the skeletal coordinates K calculated by the posture estimation unit 223 from the posture estimation unit 223. As shown in Figure 20, the action estimation unit 228 estimates the worker's actions from the time-series data of skeletal coordinates K. For example, when time-series data of skeletal coordinates K is input, the action estimation unit 228 identifies the worker's actions using a model that has been trained to identify and output the worker's actions (trained model). The worker's actions include, for example, walking, sitting, etc.

[0061] As shown in Figure 21, the workload value calculation unit 224 sets the workload value to a fixed value regardless of the worker's posture if the worker's behavior identified by the behavior estimation unit 228 is a specific behavior. For example, a specific behavior is at least one (or both, as an example) of walking and sitting. The workload value calculation unit 224 sets the workload value to a low value (e.g., "1") if the worker's behavior identified by the behavior estimation unit 228 is a specific behavior (walking or sitting). This low value may be, for example, the lowest value within the range of workload values, or it may be below the midpoint of the range of workload values, etc.

[0062] As described above, in this modified example, the processing unit 222 estimates the worker's posture based on the captured image G, identifies the worker's actions based on the captured image G, and calculates a work load value indicating the worker's physical burden based on the worker's posture and actions.

[0063] With this configuration, the processing unit 222 calculates the workload value based on the worker's posture and actions, thereby improving the accuracy of the workload value calculation.

[0064] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0065] 20…Work Information Processing Device 205,302…Display device 221…Acquisition Department 222… Processing Unit 231…Work instructions G...imaging M…Analysis result screen (screen) Mf...Warning (Information)

Claims

1. An acquisition unit that acquires captured images taken by the operator, A processing unit that, based on the captured image, presents improvement measures related to the work being performed by the worker included in the captured image, A work information processing device equipped with the following features.

2. The processing unit calculates a workload value indicating the physical burden on the worker based on the captured image, presents improvement measures based on the workload value, and outputs screen data that can be displayed by the display device, including information corresponding to the workload value and the captured image. The work information processing device according to claim 1.

3. The processing unit estimates the worker's posture based on the captured image, identifies the worker's actions based on the captured image, calculates a workload value indicating the worker's physical burden based on the posture and actions, and proposes improvement measures based on the workload value. The work information processing device according to claim 1.

4. The processing unit presents the improvement measures by adding them to the work procedure manual for the work. A work information processing device according to any one of claims 1 to 3.

Images