Work analysis apparatus and method

Abstract

Provided is a work analysis device that generates information about work performed by a worker. The work analysis device comprises an acquisition unit and a control unit. The acquisition unit acquires image data that represents images of the worker performing work. On the basis of the image data, the control unit sequentially detects the work performed by the worker and generates work detection information that represents detection results for the work. The work includes a plurality of steps performed by the worker. In accordance with the steps performed during work detected before and after non-detection periods during which work was not detected, the control unit corrects the work detection information so as to interpolate detection results for work performed during the non-detection periods.

Description

Technical Field

[0001] The present disclosure relates to a work analysis apparatus and method.

Background Art

[0002] Patent Document 1 discloses a work analysis system that detects the boundaries of human work operations and analyzes the work. The system of Patent Document 1 acquires time-series signals of the position and speed of the worker's hand from the image data of the moving image captured by the camera during the work operation, and detects minima in the time-series signal of the speed as operation boundaries. The system of Patent Document 1 uses the operation boundaries to identify the work content and the start / end time of the work by pattern matching between the acquired position and a reference pattern. The system of Patent Document 1 identifies the boundaries of the work by using the time of the operation boundary as the start or end point of the pattern matching, or by using it for correcting the work time specified by the pattern matching.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] [[ID=3,4]] The present disclosure provides a work analysis apparatus and method that can accurately detect a specific work performed by a worker.

Means for Solving the Problems

[0005] A work analysis device according to one aspect of this disclosure generates information about work performed by an operator. The work analysis device comprises an acquisition unit and a control unit. The acquisition unit acquires image data showing an image of an operator performing work. Based on the image data, the control unit sequentially detects the work performed by the operator and generates work detection information indicating the work detection results. A work includes multiple steps performed by the operator. The control unit corrects the work detection information to interpolate the work detection results during undetected periods, according to the steps taken when work was detected before and after undetected periods.

[0006] These general and specific embodiments may be implemented by systems, methods, and computer programs, or combinations thereof. [Effects of the Invention]

[0007] The work analysis apparatus and method described herein can accurately detect specific tasks performed by workers. [Brief explanation of the drawing]

[0008] [Figure 1] Diagram showing an overview of the work analysis system according to Embodiment 1. [Figure 2] Block diagram illustrating the configuration of the work analysis device in a work analysis system. [Figure 3] To explain the challenges related to work analysis equipment, a diagram is provided. [Figure 4] Diagram illustrating the operation of the work analysis device. [Figure 5] Flowchart to explain the overall operation of the work analysis device [Figure 6] Flowchart illustrating the work detection process in the work analysis device of Embodiment 1 [Figure 7] Diagram to explain the work detection process [Figure 8] A flowchart illustrating the work detection process in a work analysis device according to a modified example of Embodiment 1. [Figure 9]A diagram illustrating the work detection process in the work analysis device of Embodiment 2. [Figure 10] Flowchart illustrating the work detection process of Embodiment 2 [Figure 11] A diagram illustrating the work detection process in the work analysis device of Embodiment 3. [Figure 12] Flowchart illustrating the work detection process of Embodiment 3 [Figure 13] A diagram illustrating the work detection process in the work analysis device of Embodiment 4. [Figure 14] Flowchart illustrating the work detection process of Embodiment 4 [Figure 15] Flowchart illustrating the work detection process in Modification Example 1 [Figure 16] Flowchart illustrating the work detection process in Modification Example 2 [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below, with reference to the drawings as appropriate. However, unnecessary details may be omitted. For example, detailed explanations of already well-known matters or redundant explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The inventors provide the accompanying drawings and the following explanation so that those skilled in the art can fully understand this disclosure, and do not intend to limit the subject matter described in the claims by means of these.

[0010] (Embodiment 1) 1. Structure The work analysis system according to Embodiment 1 will be described with reference to Figure 1. Figure 1 is a diagram showing an overview of the work analysis system 1 according to this embodiment.

[0011] 1-1. System Overview As shown in FIG. 1, this system 1 includes a camera 2 and a work analysis device 5. This system 1 is applied to the purpose of analyzing the efficiency etc. of an operator W who performs work such as transportation work in a workplace 6 such as a logistics warehouse. This system 1 may include a monitor 4 for presenting a work timeline 7 for a predetermined analysis period to a user 3 such as a manager or an analyst of the workplace 6. The analysis period is a period to be analyzed by image recognition etc. using the camera 2 in this system 1, and is preset, for example, from one day to several months.

[0012] The work timeline 7 shows the work of the operator W at each moment by classification, for example, into "target work" which is the work set as the analysis target of this system 1 and "non-work" which is other work. The work analysis device 5 in this example classifies work other than the target work as non-work in the work timeline 7 (in other words, non-work is also included in work).

[0013] In the example of FIG. 1, a conveyance line 61 and a shipping port 62 are installed in the workplace 6. In this example, the work performed by the operator W in the workplace 6 includes work such as the transportation work of packages and movement without carrying a package. In the work analysis system 1 of this embodiment, for example, the transportation work is preset as the target work.

[0014] According to the work analysis system 1 according to this embodiment, for example, by presenting the work timeline 7, the user 3 can analyze the work content of the operator W in chronological order in order to consider, for example, the business efficiency improvement of the workplace 6.

[0015] The camera 2 of this system 1 is arranged so that the operator W who performs work in the workplace 6 is reflected. The camera 2 repeats an imaging operation at a predetermined cycle in the workplace 6, for example, and generates image data indicating an imaging image. The camera 2 is connected to the work analysis device 5 so that the image data is transmitted to the work analysis device 5, for example. Although one camera 2 is illustrated in FIG. 1, the camera 2 included in this system 1 is not limited to one camera, and may be two or more cameras.

[0016] The work analysis device 5 is composed of information processing equipment, such as a server device. The work analysis device 5 is connected to an external information processing equipment, such as a PC including a monitor 4, in a communicative manner. The configuration of the work analysis device 5 will be explained using Figure 2.

[0017] 1-2. Configuration of the Working Analyzer Figure 2 is a block diagram illustrating the configuration of the work analysis device 5. The work analysis device 5 illustrated in Figure 2 comprises a control unit 50, a storage unit 52, an operation unit 53, an equipment interface 54, and an output interface 55. Hereinafter, interfaces will be abbreviated as "I / F".

[0018] The control unit 50 includes, for example, a CPU or MPU that works in cooperation with software to realize predetermined functions, and controls the overall operation of the work analysis device 5. The control unit 50 reads data and programs stored in the storage unit 52, performs various calculations, and realizes various functions. For example, the control unit 50 includes a work detection model 51 that realizes a work detection function by image recognition processing.

[0019] The work detection model 51 is configured using a pre-trained model of a neural network, such as a convolutional neural network. The work detection model 51 performs image recognition processing on the image data. The work detection model 51 outputs a detection region as a detection result, which indicates the area in the image where a pre-set detection target is visible. In this embodiment, the detection target of the work detection model 51 is set to a target task (or the worker performing it), such as a transport task. The detection region is defined, for example, by the horizontal and vertical positions on the image, and indicates an area that encloses the detection target in a rectangular shape.

[0020] In this embodiment, the work detection model 51 outputs, for example, an empty value as the detection result when the area to be detected cannot be recognized in the image (i.e., the performance of the transport work cannot be detected). The detection result may include, for example, information indicating the time the image was taken. The work detection model 51 can be obtained by supervised learning, for example, by using an image showing a worker performing a transport work while holding luggage as the correct answer, and as training data, image data associated with a correct answer label indicating the correct answer. The work detection model 51 may be configured with categories that subdivide the detection target, such as the target work, or categories separate from the target work.

[0021] The pre-trained model of the work detection model 51 is not limited to a neural network, but may be any machine learning model related to image recognition. The work detection model 51 is not limited to machine learning, but may employ any image recognition algorithm, for example, it may be configured to perform rule-based image recognition processing.

[0022] The control unit 50 executes a program that includes a set of instructions for realizing the functions of the work analysis device 5, for example. This program may be provided via a communication network such as the Internet, or it may be stored on a portable recording medium. The control unit 50 may also be equipped with internal memory as a temporary storage area for holding various data and programs.

[0023] The control unit 50 may be a dedicated electronic circuit or a hardware circuit such as a reconfigurable electronic circuit designed to realize a predetermined function. The control unit 50 may be composed of various semiconductor integrated circuits such as a CPU, MPU, GPU, GPGPU, TPU, microcontroller, DSP, FPGA, and ASIC.

[0024] The memory unit 52 is a storage medium that stores the programs and data necessary to realize the functions of the work analysis device 5. The memory unit 52 is composed of, for example, a hard disk drive (HDD) or a semiconductor storage device (SSD). For example, the memory unit 52 stores the above-mentioned programs and various information such as the parameters of the trained model of the work detection model 51.

[0025] The memory unit 52 may include temporary memory elements, for example, composed of DRAM or SRAM, and may function as a work area for the control unit 50. For example, the memory unit 52 may temporarily store image data received from the camera 2 and detection results from the work detection model 51.

[0026] The operation unit 53 is a general term for the operating components that receive user input. The operation unit 53 is composed of, for example, a keyboard, mouse, trackpad, touchpad, buttons, and switches, or a combination thereof. The operation unit 53 acquires various information input by the user.

[0027] The device interface 54 is a circuit for connecting external devices such as the camera 2 to the work analysis device 5. The device interface 54 communicates according to a predetermined communication standard. The predetermined standards include USB, HDMI (registered trademark), IEEE 1395, IEEE 802.11, Bluetooth (registered trademark), etc. The device interface 54 is an example of an acquisition unit that receives various information from an external device in the work analysis device 5. In the work analysis system 1, the work analysis device 5 acquires image data showing a video captured by the camera 2, for example, via the device interface 54.

[0028] Output I / F55 is a circuit for outputting information. Output I / F55 outputs video signals and other information to external display devices such as monitors and projectors, in accordance with standards such as HDMI, for example.

[0029] The configuration of the work analysis device 5 described above is just one example, and the configuration of the work analysis device 5 is not limited to this. The work analysis device 5 may be composed of various types of computers, including a PC (personal computer). In addition to the output I / F 55, or in its place, the work analysis device 5 may be equipped with a display unit, for example, a liquid crystal display or an organic EL display as a built-in display device. Furthermore, the work analysis method of this embodiment may be executed in a distributed computing environment.

[0030] Furthermore, the work analysis device 5 may have a configuration that allows it to communicate with an external information processing device via a communication network, in addition to or instead of the above configuration. For example, the operation unit 53 may be configured to accept operations from an external information processing device connected via a communication network. Also, the output I / F 55 may transmit various types of information to an external information processing device via a communication network.

[0031] Furthermore, the acquisition unit in the work analysis device 5 may be realized through cooperation with various software in the control unit 50, etc. The acquisition unit in the work analysis device 5 may acquire various information by reading various information stored in various storage media (for example, storage unit 52) ​​into the work area of ​​the control unit 50.

[0032] 2.Operation The operation of the work analysis system 1 and work analysis device 5, which are configured as described above, will be explained below.

[0033] The work analysis system 1 illustrated in Figure 1 uses image recognition processing to detect target tasks performed by worker W in the work area 6 at each given time. This system 1 stores information indicating the detection results of target tasks, and based on the stored information, generates a work timeline 7 that sequentially visualizes the times when worker W was detected to have performed the target tasks during the analysis period.

[0034] In the image recognition described above, for example, it is conceivable that even though worker W performed the target task at that time, the performance of the target task may not be detected because the characteristic parts necessary for detecting the target task are not visible in the captured image at that time. In contrast, the work analysis device 5 of this embodiment can accurately estimate the performance of the target task by worker W by correcting the detection results of the image recognition, even if there are times when the target task is not detected in the work analysis system 1 as described above.

[0035] 2-1. Regarding the challenges In the work analysis system 1 of this embodiment, the challenges in identifying the work of worker W at each time point will be explained using Figures 3 and 4. Below, an example will be described in which the target work is transportation work.

[0036] Figure 3 is a diagram illustrating the problems related to the work analysis device 5. Figure 3 shows an image Im captured by camera 2 of worker W performing a transport operation in the workplace 6. Figure 4 is a diagram illustrating the operation of the work analysis device 5. Figures 4(A) and 4(B) illustrate work timelines 70 and 7 before and after correction by the work analysis device 5, respectively, corresponding to the example in Figure 3. Each work timeline 70 and 7 is an example of work detection information in this embodiment.

[0037] Figure 3(A) shows a scene in the transportation operation where worker W takes a load from the transport line 61. Figure 3(B) shows a scene where worker W carries the load taken from the scene in Figure 3(A). Figure 3(C) shows a scene where worker W places the load carried from the scene in Figure 3(B) at the exit 62. Thus, the transportation operation includes the steps of "taking" the load from the transport line 61, "carrying" the load to the exit 62, and "placing" the load at the exit 62.

[0038] In the example shown in Figure 3(A), image recognition in the image Im captured from camera 2 identifies the detection area R1 where worker W "takes" the cargo, and thus the transport operation is detected. Accordingly, as shown in Figure 4(A), for example, in the work timeline 70 before correction, the transport operation is detected as the target operation up to time ta, which corresponds to the scene in Figure 3(A).

[0039] Subsequently, in the example shown in Figure 3(B), worker W is carrying a package, but the package is hidden by worker W's body in the line of sight of camera 2. In this way, when the package being carried by worker W is not visible in the captured image Im, the carrying operation may not be detected by image recognition. In this case, as illustrated in Figure 4(A), the work timeline 70 before correction will not detect the target operation during the period T1 which includes the time tb corresponding to Figure 3(B).

[0040] Furthermore, in the example shown in Figure 3(C), the cargo reappears in the captured image Im, and the detection area R2 of the process where worker W "places" the cargo is recognized, thus detecting the transport operation. Accordingly, in the uncorrected work timeline 70 in Figure 4(A), the target operation is detected again from time tc, which corresponds to the scene in Figure 3(C).

[0041] As described above, when detecting the work of worker W at each time interval in the workplace 6, there is a potential problem that, even though worker W is performing a transporting task, the image recognition may fail to detect the transporting task, meaning that there may be a period T1 during which the target task is not detected.

[0042] Therefore, in this embodiment, the work analysis device 5 focuses on the worker W's steps before and after the undetected period T1 described above, and performs a correction process to interpolate the detection results of the work detection model 51. For example, each step in Figures 3(A) to 3(C) is performed sequentially in a single transport operation and can therefore be used to interpolate the detection results. Each step is an example of the steps in the target operation of this embodiment in which the worker W starts moving an object such as a package, continues moving the object, and ends moving the object. Figure 4(B) shows the work timeline 7 corrected by the work analysis device 5 from the work timeline 70 in Figure 4(A).

[0043] For example, in the situation shown in Figure 3(B), since the process of placing the luggage followed as shown in Figure 3(C), it can be inferred that the luggage was being carried. The work analysis device 5 of this embodiment performs a correction process that reflects this inference, interpolating the detection result as if the target work was being performed during the undetected period T1, as illustrated in Figure 4(B), to obtain a corrected work timeline 7. On the other hand, for the period T2 in which worker W is not performing the target work, the work analysis device 5 of this embodiment performs a correction process to prevent the detection result from being incorrectly interpolated with the target work. This makes it possible to accurately detect the target work as a specific task performed by worker W.

[0044] 2-2. Overall Operation The overall operation of the work analysis device 5 in the work analysis system 1 will be explained using Figure 5.

[0045] Figure 5 is a flowchart illustrating the overall operation of the work analysis device 5. The processes shown in this flowchart are executed, for example, by the control unit 50 of the work analysis device 5.

[0046] First, the control unit 50 acquires image data from the camera 2 during the analysis period, for example via the equipment I / F 54 (S1). For example, while worker W is working in the workplace 6, the camera 2 captures a video and generates image data showing the captured image at each time point at a predetermined period, such as the video frame period, and records it in internal memory. The camera 2 transmits the image data recorded during the analysis period to the work analysis device 5. The control unit 50 stores the acquired image data in, for example, the storage unit 52.

[0047] Next, the control unit 50 selects, for example in chronological order, one frame of image data representing the image captured at each time point from the image data acquired during the analysis period (S2). The control unit 50 records the time at which the selected frame was captured as, for example, the time on each work timeline 70, 7.

[0048] The control unit 50 detects a transport operation based on the image data of a selected frame by performing image recognition processing on the operation detection model 51 (S3). The control unit 50 stores the detection result in the storage unit 52, for example. In this operation detection process (S3), the control unit 50 interpolates past detection results based on the detection result of the frame, assuming that the target operation was detected during the immediately preceding undetected period T1.

[0049] In the work detection process of this embodiment, the work detection model 51 is made to distinguish and detect multiple steps in the transport operation, and correction processing of the detection result is performed. For example, the work detection model 51 of this embodiment is constructed using machine learning to output the detection result of the transport operation by associating the detection area with one of a set of multiple categories. The multiple categories include each step in the transport operation, such as "taking," "carrying," and "placing." For example, training data in which images of each category's steps are considered correct is used to train the work detection model 51. Details of the work detection process (S3) of this embodiment using such a work detection model 51 will be described later.

[0050] If the control unit 50 has not yet selected all frames in the image data for the analysis period (NO in S4), it repeats the processing in steps S1 to S3 for the image data at the next time point.

[0051] If all frames in the analysis period are selected (YES in S4), the control unit 50 performs a visualization process (S5) to generate a work timeline 7 that is visualized to the user from the detection results for each frame. In step S5, a corrected work timeline 7 is generated based on the detection results interpolated in the work detection process (S3).

[0052] The control unit 50 stores the work timeline 7 generated by, for example, the visualization process (S5) in the storage unit 52, and terminates the process shown in this flowchart.

[0053] Based on the above processing, the work performed by worker W at each time step is detected (S2-S4) based on the image data, and the detection results during the undetected period T1 are interpolated with the target work (S3). This allows for the accurate acquisition of a work timeline 7 from the detection results at each time step.

[0054] In step S1 described above, image data generated from camera 2 may be acquired sequentially. For example, instead of step S4, the control unit 50 may repeat the processing from step S1 onward until it obtains a detection result of the work to be performed based on the image data of the number of frames during the analysis period.

[0055] Furthermore, in the visualization process (S5), the control unit 50 may generate an analysis chart showing the ratio of target work to non-work for worker W during the analysis period, in addition to or instead of the work timeline 7. For example, for worker W, the control unit 50 counts the number of target work and non-work based on the detection results of transport work at each time interval, such as the frame cycle. Once the total number of each work during the analysis period is calculated for each worker in this way, the control unit 50 calculates the ratio of each work for each worker and generates an analysis chart. In the analysis chart, the ratio of each work is shown, for example, as the proportion of time each work takes to the analysis period.

[0056] 2-3. About the work detection process The details of the work detection process in step S3 of Figure 5 will be explained using Figures 6 and 7.

[0057] Figure 6 is a flowchart illustrating the work detection process (S3 in Figure 5) in the work analysis device 5 of this embodiment. Figure 7 is a diagram illustrating the work detection process.

[0058] In the flowchart of Figure 6, first, the control unit 50 performs image recognition processing by the work detection model 51 on the captured image Im shown by the 1 frame image data selected in step S2 of Figure 5 to detect the work (S11). For example, in the captured image Im of Figure 3(A), a detection result is obtained that includes the detection region R1 in association with the process of "taking" the package (S11).

[0059] Next, the control unit 50 determines whether or not the target operation was detected in the current frame based on the obtained detection results (S12). In the example in Figure 3(A), it is determined that the target operation was detected based on the detection results including the detection region R1 (YES in S12).

[0060] If the target operation is detected (YES in S12), the control unit 50 determines, based on the detection result, whether the detected transport operation is "place" or not (S13). In the example in Figure 3(A), the detected transport operation is "take" and not "place," so the control unit 50 proceeds to NO in step S13.

[0061] If the control unit 50 determines that the current step is not "placing" (NO in S13), it stores the time the target operation was detected based on the current detection result (S15). The time the target operation was detected is recorded, for example, by the image capture time of the frame in which the transport operation was detected. In the example of Figure 3(A), the control unit 50 stores, for example, the time ta in the storage unit 52 when the "taking" step was detected. In step S15, the control unit 50 may also store the detected step in association with the time ta when the transport operation was detected.

[0062] After the control unit 50 stores the detection result in the memory unit 52, for example (S16), it terminates the work detection process (S3) shown in this flowchart for the frame selected in step S2. Then, the control unit 50 returns to the process in the flowchart of Figure 5 and proceeds to step S4. For example, after the work detection process (S3) is performed in the frame of Figure 3(A), the process proceeds to NO in step S4, and the work detection process (S3) is performed again by selecting a frame at time tb corresponding to the scene in Figure 3(B) (S2).

[0063] In the example in Figure 3(B), the image recognition in step S11 does not detect the "carrying" process of the cargo handling operation, and therefore it is determined that the target operation is not detected (NO in S12). In this way, even if the target operation is being performed, it may not be detected, which can result in an undetected period T1 as illustrated in the work timeline 70 before correction in Figure 4(A).

[0064] If no target operation is detected (NO in S12), the control unit 50 does not perform the processes in steps S13 to S15, stores the detection result in the storage unit 52 (S16), and proceeds to step S4 in Figure 5. For example, after the operation detection process (S3) in the frame of Figure 3(B), the frame at time tc corresponding to the scene in Figure 3(C) is selected (NO in S4, S2).

[0065] In the example shown in Figure 3(C), the control unit 50 determines that the target work has been detected based on the detection area R2 acquired by the work detection (S11) (YES in S12). The control unit 50 then proceeds to step S13 with YES based on the detection result obtained, which associates the "placing" process of the luggage with the detection area R2.

[0066] If the control unit 50 determines that the current step is "place" (YES in S13), it refers to the time of the target operation stored in the immediately preceding step S15 and interpolates the detection results for the period from the time of the current frame to the time of the immediately preceding target operation with the target operation (S14). In the example in Figure 4(A), the detection results are interpolated assuming that the target operation was detected during the undetected period T1 from the time tc when the "place" step was detected in the current frame to the time ta when the immediately preceding "take" step was detected. In this way, when the "place" step, which completes one transport operation, is detected, it can be presumed that the transport operation was performed from the time ta when the "take" step, which starts the transport operation, was detected, and therefore the detection results for the undetected period T1 are interpolated with the target operation.

[0067] Next, the control unit 50 stores the time tc at which the "place" step was detected as the time when the target operation was detected (S15). After that, the control unit 50 stores the detection result (S16) and proceeds to step S4 in Figure 5.

[0068] Unlike the example in Figure 3, Figure 7 illustrates an image Im that includes a period during which worker W is not performing transport work. In this embodiment, the work analysis device 5 performs work detection processing (S3) so as not to incorrectly interpolate the detection results during periods when worker W is not performing transport work. This point will be explained using Figure 7.

[0069] Figure 7(A) shows a scene similar to that in Figure 3(C), where worker W places the cargo at the exit 62. In the uncorrected work timeline 70 in Figure 4(A), the target work is detected after time tc, for example, according to the captured image Im in Figure 7(A).

[0070] Figure 7(B) shows the scene from Figure 7(A) where worker W, who has placed the load, turns towards the transport line 61. In Figure 7(B), worker W is not carrying the load and is not performing any transport work. In this case, as illustrated in Figure 4(A), a period T2 occurs during which the target work is not detected.

[0071] Figure 7(C) shows worker W turning around from the scene in Figure 7(B) and taking the next package from the conveyor line 61. In the work timeline 70 of Figure 4(A), the target work is detected again after period T2.

[0072] First, when the frame in Figure 7(A) is selected (S2), the control unit 50 executes the processes in steps S11 to S16, similar to the example in Figure 3(C). In the example in Figure 7(A), the time when the "place" step is detected is stored as the time when the target operation is detected (S15). After that, the control unit 50 selects the next frame in Figure 7(B) (NO in S4, S2).

[0073] In the example shown in Figure 7(B), worker W is not performing a carrying task, and the target task is not detected in step S11 (NO in S12). In this example, the control unit 50 proceeds to step S16, stores the detection result, and then proceeds to step S4 in Figure 5, which is NO.

[0074] Next, when the frame shown in Figure 7(C) is selected (S2), the control unit 50 detects the process of "taking" the load (S11). The control unit 50 determines that the target operation has been detected (YES in S12) and proceeds to step S13. Since the current process is not "placing" (NO in S13), the control unit 50 does not perform any interpolation processing (S14) and proceeds with the processing from step S15 onwards. As a result, as shown in Figures 4(A) and (B), the detection result is not incorrectly interpolated for the period T2 when the worker W is not performing the target operation.

[0075] According to the above work detection process (S3), work is detected at each time (S11), and if the target work is detected (YES in S12), the time is stored (S15). If the current step is the "place" step which completes the transport work (YES in S13), the detection results from the time of the current target work to the time of the immediately preceding target work stored are interpolated with the target work (S14). This allows the detection results to be interpolated as if the target work had been detected during an undetected period T1, such as when worker W is transporting goods but is not detected.

[0076] On the other hand, if the process in this case is anything other than "placing" (NO in S13), the detection result is not interpolated even if the target operation is not detected. As a result, for example, in the work timeline 70 before correction in Figure 4(A), the detection result is not interpolated during the period T2 in which it is assumed that the target operation is not being performed, such as when worker W turns from the exit 62 to the transport line 61 without holding the load, as shown in Figure 7(B). As a result, a work timeline 7 that has been corrected with high accuracy is obtained based on the interpolated detection result, as illustrated in Figure 4(B).

[0077] Furthermore, if the "carrying" process is detected in step S11 above, the control unit 50 stores the time the target operation was detected (S15), similar to the case where the "taking" process is detected in the example of Figure 3(A). In this case, in step S15, the control unit 50 stores the time the "carrying" process was detected in the storage unit 52, separately from the time the "taking" process was detected. Next, if the "placing" process is detected (YES in S13), the control unit 50 interpolates the detection results for the period up to the time the previous "taking" process was detected (S14). This allows for accurate interpolation of the detection results even if the target operation is detected in the middle of the undetected period T1.

[0078] Furthermore, when storing the detection results (S16), the control unit 50 may not store the category of each step of the transport operation, but only whether it is a transport operation or not, that is, whether it is a target operation or a non-operation.

[0079] 2-3-1. Modified Examples of Embodiment 1 In Embodiment 1, the work analysis device 5 performed interpolation processing (S14) when the current step was "place" (YES in S13). However, interpolation processing may also be performed depending on whether the current step is the "take" step, which starts the transport operation. Such modifications will be explained with reference to Figure 8.

[0080] Figure 8 is a flowchart illustrating the work detection process in the work analysis device 5 according to a modified example of Embodiment 1. The control unit 50, for example, instead of the process in step S13 of Figure 6, determines whether the current process is "take" based on the detection result obtained in step S11 (S13A).

[0081] If the current step is "take" (YES in S13A), the control unit 50 does not execute the process in step S14 and stores the time when the step was detected as the time of the target operation (S15). If the current step is not "take" (NO in S13A), that is, if a step in the middle or at the end of a transport operation is detected, the control unit 50 interpolates the detection results from the time of the current detection result to the time of the previous target operation with the target operation (S14).

[0082] With this type of work detection process, when the process of "carrying" the cargo is detected in step S11, the target work is detected (YES in S12) and the current process is not "taking" (NO in S13), so the process proceeds to step S14. In this case, the detection results from the time the "carrying" process was detected up to the immediately preceding target work are interpolated (S14). As a result, for example, even if the "carrying" process is detected at a time in the middle of the undetected period T1, and then the "placing" process is detected afterward, in step S14, the detection results for the undetected period T1 can be interpolated by referring to the time of the immediately preceding target work.

[0083] 3. Effects, etc. As described above, the work analysis device 5 in this embodiment generates information about the work performed by worker W. The work analysis device 5 includes an equipment I / F 54 as an example of an acquisition unit and a control unit 50. The equipment I / F 54 acquires image data showing an image of worker W performing the work (S1). Based on the image data, the control unit 50 sequentially detects target work as an example of work performed by worker W (S2-S4) and generates a work timeline 7 as an example of work detection information showing the detection result of the target work (S5). The target work includes multiple steps performed by worker W. The control unit 50 corrects the work detection information, such as the work timeline 70 before correction, to interpolate the detection result of the target work in the undetected period T1, according to the steps taken when the target work is detected after an undetected period in which the target work was not detected (S3).

[0084] According to the work analysis device 5 described above, if an undetected period T1 occurs in which the performance of the target work is not detected, the detection results for the undetected period T1 are interpolated according to the process in which the target work was detected before and after the undetected period T1, and a corrected work timeline 7 is obtained (S3, S5). This makes it possible to accurately detect the target work as an example of a specific work performed by worker W.

[0085] In this embodiment, the multiple steps include a step of picking up a package as an example of a first step of starting the target work, a step of carrying the package as an example of a second step of continuing the target work, and a step of placing the package as an example of a third step of ending the target work. The control unit 50 does not interpolate the detection results of the target work during the undetected period if the step after the undetected period is the first step. For example, as in the example in Figure 7, the control unit 50 does not interpolate the detection results of the work during period T2 if the "pick up" step is detected when the target work is detected (YES in S13A) or if the "place" step is not detected (NO in S13).

[0086] Thus, in the period T2 where the target work is not detected up to the "take" step that starts the target work, it can be assumed that the target work is not being performed, and therefore the detection results for period T2 are not interpolated. As a result, a work timeline 7 that has been corrected with high accuracy is obtained without incorrectly interpolating the detection results for periods when the target work is not being performed.

[0087] In this embodiment, the control unit 50 interpolates the detection result of the target work during the undetected period in at least one of the cases where the process after the undetected period is the second process, or the process after the undetected period is the third process. For example, as in the example in Figure 3, the control unit 50 interpolates the detection result of the target work during the undetected period T1 in at least one of the cases where the "place" process is detected when the target work is detected (YES in S13) or where the "take" process is not detected (NO in S13A) (S14). As a result, it can be estimated that the target work was performed during the undetected period T1 up to the process in which the target work is continued or terminated, and thus the detection result of the target work can be interpolated.

[0088] In this embodiment, the target operation is a transport operation, which involves moving a load as an example of an object. The transport operation includes a first step of picking up the load (an example of a step to start the movement of an object), a second step of carrying the load (an example of a step to continue the movement of an object), and a third step of placing the load down (an example of a step to end the movement of an object). Note that the object in the transport operation is not limited to a load, but may be various items such as parts. The target operation is not limited to a transport operation, but may also be an assembly operation, which involves attaching parts. The assembly operation includes a first step of picking up parts (an example of a step to start the attachment of an object), a second step of carrying the parts (an example of a step to continue the attachment of an object), and a third step of attaching the parts (an example of a step to end the attachment of an object).

[0089] In this embodiment, the control unit 50 identifies the steps performed by the worker W when the target work is detected based on the image data (S11), and corrects the work detection information, such as the work timeline 70 before correction, based on the step identification result (S12-S16). For example, work detection processing (S3) can be performed with high accuracy based on the detection result of a work detection model 51 that uses machine learning to identify each step of the transportation work as each step of the target work.

[0090] The work analysis method in this embodiment is a method for generating information about work performed by worker W. This method includes the steps of: (S1) the computer control unit 50 acquiring image data showing an image of worker W performing work; and (S2-S5) sequentially detecting target work as an example of work performed by worker W based on the image data, and generating a work timeline 7 as an example of work detection information showing the detection results of the target work. The computer control unit 50 corrects the work detection information, such as the work timeline 70 before correction, to interpolate the detection results of the target work in the undetected period T1, according to the process when the target work is detected after an undetected period in which the target work was not detected (S3).

[0091] In this embodiment, a program is provided to cause the control unit of a computer to execute the work analysis method described above. According to the work analysis method of this embodiment, it is possible to accurately detect target tasks, such as transportation work, as an example of a specific task performed by worker W.

[0092] (Embodiment 2) In Embodiment 1, a work analysis device 5 was described that performs interpolation processing by machine learning that identifies each step of the transport operation using a work detection model 51. However, such machine learning is not essential. In Embodiment 2, a work analysis device 5 is described that performs interpolation processing by using the movement direction of the worker W corresponding to a specific step in the transport operation.

[0093] Hereinafter, descriptions of the configuration and operation similar to those of the work analysis device 5 according to Embodiment 1 will be omitted as appropriate, and the work analysis device 5 according to this embodiment will be described.

[0094] The operation of the work analysis device 5 of this embodiment will be explained using Figures 9 and 10. Figure 9 is a diagram illustrating the work detection process in the work analysis device 5 of this embodiment. Figure 10 is a flowchart illustrating the work detection process of this embodiment.

[0095] Figure 9(A) shows an image Im of worker W carrying goods from the transport line 61 to the discharge port 62, similar to Figure 3(B). Figure 9(B) shows an image Im of worker W turning back from the discharge port 62 to the transport line 61, similar to Figure 7(B).

[0096] As illustrated in Figures 9(A) and (B), it is assumed that worker W moves in different directions depending on whether or not he is in the middle of a cargo handling operation. Therefore, in the work analysis device 5 of this embodiment, a predetermined direction corresponding to the process by which worker W carries cargo in the work area 6 is set in advance, for example, on the captured image Im. In this embodiment, the predetermined direction is set to be the direction from the transport line 61 to the outlet 62.

[0097] In the flowchart of Figure 10, the control unit 50 determines whether the direction of movement of the worker W is in a predetermined direction (S13B), instead of determining the process identified by the work detection model 51 in the work detection process (S3) of Embodiment 1 (S13). The control unit 50, for example, detects a transport operation based on image data (S11), and determines the direction of movement of the worker W based on the position of the detection area in the acquired detection result.

[0098] For example, if the control unit 50 detects the target work (YES in S12), it compares the position of the previously detected area with the position of the currently detected area to determine whether the worker W is moving in a predetermined direction (S13B).

[0099] In the example shown in Figure 9(A), worker W is moving in a predetermined direction, and it can be inferred that worker W is carrying a load. If the control unit 50 determines that worker W is moving in the predetermined direction (YES in S13B), it interpolates the detection results from the time of the current target operation to the time of the previous target operation using the target operation (S14).

[0100] On the other hand, in the example shown in Figure 9(B), worker W is moving in a direction different from the predetermined direction, and it can be inferred that this is not a process of carrying cargo. If the control unit 50 determines that the direction of worker W's movement is not the predetermined direction (NO in S13B), it proceeds to step S15 without interpolating the detection result.

[0101] As described above, in this embodiment, the control unit 50 determines, based on the image data, whether the direction in which the worker W moved when the target work was detected (YES in S12) is a predetermined direction (S13B), and corrects the work detection information, such as the work timeline 70 before correction, based on the direction determination result (S14-S16). The predetermined direction is set on the captured image Im as an example of the image shown by the image data, and corresponds to the process of carrying luggage as an example of the second process.

[0102] Furthermore, in this embodiment, if the control unit 50 determines that the direction of movement of the worker W is in a predetermined direction (YES in S13B), it interpolates the detection results of the target work during the undetected period T1 (S14). If the control unit 50 determines that the direction of movement of the worker W is not in a predetermined direction (NO in S13B), it does not interpolate the detection results of the target work during an undetected period such as period T2.

[0103] Thus, in a workplace 6 where tasks involving regular movements, such as transporting goods, are performed, it is possible to estimate whether or not worker W was performing the target task, such as transporting goods, based on rules such as the direction of worker W's movement. As a result, in this embodiment, it is sufficient for the work detection model 51 to detect transporting work, and the identification of each stage of the transporting work can be omitted. With the work analysis device 5 of this embodiment, the detection results for the undetected period T1 can also be interpolated to obtain an appropriately corrected work timeline 7 (S5).

[0104] In the above explanation, an example was described in which the direction of movement of worker W is determined based on the position of the detection area by the work detection model 51. However, the work analysis device 5 may also use an image recognition model that recognizes the position of a person such as worker W in the captured image Im, in addition to the work detection model 51. In this case, the control unit 50 may acquire the detection results of the work detection model 51 and the image recognition model in step S11, and in step S13B, it may compare the position of worker W recognized in the previous and current steps to determine the direction of movement of worker W. Alternatively, the work detection model 51 may be machine-learned to output the detection area of ​​worker W who is not performing the target work as a separate category from the detection area of ​​worker W who is performing the target work.

[0105] (Embodiment 3) Embodiment 2 describes a work analysis device 5 that performs interpolation processing using the movement direction of worker W corresponding to a specific step in the target work in the work detection process. Embodiment 3 describes a work analysis device 5 that performs interpolation processing using the region in the workplace 6 where worker W is located corresponding to a specific step.

[0106] Hereinafter, descriptions of the configuration and operation similar to those of the work analysis device 5 in Embodiments 1 and 2 will be omitted as appropriate, and the work analysis device 5 according to this embodiment will be described.

[0107] The operation of the work analysis device 5 of this embodiment will be explained using Figures 11 and 12. Figure 11 is a diagram illustrating the work detection process in the work analysis device 5 of this embodiment. Figure 12 is a flowchart illustrating the work detection process of this embodiment.

[0108] Figure 11 shows an image Im captured of worker W turning around from the exit 62 to pick up a load on the transport line 61. Figure 11 illustrates a first exclusion region 81 that is pre-set as an area on the image Im in the work analysis device 5 of this embodiment. The first exclusion region 81 is set, for example, as an area corresponding to the position where worker W picks up a new load from the transport line 61, i.e., the position where the transport work begins, when worker W performs transport work in the work area 6. The first exclusion region 81 is an example of a predetermined region in this embodiment.

[0109] As shown in Figure 11, if the detection area R1 where the transport operation was detected is included in the first exclusion area 81, it can be estimated that worker W will perform the process of picking up the cargo, that is, the process of starting a new transport operation. In this case, it is assumed that worker W did not have any cargo during the period before the transport operation was detected. According to the work analysis device 5 of this embodiment, if the transport operation is detected within the first exclusion area 81, it is assumed that no transport operation was performed during the period immediately preceding when the target operation was not detected, so the detection result is not interpolated. On the other hand, if the transport operation is detected in an area different from the first exclusion area 81, it can be estimated that worker W will perform the process of continuing or ending the transport operation, so the detection result of the immediately preceding undetected period T1 is interpolated.

[0110] In the flowchart of Figure 12, the control unit 50 determines whether the current detection area is within the first exclusion area 81 (S13C), instead of determining the direction of movement of the worker W in the work detection process of Embodiment 2 (S13B).

[0111] If the target work is detected (YES in S12), the control unit 50 determines whether the detected area is included in the first exclusion area 81, that is, whether it is within the first exclusion area 81, based on the position of the detected area in the work detection result (S11) (S13C). The control unit 50 determines that the detected area is included in the first exclusion area 81 if, for example, the proportion of the detected area that overlaps with the first exclusion area 81 in the captured image Im is greater than or equal to a predetermined proportion. The predetermined proportion is set in advance as a proportion large enough to be considered as including a part of the worker W's body in the first exclusion area 81 (for example, 80%).

[0112] Alternatively, the decision in step S13C may be made depending on whether the center of the detection area falls within the first exclusion area 81, or by converting the coordinates indicating the position on the captured image Im into coordinates on the map of the workspace 6, and then determining the positional relationship between the detection area and the area on the map corresponding to the first exclusion area 81.

[0113] If the detection area in the current step is within the first exclusion area 81 (YES in S13C), the control unit 50 does not interpolate the detection results up to the time of the previous target operation (S14) and proceeds to step S15. In the example in Figure 11, the control unit 50 determines that the detection area R1 is included in the first exclusion area 81 (YES in S13C) and does not perform the interpolation process (S14).

[0114] On the other hand, if the detected area is not within the first exclusion area 81 (NO in S13C), the control unit 50 performs interpolation processing (S14).

[0115] As described above, in this embodiment, the control unit 50 determines, based on the image data, whether the detection area R1, defined by a position on the image as an example of the detection position where the worker W was located when the target work (an example of work) was detected, is included in the first exclusion area 81 as an example of at least one predetermined area (S13C), and corrects the work timeline 7 (an example of work detection information) based on the determination result of the detection position. The at least one predetermined area is set on the captured image Im as an example of the image shown by the image data, corresponding to at least one of the first and third processes. For example, the first exclusion area 81 is set as an example of the first process, corresponding to the process of picking up the cargo. This makes it possible to correct the work timeline 7 based on the detection position without specifically using the identification results of each process of the transport work by the work detection model 51.

[0116] Furthermore, in this embodiment, the control unit 50 recognizes a detection area R1 as an example of a detection position based on the image data (S11), and determines whether the detection position recognized after the undetected period is included in a first exclusion area 81 (an example of a predetermined area) that corresponds to the process of picking up the package (an example of a first process) in the captured image Im as an example of an image shown by the image data (S13C). If the detection position after the undetected period is not included in the first exclusion area 81 (NO in S13C), the control unit 50 interpolates the detection result of the target work in the undetected period T1 (S14). If the detection position after the undetected period is included in the first exclusion area 81 (YES in S13C), the control unit 50 does not interpolate the detection result of the target work in an undetected period such as period T2. In this way, for example, if the detection position is included in the first exclusion area 81, it can be estimated that the process after the undetected period corresponds to the process of picking up the package, i.e., the process of starting the work, and thus a work timeline 7 is obtained that is corrected with high accuracy without incorrect interpolation of the detection result in period T2.

[0117] (Embodiment 4) Embodiment 3 describes a work analysis device 5 that uses a region corresponding to the process of starting the transport operation for interpolation in the work detection process. Embodiment 4 describes a work analysis device 5 that uses a region corresponding to the process of ending the transport operation in addition to the starting process.

[0118] The following description of the work analysis device 5 according to this embodiment will be omitted as appropriate, as it will be similar to the work analysis device 5 in Embodiments 1 to 3.

[0119] The operation of the work analysis device 5 of this embodiment will be explained using Figures 13 and 14. Figure 13 is a diagram illustrating the work detection process in the work analysis device 5 of this embodiment. Figure 14 is a flowchart illustrating the work detection process of this embodiment. Figure 13(A) shows an image Im of worker W placing a load at the discharge port 62, similar to Figure 7(A). Figure 13(B) shows an image Im of worker W taking a load from the transport line 61 after placing the load in the scene shown in Figure 13(A).

[0120] Figures 13(A) and (B) illustrate the work analysis device 5 of this embodiment, which, in addition to the first exclusion region 81 set in the same manner as the work analysis device 5 of Embodiment 3, also includes a second exclusion region 82 that is pre-set as an area on the captured image Im. The second exclusion region 82 is set, for example, as an area corresponding to the position where a worker W, when performing a transport operation in the workplace 6, places a load taken from the transport line 61 and transported at the discharge port 62, i.e., the position where the transport operation ends. The first exclusion region 81 and the second exclusion region 82 are examples of predetermined areas in this embodiment, and are examples of the start region and end region, respectively.

[0121] As shown in Figure 13(A), if the detection area R2 where the transport operation was detected is included in the second exclusion area 82, it is presumed that worker W will perform the step of placing the load, that is, the step of completing one transport operation. Subsequently, as shown in Figure 13(B), if the detection area R1 is included in the first exclusion area 81, it can be presumed that worker W will perform the step of picking up the next load, that is, the step of starting the next transport operation. In this case, during the period of transition from Figure 13(A) to Figure 13(B), it is considered that worker W performed an action that was not the target operation, such as turning from the exit 62 to the transport line 61 without holding any load.

[0122] Thus, if a transport operation is detected within the second exclusion area 82 corresponding to the completion of the operation, and then detected within the first exclusion area 81 corresponding to the start of the operation, it is considered that no transport operation was performed during the period in which the target operation was not detected. In this case, the work analysis device 5 of this embodiment does not interpolate the detection results.

[0123] On the other hand, except in cases where it is presumed that the transport operation is transitioning from completion to commencement as described above, it is presumed that worker W was performing one of the steps of the transport operation during the undetected period T1 immediately preceding the detection of the transport operation. In this case, the work analysis device 5 interpolates the detection results for the undetected period T1.

[0124] In the flowchart of Figure 14, the control unit 50, instead of determining whether the current detection area is within the first exclusion area 81 (S13C) in the work detection process of Embodiment 3, determines whether the current detection area is within the first exclusion area 81 AND the detection area of ​​the previous target work is within the second exclusion area 82 (S13D).

[0125] In this embodiment, when a target operation is detected (YES in S12), the control unit 50 determines whether the current detection area is within the first exclusion area 81 and the detection area of ​​the previous target operation is within the second exclusion area 82 (S13D). The control unit 50 identifies the detection area of ​​the previous target operation by referring, for example, the time of the previous target operation stored in step S15 and the past detection results stored in step S16. The control unit 50 determines whether the current and previous detection areas are within the first exclusion area 81 and the second exclusion area 82, respectively, using the same criteria as in step S13C in the operation detection process of Embodiment 3.

[0126] For example, when work detection processing is performed on the captured image Im in Figure 13(B), the control unit 50 determines whether the current detection area R1 is within the first exclusion area 81 and whether the detection area R2 of the immediately preceding target work, as illustrated in Figure 13(A), is within the second exclusion area 82 (S13D). In the examples of Figures 13(A) and (B), the detection area R1 is within the first exclusion area 81 and the detection area R2 is within the second exclusion area 82 (YES in S13D), and the control unit 50 proceeds to step S15 without performing interpolation processing (S14).

[0127] On the other hand, if the detection area in the current detection is not included in the first exclusion area 81, or if the detection area of ​​the previous target operation is not included in the second exclusion area 82 (NO in S13D), the control unit 50 performs interpolation processing (S14).

[0128] As described above, in this embodiment, at least one predetermined region includes a first exclusion region 81 corresponding to the process of picking up luggage, as an example of a start region corresponding to a first process, and a second exclusion region 82 corresponding to the process of placing luggage, as an example of an end region corresponding to a third process. If the control unit 50 determines that at least one of the following is true (NO in S13D): the previous detection region is not included in the second exclusion region 82 as an example of a detection position before the undetected period, or the current detection region is not included in the first exclusion region 81 as an example of a detection position after the undetected period, then it interpolates the detection result of the target work (an example of work) during the undetected period T1 (S14). If the control unit 50 determines that detection region R2 is included in the second exclusion region 82 as an example of a detection position before the undetected period, and detection region R1 is included in the first exclusion region 81 as an example of a detection position after the undetected period (YES in S13D), then it does not interpolate the detection result of the target work during an undetected period such as period T2.

[0129] This allows the work detection process to estimate the steps before and after the undetected period based on the current and previous detection positions. This makes it possible to accurately determine whether or not to perform interpolation processing (S14) without specifically using the identification results of each step of the target work by the work detection model 51. In this embodiment, the control unit 50 determines whether the detection area of ​​the previous target work, as an example of a detection position recognized before the undetected period, is included in the second exclusion area 82 (an example of an end area) in the captured image Im, as an example of an image represented by the image data, and whether the current detection area, as an example of a detection position recognized after the undetected period, is included in the first exclusion area 81 (an example of a start area) (S13D).

[0130] (Other embodiments) As described above, Embodiments 1 to 4 have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited thereto and can be applied to embodiments that have been modified, substituted, added, or omitted as appropriate. Furthermore, it is possible to create new embodiments by combining the components described in each of the above embodiments. Therefore, other embodiments are described below as examples.

[0131] In the above embodiment 1, a work analysis device 5 was described that performs interpolation processing by determining the process after the undetected period in the work detection process (S3) (S13, S13A). The work analysis device 5 of this embodiment may further perform interpolation processing according to the process before the undetected period. For example, if the control unit 50 of the work analysis device 5 holds information about the process along with the time of the target work in step S15, even if an undetected period occurs, when the target work is detected afterward, it can perform the above processing by referring to the information from the previous detection of the target work. Such modifications will be explained with reference to Figures 15 and 16.

[0132] Figure 15 illustrates a flowchart of the work detection process in this modified example. In this modified example, in addition to the same processing as in Figure 8, when a target work is detected (YES in S12), the control unit 50 determines whether the step of the target work detected immediately before is "place" (S21). The control unit 50 performs the processing in step S21 by referring to the step information held in step S15 of the immediately preceding work detection process, for example. In this modified example, if the immediately preceding step is "place" (YES in S21) and the current step is "take" (YES in S13A), the control unit 50 does not interpolate the undetected period such as period T2. The control unit 50 interpolates the undetected period T1 in at least one of the cases where the immediately preceding step is not "place" (NO in S21) and where the current step is not "take" (NO in S13A) (S14).

[0133] In the above modified example, an example was described in which interpolation processing is performed not only according to the process before the undetected period, i.e., the process before the undetected period, but also according to the process after the undetected period occurred. In this modified example, the work analysis device 5 may perform interpolation processing according to the process before the undetected period without specifically determining the process after the undetected period. Figure 16 illustrates a flowchart of the work detection process in this modified example. In this modified example, the control unit 50 makes the same determination as in step S21 in Figure 15, and if the immediately preceding process is "place" (YES in S21), it does not interpolate the detection result of an undetected period such as period T2. If the immediately preceding process is not "place" (NO in S21), i.e., if the immediately preceding process is "take" or "carry", the control unit 50 interpolates the undetected period T1.

[0134] As described above, in this embodiment, the work analysis device 5 has the same configuration as in each of the above embodiments, and the control unit 50 may correct the work detection information, such as the work timeline 70 before correction, so as to interpolate the detection result of the target work in the undetected period T1, according to the process when the target work is detected before or after the undetected period in which the target work was not detected. This also makes it possible to accurately detect the target work as an example of a specific work performed by worker W.

[0135] Furthermore, in this embodiment, the control unit 50 may not interpolate the detection results during an undetected period such as period T2 if the step before the undetected period is the third step and the step after the undetected period is the first step. Also, the control unit 50 may interpolate the detection results of the work during the undetected period T1 in at least one of the cases where the step before the undetected period is the first or second step, and the step after the undetected period is the second or third step. Also, the control unit 50 may not interpolate the undetected period if the step before the undetected period is the third step, but may interpolate the undetected period if the step before the undetected period is the first or second step.

[0136] In each of the embodiments described above, during the work detection process, interpolation processing (S14) was performed according to the process (S13) at the time the target work was detected. In this embodiment, the work analysis device 5 may, for example, in step S13, perform interpolation processing (S14) according to the period from that time to the time of the previous target work, in addition to the process at that time. For example, interpolation may be performed if the period is shorter than a predetermined threshold. The predetermined threshold is set in advance as a period short enough to be considered as the period required for worker W to carry the cargo.

[0137] In each of the embodiments described above, an example was explained in which the target work of the work analysis system 1 is one type of transport work. The work analysis system 1 of this embodiment may be applied to multiple types of work as target work. For example, when detecting multiple types of transport work that transport different types of objects, interpolation processing may be performed on the detection results of each transport work in the same manner as in each of the embodiments described above.

[0138] In the embodiments described above, an example was explained in which the target work of the work analysis system 1 is a transportation operation. In this embodiment, the target work of the work analysis system 1 is not limited to a transportation operation, but may be any operation that includes multiple steps.

[0139] Furthermore, in each of the embodiments described above, an example was given in which the work analysis system 1 is applied to a workplace 6 such as a logistics warehouse. In this embodiment, the workplace, or site, to which the work analysis system 1 and work analysis device 5 are applied is not limited to the aforementioned workplace 6, but may be various sites such as a factory or a store's sales floor. Also, the transportation work to be analyzed by the work analysis system 1 is not limited to the example described above, but may include various processes depending on the type of site. In addition, the worker to be analyzed by the work analysis system 1 is not limited to a person such as worker W, but may be any mobile body capable of performing various tasks. For example, the mobile body may be a robot, or various manned or unmanned vehicles.

[0140] As described above, embodiments have been explained as examples of the technology in this disclosure. For this purpose, accompanying drawings and a detailed description have been provided.

[0141] Therefore, the components described in the attached drawings and detailed descriptions may include not only components essential for solving the problem, but also components that are not essential for solving the problem, provided that they illustrate the technology described above. For this reason, the mere presence of these non-essential components in the attached drawings and detailed descriptions should not be immediately assumed to mean that they are essential. [Industrial applicability]

[0142] This disclosure is applicable to data analysis applications for analyzing worker performance in various environments, such as logistics sites or factories.

Claims

1. A work analysis device that generates information about the work performed by a worker, An acquisition unit that acquires image data showing the image captured by the worker performing the aforementioned work, A control unit that sequentially detects the work performed by the worker based on the image data and generates work detection information indicating the result of the work detection. Equipped with, The aforementioned work includes multiple steps performed by the aforementioned worker, The control unit corrects the work detection information so as to interpolate the work detection results during the undetected period, according to the process at which the work was detected before and after the undetected period in which the work was not detected. The aforementioned plurality of steps include a first step of starting the work, a second step of continuing the work, and a third step of ending the work. The control unit interpolates the detection result of the work during the undetected period in at least one of the cases where the process after the undetected period is the second process, or the process after the undetected period is the third process. Work analysis device.

2. The control unit does not interpolate the detection result of the operation during the undetected period if the process following the undetected period is the first process. The work analysis apparatus according to claim 1.

3. The aforementioned operation is a transportation operation that involves moving an object. The transport operation includes a first step of starting the movement of the object, a second step of continuing the movement of the object, and a third step of ending the movement of the object. The work analysis apparatus according to claim 1 or 2.

4. The control unit identifies the steps performed by the worker at the time of work detection based on the image data, and corrects the work detection information based on the step identification result. The work analysis apparatus according to any one of claims 1 to 3.

5. The control unit determines, based on the image data, whether the direction in which the worker moved at the time of detection of the work is a predetermined direction, and corrects the work detection information based on the result of the direction determination. The predetermined direction is set on the image shown by the image data, corresponding to the second step. The work analysis apparatus according to any one of claims 1 to 3.

6. The control unit determines, based on the image data, whether the detection position where the worker was located at the time of detection of the work is included in at least one predetermined area, and corrects the work detection information based on the determination result of the detection position. The at least one predetermined region is set on the image shown by the image data in correspondence with at least one of the first and third steps. The work analysis apparatus according to any one of claims 1 to 3.

7. The at least one predetermined region includes a starting region corresponding to the first step and an ending region corresponding to the third step, The control unit, If it is determined that at least one of the following is true: the detection position before the undetected period is not included in the end region, or the detection position after the undetected period is not included in the start region, the detection result of the operation during the undetected period is interpolated. If it is determined that the detection position before the undetected period is included in the end region, and the detection position after the undetected period is included in the start region, then the detection result of the operation during the undetected period is not interpolated. The work analysis apparatus according to claim 6.

8. A work analysis method for generating information about the work performed by a worker, The computer's control unit, The steps include: acquiring image data showing the image captured by the operator; Based on the aforementioned image data, the steps include sequentially detecting the work performed by the worker and generating work detection information indicating the results of the work detection. Includes, The aforementioned work includes multiple steps performed by the aforementioned worker, The control unit of the aforementioned computer, The work detection information is corrected to interpolate the detection results of the work during the undetected period, according to the process at which the work was detected before and after the undetected period in which the work was not detected. The aforementioned plurality of steps include a first step of starting the work, a second step of continuing the work, and a third step of ending the work. The control unit interpolates the detection result of the work during the undetected period in at least one of the cases where the process after the undetected period is the second process, or the process after the undetected period is the third process. Work analysis method.

9. A program for causing a computer control unit to execute the work analysis method described in claim 8.

Images