Work analysis device

Abstract

To provide a work analysis device capable of reducing the burden on a user to check work status by checking multiple tasks that seem inefficient at once, and / or checking multiple tasks with different efficiencies at once, and a work analysis method.SOLUTION: A work analysis device includes: a video acquisition unit that acquires video of footage of a specific task being performed multiple times by a worker; a work breakdown unit that calculates the total work time required for each task and non-work time during which no work was actually performed based on the video; a decomposition result display unit that plots the work on a coordinate plane with total work time and non-work time as the axes, and displays marks; a mark selection unit that selects a mark arbitrarily from the displayed marks based on the operation by the user; and a video display unit that displays the video of the work corresponding to the selected mark.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to an apparatus for analyzing work content performed by a worker. [Background technology]

[0002] Analyzing whether workers' work in warehouses, factories, etc. is being done efficiently and without waste is important for shortening work time, improving work quality, and ultimately reducing costs, etc. To that end, there is a device that performs work analysis based on images of the work being done (Patent Document 1).

[0003] In conventional technology, a task data analysis device displays a comparison image that shows the required time for each of a number of tasks so that it can be compared across a number of categories, accepts a category designation operation from this comparison image, and displays an image of the section corresponding to the designated category on the display.

[0004] According to conventional techniques, by specifying a section (category) that is considered to be inefficient from a comparison image such as a bar graph, it is possible to check the work status by observing the image corresponding to that section. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2023-82552 Summary of the Invention [Problem to be solved by the invention]

[0006] However, with conventional technology, images could only be specified by category, making it impossible to simultaneously check images of multiple tasks in different categories or to compare images of tasks with different efficiencies.

[0007] Therefore, the present disclosure aims to provide a work analysis device and a work analysis method that can simultaneously check footage of multiple tasks that are considered inefficient or multiple tasks with different efficiencies, thereby reducing the burden on users when checking the status of their work. [Means for solving the problem]

[0008] The work analysis device disclosed herein includes an image acquisition unit that acquires video footage of a specified task being performed multiple times by a worker; a task breakdown unit that calculates the total work time and non-work time for each task based on the video footage; a breakdown result display unit that plots and displays marks corresponding to the tasks on a coordinate plane with the total work time and non-work time as the coordinate axes; a mark selection unit that selects an arbitrary mark from the displayed marks based on user operation; and an image display unit that acquires and displays the video footage of the task corresponding to the selected mark. [Effects of the Invention]

[0009] According to the present disclosure, for work repeatedly performed by a worker, a user can easily check, compare, and analyze video footage of the work selected by the user. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a functional configuration diagram of a work analysis device according to a first embodiment. [Figure 2] FIG. 1 is a diagram illustrating a hardware configuration of an operation analysis device according to a first embodiment. [Figure 3] 4 is a flowchart of the operation of the work analysis device of the first embodiment. [Figure 4] FIG. 10 is a functional configuration diagram of a work analysis device according to a second embodiment. [Figure 5] 10 is a flowchart of the operation of the work analysis device of the second embodiment. [Figure 6] FIG. 10 is a diagram showing the correlation between total work time and non-work time for each work task. [Figure 7A]FIG. 10 is a diagram showing a case where the "good" curve (ellipse) is not displayed. [Figure 7B] FIG. 10 is a diagram showing a case where the "good" curve (ellipse) is not displayed. [Figure 8] FIG. 10 is a diagram showing a procedure for selecting an arbitrary point on a coordinate plane. [Figure 9] FIG. 10 is a diagram showing a procedure for selecting points in an arbitrary area on a coordinate plane. [Figure 10] FIG. 10 is a diagram showing a procedure for selecting points in an arbitrary rectangular area on a coordinate plane. [Figure 11] FIG. 10 is a diagram showing a procedure for selecting points of an arbitrary polygonal region on a coordinate plane. [Figure 12] FIG. 2 is a diagram showing a video display screen. [Figure 13] FIG. 10 is a diagram showing a first modified example of a plot display screen. [Figure 14] FIG. 10 is a diagram showing a second modified example of the plot display screen. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. Therefore, the components, the arrangement positions and connection forms of the components, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0012] In addition, each drawing is a schematic diagram and is not necessarily an exact illustration. In each drawing, substantially the same components are denoted by the same reference numerals, and redundant explanations will be omitted or simplified.

[0013] [First embodiment] 1 is a functional configuration diagram of a first embodiment of a work analysis device 1 of the present disclosure. A video acquisition unit 11 acquires video footage of a predetermined work being performed multiple times by a worker. Here, "work" refers to a series of actions performed by a worker, and a single work may be completed by performing the series of actions only once, or by repeating the series of actions multiple times.

[0014] For example, in a shipping operation in a warehouse where a predetermined number of packages are loaded onto a truck, a series of actions such as lifting and carrying the packages and loading them onto a loading platform are repeated for each package, and one operation is completed when all the packages have been loaded. Also, in an individual packaging and packing operation in a warehouse where predetermined items are packed into containers, a series of actions such as taking the predetermined items out of a storage location in the warehouse and packing them into containers is repeated for each packing container, and one operation is completed when all the items have been packed.

[0015] The video can be acquired from a camera connected to the work analysis device 1 and pointed at the workplace. It is also possible to acquire video stored in a system linked to a warehouse management system (WMS). Furthermore, a recording device that stores video footage of the work situation captured by a camera can be constructed separately from the work analysis device 1, and the video can be acquired from that recording device.

[0016] The work breakdown unit 12, based on the video footage of the specified work, identifies whether or not the work is actually being performed in accordance with the chronological order of the video, and calculates the total work time and non-work time for each task. Here, the total work time refers to the time required from the start to the end of each specified task. For example, in the case of the above-mentioned shipping task, it is the time from when loading of goods onto a truck begins to when all goods are finished being loaded. Furthermore, non-work time refers to time during, before, or after a task when no work is actually being performed, such as preparation time for the task, breaks taken during the task, waiting time, and cleanup time.

[0017] To calculate the total working time and non-working time, for example, a video of a worker's actions is input to a trained model that has undergone machine learning to estimate whether the worker is performing a specific task based on the video. At this time, action recognition is performed using a person image (worker) extracted from the video.

[0018] Specifically, for example, action recognition is performed by classifying actions using an action recognition engine (trained model) constructed by machine learning, that is, action recognition is performed using multiple classifiers that determine to which class (task) the recognition target (human action) belongs.In particular, for packaging and packing work in warehouses, interaction recognition technology is used, that is, technology that recognizes human actions by focusing on the interaction between a person and an object.

[0019] Furthermore, when multiple workers are performing inbound and outbound operations in a warehouse, the workers working in the workplace are tracked based on the video to generate person tracking information. At this time, a person detection process is performed to detect people from the video, but when multiple videos are used by capturing images of the workplace with multiple cameras, a person re-identification process is performed to match people detected from each video as the same person. The person tracking information obtained based on the person re-identification process includes location information for each person (ID) at each time.

[0020] The trained model then obtains an inference result based on the timeline of the video, indicating whether or not each worker (ID) shown in the video is currently performing a specific task. The system then identifies the time from when the worker starts to finish a task in the video to calculate the total task time. The system can also calculate non-task time as time when no task is being performed. In addition, in conjunction with the calculation of the total task time and non-task time, a video clip of each task can be obtained based on the start and end times of the task.

[0021] The decomposition result display unit 13 displays a display screen on which a mark corresponding to each task is plotted on a coordinate plane with the total working time and non-working time as the coordinate axes, based on the total working time and non-working time for each task. The marks are dots, squares, etc., but marks with different colors, shades, etc. may also be used as long as they can be displayed on the coordinate plane. Then, marks are plotted on the coordinate plane for the number of tasks for which the task has been decomposed.

[0022] The mark selection unit 14 acquires marks arbitrarily selected by the user from the marks displayed on the coordinate plane. That is, the selection is performed, for example, by selecting one or more marks on the coordinate plane displayed on the screen with a mouse or the like, and the operation corresponding to the selected mark can be identified.

[0023] The video display unit 15 acquires and displays a video (clip) of the work corresponding to the selected mark. By linking the video of the work breakdown according to the start time and end time of the work corresponding to the mark displayed on the coordinate plane, when the user selects a mark, the video recorded in the storage device is acquired and displayed on the screen.

[0024] As described above, each task is plotted on a coordinate plane with the total working time and non-working time as the respective axes, allowing the efficiency of each task (total working time, non-working time, ratio of non-working time to total working time, etc.) to be visually grasped. Furthermore, for example, by selecting an inefficient task and checking its status on video, it is easy to identify the cause of the inefficiency and consider ways to improve the work procedure. Furthermore, by selecting an efficient task and an inefficient task and comparing their videos, it is possible to consider ways to improve the task.

[0025] 2 is a diagram showing the hardware configuration of the work analysis device 1 of the first embodiment. A processor 21 is connected to a memory 22, a storage 23, a drive 24, a user interface 25, and a communication device 26 via a bus.

[0026] The processor 21 executes the task analysis program stored in the storage 23 while temporarily storing data in the memory 22. The storage 23 is a storage device such as an SSD, and stores the operating system, the task analysis program, video, various data, etc. The task analysis program performs its functions in cooperation with the operating system.

[0027] The drive 24 is an optical drive, a USB memory, or the like, and is used for loading programs, transferring data, and the like.

[0028] The user interface 25 is a device that displays the output of the disassembly result display unit 13 and accepts the selection of work by the user. Specifically, it is a display, keyboard, mouse, touch panel, etc.

[0029] The communication device 26 is a device for connecting to a network such as the Internet or a LAN. Although only the work analysis device 1 is described here, the work analysis device 1 can be configured as a server device (on-premise or cloud) connected to a network with multiple cameras, recorders, user terminals, etc.

[0030] 3 is a flowchart of the operation of the work analysis device 1 of the first embodiment. The processor 21 of the work analysis device 1 (hereinafter sometimes abbreviated as "work analysis device") acquires video footage of a predetermined work being performed multiple times by a worker (step S1).

[0031] Next, the work analysis device 1 calculates the total work time and non-work time for each work session based on the acquired video (step S2), and the results are recorded in the storage 23.

[0032] Next, the work analysis device 1 plots and displays a mark corresponding to each task on a coordinate plane whose coordinate axes are total work time and non-work time (step S3). The marks are dots, squares, etc., and the number of marks plotted on the coordinate plane is equal to the number of tasks for which the work has been broken down.

[0033] Next, the work analysis device 1 acquires a mark selected by the user from the marks of each work plotted and displayed on the coordinate plane (step S4).

[0034] Next, the work analysis device 1 acquires a video of the work corresponding to the selected mark and displays it on the screen (step S5).

[0035] As described above, each task is plotted on a coordinate plane with the total working time and non-working time as the respective axes, allowing the efficiency of each task (total working time, non-working time, ratio of non-working time to total working time, etc.) to be visually grasped. Furthermore, for example, by selecting an inefficient task and checking its status on video, it is easy to identify the cause of the inefficiency and consider ways to improve the work procedure. Furthermore, by selecting an efficient task and an inefficient task and comparing their videos, it is possible to consider ways to improve the task.

[0036] [Second embodiment] FIG. 4 is a functional configuration diagram of a second embodiment of a work analysis device 1 of the present disclosure. As with the first embodiment, the work analysis device 1 in the second embodiment can be configured as a network-connected server device (on-premise or cloud). Multiple cameras 2 capture images of workers at work. A camera image acquisition unit 3 acquires images from the cameras 2 and records them in a video database 4. A work breakdown unit 5 estimates, based on the images, whether or not the worker is actually performing a task, and calculates the total task time from the start to the end of each task, the non-task time when no work is actually being performed, and the ratio of the non-task time to the total task time, and records these in an analysis result database 6.

[0037] The data analysis unit 7 calculates a standard value for the ratio of non-work time to total work time. The ratio of non-work time to total work time is one indicator of work efficiency. Statistical methods such as calculating the average value for each work or calculating the median value for each work can be used to calculate the standard value.

[0038] Furthermore, the data analysis unit 7 classifies each task based on the standard value into, for example, three levels: "Challenge," "Standard," and "Good," or five levels: "1" to "5." For example, the classification may be such that a predetermined range based on the standard value is "Standard," "Above the Standard" range (high percentage of non-work time) is "Challenge," and "Below the Standard" range (low percentage of non-work time) is "Good." Furthermore, classification may be performed using a statistical method based on standard deviation, or by dividing the range from minimum to maximum into equal parts. The classification results are then recorded in the analysis result database 6.

[0039] Based on the total working time and non-working time for each task, the analysis result visualization unit 8 plots a mark corresponding to each task on a coordinate plane with the total working time and non-working time as the coordinate axes, and displays the plot on the display device 9. The marks are dots, squares, etc., and the same number of marks as the number of tasks for which the task decomposition was performed are plotted on the coordinate plane. Note that when many similar tasks are occurring, it is difficult to separately display marks corresponding to all the tasks, so it is also possible to display a representative mark from a group of marks included in a specified range (grid) on the coordinate plane.

[0040] The analysis result visualization unit 8 also displays the classification of each task on a coordinate plane so that it is clear which classification each task falls into. The display may be visual, for example, by color-coding the marks plotted on the coordinate plane according to classification, or by displaying the area on the coordinate plane for each classification with a frame.

[0041] The video distribution unit 10 acquires video associated with a mark arbitrarily selected by the user from among the marks displayed on the coordinate plane by the user interface 25 (see FIG. 2) of the work analysis device 1. Here, the selection by the user interface 25 is made by, for example, selecting one or more marks on the coordinate plane displayed on the screen with a mouse or the like.

[0042] That is, the single mark selection operation may be a tap operation using the user's finger or stylus, a touch operation for a predetermined period of time, or a click operation using a mouse. The multiple mark selection operation may be an area selection operation that surrounds multiple marks using the user's finger, stylus, or cursor (see FIG. 6). When one or more marks are selected, the display form of the selected marks may be changed so that the user can easily confirm whether or not they have been selected.

[0043] The video distribution unit 10 then obtains the video of the work corresponding to the selected mark from the video database 4. By linking the marks displayed on the coordinate plane with the video of the work breakdown, the video recorded in the video database 4 can be obtained when the user selects a mark. The analysis result visualization unit 8 then displays the video (video) on the display device 9. When checking the video (video), a playback window may be set in part of the display screen to play the video, or the video may be played by transitioning to the playback window (see FIG. 12) or popping it up.

[0044] As described above, each task is plotted on a coordinate plane with the total working time and non-working time as the respective axes, so the efficiency of each task (total working time, non-working time, ratio of non-working time to total working time, etc.) can be visually grasped. Furthermore, for example, by selecting an inefficient task and checking its status on video, it is easy to identify the cause of the inefficiency and consider ways to improve the work procedure. Furthermore, by selecting an efficient task and an inefficient task and comparing their videos, it is possible to consider ways to improve the tasks.

[0045] 5 is a flowchart of the operation of the work analysis device 1 of the second embodiment. The processor 21 of the work analysis device 1 (hereinafter sometimes abbreviated as "work analysis device") acquires video footage of a predetermined work being performed multiple times by a worker (step S11).

[0046] Next, the work analysis device 1 calculates the total work time and non-work time for each work session based on the acquired video (step S12), and the results are recorded in the storage 23.

[0047] Next, the work analysis device 1 calculates a standard value for the ratio of non-work time to total work time (step S13). For example, the average value of the ratio of non-work time to total work time is calculated and this average value is set as the standard value. In addition to the average value, the method of calculating the standard value, such as the median, can be selected as appropriate depending on the variance of the values, etc.

[0048] Next, the work analysis device 1 plots and displays each task on a coordinate plane with the total task time and non-task time as the coordinate axes (step S14). Fig. 6 shows an example of a display screen 30 on which a task time distribution map is displayed. The display screen 30 is arranged with a coordinate plane 31, a display setting area 32, a video check button 40, a display previous task button 41, and a display next task button 42. The display setting area 32 is arranged with a problem button 33, a standard button 34, and a good button 35. The problem button 33, standard button 34, and good button 35 can each be operated by the user, and can switch between displaying and hiding curves (ellipses) that indicate the ranges of each classification: "problem," "standard," and "good" (see Figs. 7A and 7B).

[0049] Furthermore, an arbitrary selection button 36, an arbitrary area selection button 37, an arbitrary rectangular area selection button 38, and an arbitrary polygonal area selection button 39 are arranged in a mark selection box 43 on the left side of the coordinate plane 31. When performing a single mark selection operation or a multiple mark selection operation, the user can select the mark to be checked using the button according to their preference.

[0050] The horizontal axis of the coordinate plane 31 represents total work time, and the vertical axis represents non-work time. Points (marks) corresponding to the total work time and non-work time for each task are displayed on the coordinate plane 31.

[0051] Returning to Figure 5, next, the work analysis device 1 classifies the work according to efficiency using the standard value as a reference (step S15). For example, using the standard value as a reference, it may be classified into three levels: "Issues" if the work is 20% or more higher than the standard value, "Good" if the work is 20% or more lower than the standard value, and "Standard" otherwise. Alternatively, the standard deviation may be calculated and the classification may be statistically determined.

[0052] Next, the work analysis device 1 displays the classification results on the coordinate plane 31 (step S16). Fig. 6 is a diagram displaying the correlation between total work time and non-work time for each task. In Fig. 6, on the coordinate plane 31, curves (ellipses) that serve as a guide for classifying tasks according to the ratio of non-work time to total work time are displayed in different colors as "problem," "standard," and "good." In Fig. 6, there are marks where the curves partially overlap and straddle two categories, but these curves do not necessarily need to be drawn precisely and are used as a guide for the user to visually understand the range of the categories.

[0053] Tasks included in the "Challenge" curve (ellipse) have a high ratio of non-work time to total work time, and are considered to have low work efficiency. Tasks included in the "Standard" curve (ellipse) have a medium ratio of non-work time to total work time, and are considered to have standard work efficiency. Tasks included in the "Good" curve (ellipse) have a low ratio of non-work time to total work time, and are considered to have good work efficiency.

[0054] As mentioned above, the curves (ovals) for "Challenge," "Standard," and "Good" can be hidden as needed by user operation. Figures 7A and 7B are diagrams showing the case where the "Good" curve (oval) is hidden. As shown in Figure 7A, when the Good button 35 in the display setting area 32 is clicked with the mouse, the "Good" curve (oval) is hidden as shown in Figure 7B. In this way, it is possible to display only the curves (ovals) for a specific classification (Challenge, Standard, Good) to make them easier to see. Similarly, for the Challenge button 33 and the Standard button 34, it is possible to select whether to display or hide the "Challenge" curve (oval) or the "Standard" curve (oval).

[0055] Returning to Fig. 5, the user selects any task from the marks of each task plotted and displayed on the coordinate plane 31 (step S17). In the coordinate plane of Fig. 6, when the user operates the problem button 33, the standard button 34, or the good button 35 in the display setting area 32 (shown in the selected state in Fig. 6), the selected curve is displayed.

[0056] FIG. 8 is a diagram showing the procedure for selecting an arbitrary point on the coordinate plane 31. When the user selects the optional selection button 36 by operating the mouse, the user can select an arbitrary mark on the coordinate plane 31 by operating the mouse. In this case, one or more marks can be selected arbitrarily. This operation using the optional selection button 36 is effective, for example, when selecting multiple marks that are located apart from each other. In this case, it is recommended to select the mark by referring to the curves (ellipses) of "problem," "standard," and "good" displayed on the coordinate plane 31.

[0057] FIG. 9 is a diagram showing the procedure for selecting points in an arbitrary area from points on the coordinate plane 31. The user selects the arbitrary area selection button 37 by operating the mouse, and when the user draws an arbitrary area (dashed line) by operating the mouse, points within this area can be selected. One or more points can be selected arbitrarily. At this time, points can be selected by referring to the curves (ellipses) of "Issue," "Standard," and "Good" displayed on the coordinate plane 31. This makes it easy to select multiple points simultaneously. Operation using this arbitrary area selection button 37 is effective, for example, when selecting multiple marks in the "Issue" area all at once.

[0058] FIG. 10 shows the procedure for selecting points in an arbitrary rectangular area from points on the coordinate plane 31. The user selects the arbitrary rectangular area selection button 38 with the mouse and draws an arbitrary rectangular area (dashed line) with the mouse, allowing the user to select points within this area. One or more points can be selected arbitrarily. At this time, points can be selected by referring to the curves (ellipses) of "Issued," "Standard," and "Good" displayed on the coordinate plane 31. This makes it easy to select multiple points across "Issued," "Standard," and "Good," which have similar total work time but different non-work time. Operation using the arbitrary rectangular area selection button 38 is effective, for example, when selecting multiple marks at a specific coordinate position on the coordinate plane 31 all at once.

[0059] FIG. 11 illustrates the procedure for selecting points within an arbitrary polygonal area from points on the coordinate plane 31. The user selects the arbitrary polygonal area selection button 39 with the mouse and then draws an arbitrary polygonal area (dashed line) with the mouse, allowing the user to select points within this area. One or more points can be selected arbitrarily. Points can be selected by referring to the curves (ellipses) of "Issued," "Standard," and "Good" displayed on the coordinate plane 31. This makes it easy to select multiple points across "Issued," "Standard," and "Good," which have different total work hours and non-work hours. Like the arbitrary area selection button 37, the operation using the arbitrary polygonal area selection button 39 is effective for selecting multiple marks at once, but it is also effective for adjusting the range when there are marks that you do not want to include in the selection range.

[0060] Returning to FIG. 5, when a user operates any of the arbitrary selection button 36, arbitrary area selection button 37, arbitrary rectangular area selection button 38, or arbitrary polygonal area selection button 39 in the mark selection box 43 in FIG. 6 to select an arbitrary mark on the coordinate plane 31 and then operates the video review button 40, the work analysis device 1 acquires a video (video) of the selected work and displays (plays) it on the screen (step S18). In FIGS. 6 to 11, when a mark is selected and the video review button 40 is clicked with the mouse, the video of the work corresponding to the selected mark can be displayed and viewed. Here, the video is played by clicking the video review button 40 with the mouse; however, the video review button 40 may be omitted and the video may be played by clicking with the mouse within the area where the selected mark or multiple marks are selected.

[0061] 12 shows an example of a video confirmation screen 50 that displays a video of the work situation corresponding to the selected mark. Here, three classification marks 55, 56, and 57 are arranged to the left of the playback area 53, and the video of the work situation can be confirmed by referring to one of the highlighted classification marks 55, 56, or 57.

[0062] The three classification marks 55, 56, and 57 represent the classifications of "problem," "standard," and "good" from top to bottom, and can be displayed in different display formats (colors, etc.) so that they can be distinguished from one another. In Figure 12, classification mark 55 (problem) is highlighted, and the corresponding video is displayed in playback area 53, but when video in the "standard" or "good" classification is selected for display, classification mark 56 or 57 is highlighted.

[0063] Furthermore, when the video confirmation button 40 is operated with multiple marks selected (see FIG. 9), thumbnails (five thumbnails in this example) of video footage of the work corresponding to all of the selected marks are displayed side by side in the thumbnail display area 51. Here, only thumbnails of the work corresponding to the selected marks are displayed in the thumbnail display area 51, but a predetermined number of thumbnails corresponding to work performed before and after the work corresponding to the selected marks can be additionally displayed (not shown). In this case, by displaying video footage of the work performed from thumbnails of the work performed before and after the selected marks, the contents of the work performed before and after the selected marks can be efficiently confirmed and verified. When additional thumbnails corresponding to the work performed before and after the selected marks are displayed, it is desirable to display them in a frame size smaller than the frame size of the thumbnail of the work corresponding to the selected marks.

[0064] A video corresponding to the mark selected from the thumbnails is played in the playback area 53. In the example of FIG. 12, of the five thumbnails corresponding to the selected mark, the central thumbnail 52 is selected (its frame is highlighted), and its video is displayed in the playback area 53. The video displayed in the playback area 53 can be operated using operation buttons 54. From left to right, the operation buttons 54 are provided with functions for skipping (fast rewinding), fast rewinding, stopping, pausing, playing, fast forwarding, and skipping (fast forwarding) the video displayed in the playback area 53. When playing back the video, playback operations are not limited to the operation buttons 54, and a seek bar 58 located at the bottom of FIG. 12 may also be used to perform playback operations.

[0065] Furthermore, multiple videos may be played back side by side in the playback area 53. This allows, for example, the work of an efficient worker and an inefficient worker to be easily compared by simultaneously playing back the videos, and the work of an inefficient worker can be easily verified.

[0066] As described above, the work analysis device 1 of the second embodiment plots each task on the coordinate plane 31 based on video footage of the task and then classifies them based on task efficiency, allowing the distribution of task efficiency to be seen at a glance. Any task can then be selected on the task plane and the details of that task can be easily confirmed on the video.

[0067] Figure 13 shows a first modified example of the plot display screen 30 for shipping work. In this example, the size of the mark changes depending on the number of shipments, and the color of the mark changes depending on the shipping destination. This makes it easy to analyze work efficiency according to the number of shipments and shipping destination.

[0068] Figure 14 shows a second variation of the plot display screen 30 for shipping work. In this example, the size of the mark changes depending on the number of shipments, the color of the mark changes depending on the type of outgoing item, and the shape of the mark changes depending on the size of the outgoing item. This makes it easy to analyze work efficiency according to the number of shipments, the type of outgoing item, and the size of the outgoing item.

[0069] As described above, the present disclosure has been described based on the embodiments and modifications, but the embodiments and modifications can be implemented in appropriate combinations. In the above example, the work analysis device 1 is configured using a single computer, but the work analysis device 1 may also be configured using two computers, for example, by having one computer function as the image acquisition unit and the work decomposition unit, and another computer function as the decomposition result display unit, mark acquisition unit, and image display unit.

[0070] The work analysis device 1 of the present disclosure can be applied to the analysis of various tasks in which workers repeatedly perform predetermined tasks, such as packaging work, manufacturing and assembly work in factories, in addition to shipping work.

[0071] The present disclosure includes the following aspects.

[0072] (1) a video acquisition unit that acquires video of a predetermined task being performed multiple times by a worker; a work breakdown unit that calculates the total work time and non-work time for each task based on the video; a decomposition result display unit that plots and displays marks corresponding to the operations on a coordinate plane having total operation time and non-operation time as its coordinate axes; a mark selection unit for selecting an arbitrary mark from the displayed marks based on a user operation; a video display unit that acquires and displays the video of the work corresponding to the selected mark; A work analysis device having the above.

[0073] (2) a ratio calculation unit that calculates a ratio of the non-work time to the total work time for each work; a classification unit that classifies the work based on the ratio; a classification display unit that displays the results of the classification on the coordinate plane; The work analysis device according to (1), comprising:

[0074] (3) The classification display unit switches between displaying and hiding the plurality of classifications. (2) A work analysis device according to the present invention.

[0075] (4) The work breakdown unit inputs the video into a trained model, and obtains from the trained model an inference result as to whether or not the worker shown in the video is performing a predetermined task in accordance with the time series of the video, and calculates the time in the video when the worker is not performing a task as non-work time. The work analysis device according to (1).

[0076] (5) The mark selection unit selects a mark within an area arbitrarily drawn by a user on the coordinate plane. A work analysis device according to any one of (1) to (4).

[0077] (6) The mark selection unit selects a mark within a rectangular area arbitrarily drawn by a user on the coordinate plane. A work analysis device according to any one of (1) to (4).

[0078] (7) The mark selection unit selects a mark within a polygonal area arbitrarily drawn by a user on the coordinate plane. A work analysis device according to any one of (1) to (4).

[0079] (8) The work is a warehouse removal work, the disassembly result display unit plots and displays different marks so that the number of shipments or the shipping destinations for each operation can be identified. The work analysis device according to (1).

[0080] (9) The work is shipping and packing work in a warehouse. the disassembly result display unit plots and displays different marks so that the number of shipments, the type of delivered items, or the size of the delivered items in each operation can be identified; The work analysis device according to (1).

[0081] (10) The decomposition result display unit displays the mark in an identifiable manner by changing the size, color, or shape of the mark. The work analysis device according to (8) or (9).

[0082] (11) The video display unit includes a thumbnail display unit that displays a thumbnail of a video of the work corresponding to the selected mark; a playback display unit that displays a video of a corresponding work from the thumbnail arbitrarily selected by the user; The work analysis device according to (1),

[0083] (12) The computer acquiring a video of a predetermined task being performed multiple times by a worker; calculating a total work time and non-work time for each task based on the video; a step of plotting and displaying a mark corresponding to the work on a coordinate plane having total work time and non-work time as coordinate axes; selecting a mark from the displayed marks based on a user operation; acquiring and displaying the video of the work corresponding to the selected mark; A work analysis method to carry out.

[0084] (13) To the computer, acquiring a video of a predetermined task being performed multiple times by a worker; calculating a total work time and non-work time for each task based on the video; a step of plotting and displaying a mark corresponding to the work on a coordinate plane having total work time and non-work time as coordinate axes; selecting a mark from the displayed marks based on a user operation; acquiring and displaying the video of the work corresponding to the selected mark; A work analysis program to carry out the above. [Industrial Applicability]

[0085] This can be suitably applied when analyzing the work performed by a worker. [Explanation of symbols]

[0086] 1 Work analysis device 11 Video acquisition unit 12 Work Breakdown Section 13 Decomposition result display section 14 Mark selection section 15 Video display section 30 display screen 31 Coordinate plane 43 Mark Selection Box 50 Video confirmation screen

Claims

1. a video acquisition unit that acquires video of a predetermined task being performed multiple times by a worker; a work breakdown unit that calculates the total work time and non-work time for each task based on the video; a decomposition result display unit that plots and displays marks corresponding to the operations on a coordinate plane having total operation time and non-operation time as its coordinate axes; a mark selection unit for selecting an arbitrary mark from the displayed marks based on a user operation; a video display unit that acquires and displays the video of the work corresponding to the selected mark; A work analysis device having the above.

2. a ratio calculation unit that calculates a ratio of the non-work time to the total work time for each work; a classification unit that classifies the work based on the ratio; a classification display unit that displays the results of the classification on the coordinate plane; The work analysis device according to claim 1 , further comprising:

3. the classification display unit switches between displaying and hiding the plurality of classifications, The work analysis device according to claim 2 .

4. The work breakdown unit inputs the video into a trained model, and obtains from the trained model an inference result as to whether or not a worker shown in the video is performing a predetermined task in accordance with the time series of the video, and calculates a time in the video when the worker is not performing a task as non-work time. The work analysis device according to claim 1 .

5. the mark selection unit selects a mark within an area arbitrarily drawn by a user on the coordinate plane; The work analysis device according to any one of claims 1 to 4.

6. the mark selection unit selects a mark within a rectangular area arbitrarily drawn by a user on the coordinate plane; The work analysis device according to any one of claims 1 to 4.

7. the mark selection unit selects a mark within a polygonal area arbitrarily drawn by a user on the coordinate plane; The work analysis device according to any one of claims 1 to 4.

8. The work is a carrying-out work in a warehouse, the disassembly result display unit plots and displays different marks so that the number of shipments or the shipping destinations for each operation can be identified. The work analysis device according to claim 1 .

9. The work involves shipping and packing in a warehouse, the disassembly result display unit plots and displays different marks so that the number of shipments, the type of delivered items, or the size of the delivered items in each operation can be identified; The work analysis device according to claim 1 .

10. the decomposition result display unit displays the mark in an identifiable manner by changing the size, color or shape of the mark; The work analysis device according to claim 8 or 9.

11. the video display unit includes a thumbnail display unit that displays a thumbnail of a video of the work corresponding to the selected mark; a playback display unit that displays a video of a corresponding work from the thumbnail arbitrarily selected by the user; The work analysis device according to claim 1 , further comprising:

12. The computer acquiring a video of a predetermined task being performed multiple times by a worker; calculating a total work time and non-work time for each task based on the video; a step of plotting and displaying a mark corresponding to the work on a coordinate plane having total work time and non-work time as coordinate axes; selecting a mark from the displayed marks based on a user operation; acquiring and displaying the video of the work corresponding to the selected mark; A work analysis method to carry out.

13. On the computer, acquiring a video of a predetermined task being performed multiple times by a worker; calculating a total work time and non-work time for each task based on the video; a step of plotting and displaying a mark corresponding to the work on a coordinate plane having total work time and non-work time as coordinate axes; selecting a mark from the displayed marks based on a user operation; acquiring and displaying the video of the work corresponding to the selected mark; A work analysis program to carry out the above.

Images