Operation analysis device and operation analysis method
The work analysis device and method improve work quality by evaluating work based on finished states and visualizing task performance, addressing the speed-focused limitations of conventional methods by providing detailed quality assessments and improvement suggestions.
Patent Information
- Application Number
- PCT/JP2024/018536
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-22
- Filing Date
- 2024-05-20
- Publication Date
- 2024-11-28
AI Technical Summary
Conventional work analysis technologies focus on the speed of work, emphasizing added value coefficients, which leads to inadequate evaluation and improvement of work quality.
A work analysis device and method that evaluates work quality based on the finished state of the work object, using video analysis to recognize and visualize the quality of work performed, allowing for the identification of value-added and non-value-added tasks, and generating visualized images to represent execution times and implementation status.
Enables users to assess and improve the quality of work by focusing on the work result, providing detailed insights into work performance and suggesting appropriate improvements.
Smart Images

Figure JP2024018536_28112024_PF_FP_ABST
Abstract
Description
Work analysis device and work analysis method
[0001] The present disclosure relates to an operation analysis device and an operation analysis method that output analytical information regarding the operation status of a worker based on a video of the worker's operation.
[0002] At manufacturing and logistics centers, work efficiency can be improved by analyzing the work status of workers and reviewing work procedures based on the analysis results.
[0003] A known technique for analyzing such worker work situations is to measure the work time for each work item based on video footage of the worker's work and output a value-added graph in which the work time is set on the horizontal axis and the value-added coefficient for each work item is set on the vertical axis (see Patent Document 1). This technique allows the user to visually identify low-value-added tasks and tasks that take a long time from the value-added graph.
[0004] Japanese Patent Application Laid-Open No. 2003-6286
[0005] In conventional techniques, work performed by a worker is evaluated based on the added value that the work adds to the object (workpiece) that the worker is working on. However, because the level of added value is predetermined for each work item, work related to the same work item is evaluated based on the length of time it takes. This results in an emphasis on the speed of the work, which leads to the problem of insufficient progress in improving the quality of work and leveling it.
[0006] Therefore, the main purpose of the present disclosure is to provide a work analysis device and a work analysis method that enable a user to examine the work status of a worker while also taking into account the quality of the work performed by the target worker, thereby enabling sufficient progress in improving and leveling the quality of work.
[0007] The work analysis device disclosed herein is a work analysis device that uses a processor to execute a process of outputting analytical information about a worker's work status based on video footage of the worker's work. The processor is configured to recognize the work performed by the worker based on the video footage, evaluate the work quality of the work performed by the worker based on the work results, and generate the analytical information that visualizes the work status including the evaluation results of the work quality.
[0008] Furthermore, the work analysis method disclosed herein is a work analysis method in which a processor executes a process to output analytical information regarding a worker's work status based on video footage of the worker's work, and is configured to recognize the work performed by the worker based on the video footage, evaluate the work quality related to the work performed by the worker based on the work results, and generate the analytical information in which the work status is visualized, including the evaluation results of the work quality.
[0009] According to the present disclosure, it is possible to evaluate the work performed by a target worker by focusing on the work result, i.e., the finished state of the object being worked on, and to visualize and present the work status including the evaluation results of the work quality to the user. This allows the user to consider the work status of the target worker while also taking into account the quality of the work performed by the target worker, thereby enabling sufficient progress in improving and leveling the quality of the work.
[0010] Overall configuration diagram of a work analysis system according to this embodiment Block diagram showing the general configuration of a server Block diagram showing an overview of the processing performed by the server Explanatory diagram showing how time scores are acquired in the task time evaluation processing Explanatory diagram showing suggested content (messages) according to combinations of quality scores and time scores Flow diagram showing the procedure of processing performed by the server Explanatory diagram showing a graph screen displayed on a user terminal Explanatory diagram showing a situation when the display condition setting unit on the graph screen is operated Explanatory diagram showing another example of a graph screen displayed on a user terminal Explanatory diagram showing a timeline screen displayed on a user terminal Explanatory diagram showing a video playback screen displayed on a user terminal
[0011] The first invention made to solve the above problem is a work analysis device that uses a processor to execute a process to output analytical information about a worker's work status based on video footage of the worker's work. The processor is configured to recognize the work performed by the worker based on the video footage, evaluate the work quality of the work performed by the worker based on the work results, and generate the analytical information that visualizes the work status including the evaluation results of the work quality.
[0012] This allows the system to evaluate the work performed by the target worker by focusing on the work results, i.e., the finished state of the object being worked on, and to visualize and present the work status including the evaluation results of the work quality to the user. This allows the user to consider the work status of the target worker while also taking into account the quality of the work performed by the worker, thereby enabling sufficient progress in improving and leveling the quality of work.
[0013] In addition, in a second invention, the processor is configured to recognize the value-added work and non-value-added work performed by the worker based on the video, and generate the analysis information that distinguishes between the value-added work and the non-value-added work.
[0014] This allows the user to properly understand the work status of the target worker.
[0015] In a third aspect of the present invention, the processor generates a visualized image that represents, as the analysis information, a cumulative execution time of the value-added work and the non-value-added work.
[0016] This allows the user to check the breakdown of the work performed by the target worker. In this case, the visualized image is, for example, a stacked graph. Alternatively, a visualized image may be generated that shows the accumulated execution time for each of multiple work items.
[0017] In a fourth aspect of the present invention, the processor generates, as the analysis information, a visualized image that represents the implementation status of the value-added work and the non-value-added work on a time axis.
[0018] This allows the user to understand how the target worker is progressing with the work. In this case, the visualized image is, for example, a timeline chart. Note that a visualized image may be generated that represents the implementation status of each of multiple work items on a time axis.
[0019] In a fifth aspect of the present invention, the processor recognizes the work performed by the worker by action recognition using a plurality of classifiers that determine a class to which a recognition target belongs based on the video.
[0020] This allows for accurate recognition of the work performed by the target worker. In this case, for example, it is possible to identify the work item of the work performed by the target worker. Furthermore, it is possible to determine whether the work performed by the target worker is a value-added work.
[0021] In a sixth aspect of the present invention, the processor selects the content of the proposal to be presented to the user based on the evaluation results of the work quality and work time of the worker.
[0022] This allows appropriate suggestions to be made to the user to improve their work.
[0023] A seventh invention is a work analysis method in which a processor executes a process to output analytical information regarding a worker's work status based on video footage of the worker's work, and is configured to recognize the work performed by the worker based on the video footage, evaluate the work quality related to the work performed by the worker based on the work results, and generate the analytical information in which the work status is visualized, including the evaluation results of the work quality.
[0024] As a result, similar to the first invention, the user can consider the work status of the worker in question while also taking into account the quality of the work performed by that worker, thereby making it possible to fully improve and level out the quality of work.
[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[0026] FIG. 1 is a diagram showing the overall configuration of a work analysis system according to this embodiment.
[0027] This system analyzes the work status of workers at manufacturing or logistics centers and presents the analysis results to managers and workers. The system includes multiple cameras 1, a recorder 2, a server 3 (work analysis device), and a user terminal 4. The cameras 1, recorder 2, server 3, and user terminal 4 are connected via a network.
[0028] The camera 1 captures images of workers performing their work in a work area. The example shown in Fig. 1 is a truck berth where cargo is carried in and out.
[0029] The recorder 2 records the video output from the camera 1.
[0030] The server 3 acquires video from the camera 1 or the recorder 2, analyzes the work situation of the worker based on the video, and outputs the analysis results.
[0031] The user terminal 4 is used by users such as the work manager or the worker himself / herself, and is configured as a PC, a tablet terminal, etc. The user terminal 4 displays the analysis results output from the server 3, and the user can view the analysis results.
[0032] The processing performed by the server 3 may be real-time processing performed simultaneously with shooting, or post-processing performed after shooting. In the case of real-time processing, the video captured by the camera 1 is transmitted to the server 3, and in the case of post-processing, the video stored in the recorder 2 is transmitted to the server 3. In the case of post-processing, the server 3 may directly read the video from various storage media.
[0033] Next, the server 3 will be described. Fig. 2 is a block diagram showing a schematic configuration of the server 3. Fig. 3 is a block diagram showing an overview of the processing performed by the server 3. Fig. 4 is an explanatory diagram showing how the time score is obtained in the task time evaluation processing P11.
[0034] As shown in FIG. 2, the server 3 includes a communication unit 31 , a storage unit 32 , and a processor 33 .
[0035] The communication unit 31 communicates with the camera 1 , the recorder 2 , and the user terminal 4 .
[0036] The storage unit 32 stores programs and the like to be executed by the processor 33. The storage unit 32 also stores area information related to the workplace, worker information related to the workers, and work data generated by the processor 33 related to the work status of the workers.
[0037] The processor 33 performs various processes by executing programs stored in the storage unit 32. In this embodiment, the processor 33 performs a person tracking process P1, an activity recognition process P2, an activity evaluation process P3, a counting process P4, a visualization process P5, a proposal content selection process P6, a display information generation process P7, and the like.
[0038] In person tracking processing P1, processor 33 tracks workers performing tasks in the workplace based on the video from camera 1, and generates person tracking information. At this time, person detection processing is performed to detect people from the video from camera 1. Furthermore, if multiple cameras 1 are installed, person re-identification processing is performed to associate people detected from the video from each camera 1 as the same person. The person tracking information acquired based on the person re-identification processing includes position information for each person (ID) at each time.
[0039] As shown in FIG. 3 , in the task recognition process P2, the processor 33 recognizes the tasks (value-added tasks and non-value-added tasks) being performed by the target worker based on the image captured by the camera 1. Value-added tasks and non-value-added tasks can be set in advance depending on the tasks to be analyzed. For example, scheduled tasks (i.e., tasks requested of the worker) are set as value-added tasks, and other tasks are set as non-value-added tasks. More specifically, tasks such as loading and unloading luggage, which will be described later, are set as value-added tasks. On the other hand, tasks other than these value-added tasks and states in which tasks are interrupted before or after value-added tasks (e.g., a state in which tasks are waiting to be performed) are set as non-value-added tasks.
[0040] At this time, action recognition is performed using a person image extracted from the video captured by camera 1. Specifically, action recognition is performed by classifying actions using, for example, an action recognition engine (machine learning model) constructed by machine learning, i.e., using multiple classifiers that determine to which class (task) the object to be recognized (the person's action) belongs. Such an action recognition engine can be trained using training data related to value-added tasks and non-value-added tasks that have been prepared in advance. This makes it possible to determine whether the recognized task is a value-added task or a non-value-added task based on the multiple classes (tasks) to which the object to be recognized belongs. Furthermore, the task recognition process P2 uses, for example, an interaction recognition technology, i.e., a technology that recognizes a person's actions by focusing on the interaction between a person and an object.
[0041] Furthermore, in the task recognition process P2, the completion of a task is detected when a series of actions recognized as a single task is performed, and the period during which the task was performed, i.e., the period during which each task was performed, including the value-added task and the non-value-added task, is identified. Note that if a short break is included between tasks, the break for that task may be considered to be included in the performance period of the value-added task, provided that the break is shorter than a predetermined time. In this case, if the break is longer than the predetermined time, the break for that task is considered to be included in the performance period of the non-value-added task.
[0042] The task evaluation process P3 includes a task time evaluation process P11, a task quality evaluation process P12, and a task overall evaluation process P13.
[0043] In the task time evaluation process P11, the processor 33 evaluates the task time of the target worker by comparing the task time with a preset standard time, and obtains a time score. The task time is calculated by adding up the time required for each value-added task and non-value-added task. The standard time is set for each task based on past task times at the target workplace.
[0044] In this embodiment, task time is evaluated on a three-level scale, and a time score is determined on a three-level scale (1 to 3). As shown in FIG. 4 , the task time is compared with first to third thresholds to evaluate the task time. Specifically, if the task time is less than the first threshold, the task is evaluated as being too fast, and the time score is set to "1." The first threshold is preferably set as a time value at which even an experienced worker has difficulty maintaining standard task quality. If the task time is equal to or greater than the first threshold and less than the second threshold, the task is evaluated as being fast, and the time score is set to "3." If the task time is equal to or greater than the second threshold and less than the third threshold, the task is evaluated as being standard, and the time score is set to "2." If the task time is equal to or greater than the third threshold, the task is evaluated as being slow, and the time score is set to "1."
[0045] 3, in the task quality evaluation process P12, the processor 33 determines the state of completion of the task (task result), i.e., the state of the object being worked on, and based on the determination result, evaluates the quality of the task performed by the worker and obtains a quality score. The task quality evaluation process P12 is performed every time the task recognition process P2 detects the end of the task.
[0046] At this time, an object to be worked on is detected from the image captured by camera 1, and object recognition is performed on the image of the object. Specifically, object recognition is performed by class classification using an object recognition engine (machine learning model) constructed by machine learning, that is, object recognition is performed using a plurality of classifiers that determine to which class (object state corresponding to work quality) the recognition target (object to be worked on) belongs.
[0047] For example, in the task of loading luggage, the condition of the luggage loaded into the cart is evaluated. Specifically, if the luggage is loaded to capacity and neatly, the quality score will be high, and conversely, if the luggage is loaded in a messy manner with gaps, the quality score will be low.
[0048] In this embodiment, the work quality is evaluated on a three-level scale, and the quality score is determined on a three-level scale (1 to 3). If the work is done carefully and the work quality is good, the quality score is "3." If the work is normal and the work quality is standard, the quality score is "2." If the work is sloppy and the work quality is poor, the quality score is "1."
[0049] In addition, the work quality evaluation process P12 may be performed using images other than those used in the work recognition process P2, for example, images captured by another camera 1, or may be performed using detection data from a sensor other than images.
[0050] In the task comprehensive evaluation process P13, the processor 33 comprehensively evaluates the task status of the target worker based on the time score obtained in the task time evaluation process P11 and the quality score obtained in the task quality evaluation process P12, and obtains a task score. Specifically, the task score is calculated by averaging the time score and the quality score. Note that the task score may be calculated using a weighted average in addition to a simple average, or may be calculated using statistical processing other than averaging.
[0051] In the aggregation process P4, the processor 33 aggregates the results of the work recognition process P2, the work time evaluation process P11, the work quality evaluation process P12, and the overall work evaluation process P13 based on the display conditions specified by the user (workplace, date and time, and worker).
[0052] Specifically, the work time for each value-added task and non-value-added task in a specified period (e.g., a time period) is calculated based on information about the period during which each value-added task and non-value-added task was performed (start time, end time). At this time, the work time for each task performed multiple times within the specified period is added together to calculate the total work time for the specified period.
[0053] In addition, the score for the entire specified period (e.g., time period) is calculated by averaging the scores (time score, quality score, and task score) for each task performed multiple times within the specified period (e.g., time zone).
[0054] In the visualization process P5, the processor 33 generates information that visualizes the work status of the target worker, i.e., the status of value-added work and non-value-added work, based on the results of the work recognition process P2 and the aggregation process P4.
[0055] Specifically, a graph (visualized image) is generated that shows the cumulative execution time of each value-added task and non-value-added task. A timeline chart (visualized image) is also generated that shows the execution status of value-added and non-value-added tasks on a time axis. A score mark (visualized image) is also generated that shows the evaluation results (time score, quality score, and task score) of the target worker's work status.
[0056] In the suggestion content selection process P6, the processor 33 selects content (message) to suggest to the user for improving the work based on the evaluation results regarding the work quality and work time, i.e., the quality score and time score (see Figure 5).
[0057] In the display information generation process P7, the processor 33 generates display information related to the screens to be displayed on the user terminal 4. Specifically, display information related to a graph screen 101 (see FIG. 7 ) including the graph (visualized image) generated in the visualization process P5, a timeline screen 201 (see FIG. 10 ) including the timeline chart (visualized image) generated in the visualization process P5, and a video playback screen 301 (see FIG. 11 ) including the proposal content (message) selected in the proposal content selection process P6 and the video from camera 1 is generated.
[0058] Next, a description will be given of the proposal content selection process P6 performed by the server 3. Fig. 5 is an explanatory diagram showing proposal content (messages) according to combinations of quality scores and time scores.
[0059] The server 3 selects a suggestion (message) to be sent to the user for improving the work based on the evaluation results of the work quality and work time, i.e., the quality score and the time score. The message is displayed on the video playback screen 301 (see FIG. 11 ).
[0060] Specifically, in the first case, if the quality score is high and the time score is also high, a message praising the work is displayed. Also, a message suggesting an improvement proposal hearing, i.e., a hearing with other workers to improve the work, is displayed. For example, a message such as "This is going well. Let's aim for further overall optimization by dividing the work with ID 17, and also take a break" is displayed.
[0061] In the second case, if the quality score is high and the time score is standard, a message praising the work is displayed. Also, a message suggesting an improvement proposal, i.e., a request to interview other workers to improve the work, is displayed. For example, a message such as "This is going well. If you still have time, consider sharing the work with ID 20" is displayed.
[0062] In the third case, if the quality score is high but the time score is low, a message suggesting ways to shorten the work time is displayed. For example, a message such as "You have done a careful job, so next time please consider ways to shorten the time" is displayed.
[0063] In the fourth case, if the quality score is low but the time score is standard, a message urging the worker to work carefully is displayed. For example, the message "Since you are working within the appropriate time, please consider ways to improve the quality of your work" is displayed.
[0064] In the fifth case, when the quality score is low but the time score is high, a message is displayed encouraging the worker to take their time and work carefully. For example, a message such as "Since you have completed the work in a short time, please consider ways to improve the quality of your work" is displayed.
[0065] In the sixth case, when the quality score is standard and the time score is also standard, a message is displayed suggesting improvements based on previous interviews. For example, a message such as "There is also a work method called XX, so why not try it?" is displayed.
[0066] In addition, in the seventh case, when the quality score is standard but the time score is low, a message is displayed suggesting sharing the work with members who have high time scores. For example, a message such as "Let's consider sharing the work (example of sharing partners: ID16, ID18)" is displayed. Note that workers with high time scores are extracted based on the time scores for each worker obtained in the past.
[0067] In the eighth case, when the quality score is low and the time score is also low, a message is displayed asking for a detailed review of the work status. For example, a message such as "Let's review together how to proceed with the work" is displayed.
[0068] In this embodiment, since the work time of each worker is acquired, imbalances in the work time of each worker may be detected by comparing the work times of each worker. In this case, a proposal to divide the work between high-load members and low-load members may be made to the workers or the manager.
[0069] Next, a description will be given of the procedure of the process performed by the server 3. FIG.
[0070] First, the processor 33 tracks a worker working in the workplace based on the image (frame) from the camera 1 (person tracking process P1) (ST101). Next, the processor 33 recognizes the work (value-added work, non-value-added work) being performed by the target worker based on the image from the camera 1 (work recognition process P2) (ST102).
[0071] Next, processor 33 determines whether the work being performed by the target worker has been completed (ST103). At this time, the completion of the work is detected when a series of actions recognized as one work is performed, and the periods during which each of the value-added work and the non-value-added work was performed are identified.
[0072] Here, when the work being performed by the target worker is completed (Yes in ST103), the processor 33 then determines the work results, i.e., the state of the object being worked on, and based on the determination results, evaluates the quality of the work performed by the worker and obtains a quality score (work quality evaluation process P12) (ST104).
[0073] Next, the processor 33 evaluates the work time of the target worker by comparing it with a preset standard time, and obtains a time score (work time evaluation process P11) (ST105). The work time is calculated by adding up the time spent on each of the value-added work and the non-value-added work.
[0074] Next, the processor 33 comprehensively evaluates the work status of the target worker based on the time score obtained in the work time evaluation process P11 and the quality score obtained in the work quality evaluation process P12, and obtains a work score (work comprehensive evaluation process P13) (ST106).
[0075] Next, the processor 33 generates information that visualizes the work status of the target worker, i.e., the implementation status of the value-added work and non-value-added work (visualization process P5) (ST107). Specifically, a graph 121 (see FIG. 7), a timeline chart 211 (see FIG. 10), and scoremarks 122, 124, 125, 321, 322, and 323 are generated.
[0076] Furthermore, the processor 33 selects content (message) to be suggested to the user for improving the work based on the evaluation results regarding the work quality and work time, i.e., the quality score and time score (suggestion content selection process P6) (ST108).
[0077] On the other hand, if the work being performed by the target worker has not been completed, i.e., if the worker is still working (No in ST103), the work quality evaluation process P12 (ST104) is omitted and the process proceeds to the work time evaluation process P11 (ST105).
[0078] Next, a description will be given of the graph screen 101 displayed on the user terminal 4. Fig. 7 is an explanatory diagram showing the graph screen 101. Fig. 8 is an explanatory diagram showing a state when the display condition setting unit 102 on the graph screen 101 is operated.
[0079] The graph screen 101 is provided with a display condition setting section 102. The display condition setting section 102 is provided with a work site designation section 111, a date and time designation section 112, a worker designation section 113, and a setting content display section 114.
[0080] The work area designation unit 111 allows the user to designate a work area to be displayed. As shown in FIG. 8A, when the user operates the work area designation unit 111, a selection menu 131 is displayed. A plurality of work areas (e.g., work areas #1 to #4) are displayed in the selection menu 131. The user can select one of the plurality of work areas in the selection menu 131.
[0081] The date and time specification unit 112 allows the user to specify the date and time to be displayed. As shown in FIG. 8(B) , when the user operates the date and time specification unit 112, a calendar 141, a time period input unit 142, and a "GO" button 143 are displayed. The user can input the date (period) to be displayed in the calendar 141. The user can also input the time period to be displayed in the time period input unit 142. When the user inputs the date and time to be displayed and operates the "GO" button 143, the date and time to be displayed is set based on the input content.
[0082] The worker designation unit 113 allows the user to designate a worker to be displayed. As shown in FIG. 8C , when the user operates the worker designation unit 113, a worker input unit 151 and a "GO" button 152 are displayed. The user can input the ID or name of the worker to be displayed in the worker input unit 151. When the user inputs the ID or name of the worker to be displayed and operates the "GO" button 152, the worker to be displayed is set based on the input content.
[0083] As shown in Fig. 7, the setting content display section 114 displays setting contents according to user's designation operations in the work place designation section 111, the date and time designation section 112, and the worker designation section 113. Specifically, the setting content display section 114 displays information about the work place, date and time, and worker designated by the user. The setting content display section 114 also has a "clear" button 115. When the user operates the "clear" button 115, the setting content is erased.
[0084] The graph screen 101 also has a graph display section 103. The graph display section 103 displays a graph 121 (visualized image) that shows the daily work time for the target worker. The graph 121 shows the work time for each of the value-added work and non-value-added work in a stacked graph. This allows the user to understand the work time for each of the value-added work and non-value-added work.
[0085] The graph display unit 103 also displays a score mark 122 (visualized image) that indicates the task score for each day. In the score mark 122, the task score is expressed as a filled-in range of stars. Note that a specified period (time period) may include multiple tasks. In this case, the task score for the specified period is calculated by averaging the task scores of each task included in the specified period.
[0086] In addition, when the user performs a mouse-over operation (temporary selection operation) on the display area of each day in the graph display unit 103, a score display box 123 pops up. A score mark 124 representing the time score and a score mark 125 representing the quality score are displayed in the score display box 123. This allows the user to check the time score and quality score that are the basis of the task score.
[0087] Furthermore, when the user clicks on the display area of each day (full selection operation) in the graph display section 103, the screen transitions to a timeline screen 201 (see FIG. 10) relating to the selected day.
[0088] Next, a description will be given of another example of the graph screen 101. FIG.
[0089] In this example, similar to the example shown in Fig. 7, a graph 121 (visualized image) showing the daily work time for the target worker is displayed in the graph display unit 103. Meanwhile, in the example shown in Fig. 7, the execution time of each of the value-added work and non-value-added work is expressed as a stacked graph in the graph 121, but in this example, the work time for each work item (A to D) is expressed as a stacked graph.
[0090] Next, a description will be given of the timeline screen 201 displayed on the user terminal 4. FIG.
[0091] The timeline screen 201 is provided with a timeline display section 202. A timeline chart 211 is displayed in the timeline display section 202. In the timeline chart 211, the horizontal axis represents time, and bars 212 representing the work status of each worker (e.g., ID=15 to 19) are displayed aligned vertically.
[0092] The timeline chart 211 visualizes the implementation status of value-added work and non-value-added work on a time axis. Specifically, bars 212 on the timeline chart 211 are drawn in different forms (e.g., colors) for value-added work periods and non-value-added work periods. This allows the user (the work manager, the worker himself) to check the implementation status of value-added work and non-value-added work for each worker.
[0093] During the specified time period (for example, 17:00-18:00), the worker stays outside the monitoring area (the area photographed by camera 1) except during the period when value-added work and non-value-added work are being performed.
[0094] In the timeline display section 202 , when the user operates the bar 212 of the timeline chart 211 , a video playback screen 301 (see FIG. 11 ) pops up on the timeline screen 201 .
[0095] In this example, the timeline chart 211 visualizes the implementation status of value-added work and non-value-added work on a time axis, but it may also visualize the implementation status of work for each work item on a time axis.
[0096] Next, a description will be given of the video playback screen 301 displayed on the user terminal 4. FIG.
[0097] The video playback screen 301 is provided with a video display section 311. The video display section 311 displays the video captured by the camera 1. The video display section 311 is also provided with a playback operation section 312. The playback operation section 312 allows the user to perform operations related to video playback. This allows the user to check the actual work situation in detail. The example shown in FIG. 11 is a case where workers are loading and unloading cargo at a truck berth.
[0098] Here, on the timeline screen 201 (see Figure 10), the period (the period from the start time to the end time of the work) for which the work (value-added work, non-value-added work) is to be performed is specified by operating the bar 212 on the timeline chart 211, and the video for that specified period is played on the video playback screen 301.
[0099] The video playback screen 301 also has a score display section 313. The score display section 313 displays a score mark 321 representing the task score, a score mark 322 representing the quality score, and a score mark 323 representing the time score.
[0100] The video playback screen 301 also has a message display section 314. Messages suggested to users (work managers and workers) for work improvement are displayed in the message display section 314. The messages are selected based on the evaluation results of the work quality and work time, i.e., the quality score and time score (see FIG. 5).
[0101] As described above, in this embodiment, by selecting a work period on the timeline screen 201 (see FIG. 10 ), video of that work period can be played back on the video playback screen 301. Therefore, for example, a user can focus on a worker with a low work rank on the graph screen 101 (see FIG. 7 ) and select the work period of interest on the timeline screen 201, thereby viewing video of the work period of interest of the focused worker on the video playback screen 301. This allows the user to check in detail the actual work status of the focused worker during the work period of interest. Furthermore, messages are displayed on the video playback screen 301, which can be useful for work managers to provide guidance for work improvement and for the workers' own self-development. Note that, in the video playback, edited video in which a worker's figure frame, a flow line representing the worker's movement trajectory, a mask image to protect the worker's privacy, and the like are superimposed on the captured video can be used to check the work status.
[0102] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. Furthermore, it is also possible to combine the components described in the above embodiments to create new embodiments.
[0103] The work analysis device and work analysis method disclosed herein have the effect of enabling a user to consider the work status of a target worker while taking into account the quality of the work performed by that worker, thereby enabling sufficient progress in improving and leveling out the quality of work, and are useful as a work analysis device and work analysis method that output analytical information about the work status of a worker based on video footage of the worker's work.
[0104] 1: Camera 2: Recorder 3: Server (work analysis device) 4: User terminal 33: Processor 101: Graph screen 103: Graph display section 121: Graph (visualized image) 122: Score mark 123: Score display box 124, 125: Score mark (visualized image) 201: Timeline screen 202: Timeline display section 211: Timeline chart (visualized image) 301: Video playback screen 311: Video display section 313: Score display section 314: Message display section 321, 322, 323: Score mark (visualized image)
Claims
1. A work analysis device that uses a processor to execute a process of outputting analytical information regarding a worker's work status based on video footage of the worker's work, wherein the processor: recognizes the work performed by the worker based on the video footage; evaluates the work quality of the work performed by the worker based on the work results; and generates the analytical information that visualizes the work status including the evaluation results of the work quality.
2. The work analysis device according to claim 1, characterized in that the processor recognizes value-added work and non-value-added work performed by a worker based on the video, and generates the analysis information that distinguishes between the value-added work and the non-value-added work.
3. The work analysis device according to claim 2, characterized in that the processor generates a visualized image representing the accumulated execution time of the value-added work and the non-value-added work as the analysis information.
4. The work analysis device according to claim 2, characterized in that the processor generates, as the analysis information, a visualized image that represents the implementation status of the value-added work and the non-value-added work on a time axis.
5. The work analysis device according to claim 1, characterized in that the processor recognizes the work performed by the worker by action recognition using a plurality of classifiers that determine the class to which the recognition target belongs based on the video.
6. The work analysis device according to claim 1, characterized in that the processor selects the content of the proposal to be presented to the user based on the evaluation results of the work quality and work time of the worker.
7. A work analysis method in which a processor executes a process to output analytical information regarding a worker's work status based on video footage of the worker's work, the work analysis method being characterized by: recognizing the work performed by the worker based on the video footage; evaluating the work quality related to the work performed by the worker based on the work results; and generating the analytical information in which the work status including the evaluation results of the work quality is visualized.
Citation Information
Patent Citations
Work added-value evaluation system, work added-value evaluation method, and program for implementing method with computer
JP2003006286A
Manufacturing process supervising support system
JP2020173753A
Seatbelt retractor with webbing guide
KR102673182B1
Work analyzing system and work analyzing method
WO2019221081A1
Operation analyzing device and operation analyzing method
WO2021131552A1
