Work situation analysis system and work situation analysis method

Abstract

To provide a work state analysis system and a work state analysis method which can automatically determine whether or not a work by a worker is appropriate while reducing a data volume to be handled.SOLUTION: In a work state analysis system 1, a communication device 21 acquires captured data obtained by capturing a work state of a worker. A processing device 22 has a work estimation model constructed by inputting a still picture obtained by capturing works by the worker and the works shown by the still picture as teacher data to perform machine learning, inputs the still picture based on the captured data into the work estimation model to estimate the works shown by each still picture, and creates first work estimation data in which estimation results of the works are chronologically arranged. The estimation results are further corrected so that the estimation results of the same works continue to the first work estimation data to create second work estimation data, and the order of the works is estimated on the basis of the created second work estimation data. Then, it is determined whether or not the order of the works performed by the worker is appropriate on the basis of the estimated order of the works.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention mainly relates to a work situation analysis system that analyzes the work situation of a worker. [Background technology]

[0002] Patent Document 1 discloses a field work implementation status management device that creates accurate work record information for a workplace. Specifically, the field work implementation status management device includes a work data acquisition unit, a work location information acquisition unit, a work time information acquisition unit, and an imaging unit. The work data acquisition unit is attached to a worker's equipment or the like and acquires data related to the work. The work location information acquisition unit acquires the location where the work is being performed by communicating with an external server. The work time acquisition unit acquires the time when the work is being performed by communicating with an external server. The imaging unit acquires video information related to the work. The field work implementation status management device creates work record information by associating a collection of multiple pieces of information in different formats with one work by adding information acquired by the work data acquisition unit, work location information acquisition unit, and work time information acquisition unit based on the video information acquired by the imaging unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-84068 Summary of the Invention [Problem to be solved by the invention]

[0004] In the method disclosed in Patent Document 1, the device cannot determine whether the work performed by the worker is appropriate, so a manager must make the determination based on video, etc. Furthermore, video information related to work is large in data volume, which can easily make it difficult to handle.

[0005] The present invention has been made in consideration of the above circumstances, and its main purpose is to provide a work status analysis system that can automatically determine whether a worker's work is appropriate while reducing the amount of data handled. [Means for solving the problem]

[0006] The problem to be solved by the present invention is as described above. Next, the means for solving this problem and the effects thereof will be explained.

[0007] According to a first aspect of the present invention, there is provided a work status analysis system having the following configuration. That is, the work status analysis system includes a communication device and a processing device. The communication device acquires photographic data capturing the work status of a worker. The processing device analyzes the work status based on the photographic data. The processing device has a work estimation model constructed by performing machine learning using still images of the worker's work and the work indicated in the still images as training data. The processing device performs a first work estimation process to input still images based on the photographic data acquired by the communication device into the work estimation model, thereby estimating the work indicated by each still image and creating first work estimation data in which the work estimation results are arranged in chronological order. The processing device performs a second work estimation process to correct the first work estimation data so that estimation results for the same work are consecutive, thereby creating second work estimation data. The processing device performs a work order estimation process to estimate the order of work based on the second work estimation data. The processing device performs a determination process to determine whether the order of work performed by the worker is appropriate by comparing the order of work estimated in the work order estimation process with predetermined criteria regarding the order of work. The processing device performs a task time estimation process to estimate a task time for each task based on the second task estimation data. When the task time for each task estimated by the task time estimation process is equal to or greater than a predetermined minimum task time and equal to or less than a predetermined maximum task time, the processing device determines that the task time for the task is appropriate. According to a second aspect of the present invention, there is provided a work status analysis system having the following configuration. That is, the work status analysis system includes a communication device and a processing device. The communication device acquires photographic data capturing the work status of a worker. The processing device analyzes the work status based on the photographic data. The processing device has a work estimation model constructed by performing machine learning using still images capturing the work of the worker and the work indicated in the still images as training data. The processing device inputs still images based on the photographic data acquired by the communication device into the work estimation model, thereby estimating the work indicated by each still image and creating first work estimation data in which the work estimation results are arranged in chronological order. The processing device then performs second work estimation processing to correct the first work estimation data so that estimation results for the same work are consecutive, thereby creating second work estimation data. The processing device then performs work order estimation processing to estimate the order of work based on the second work estimation data. The processing device performs a determination process to determine whether the order of work performed by the worker is appropriate by comparing the order of work estimated in the work order estimation process with predetermined criteria regarding the order of work. If the estimation result of the task estimation model, which inputs a still image based on the photographic data acquired by the communication device, cannot be classified into any of the tasks learned in advance, the processing device determines that the task corresponds to other tasks.

[0008] According to a third aspect of the present invention, there is provided a method for manufacturing a semiconductor device comprising: Each process is performed by a computerA work status analysis method is provided. Specifically, photographic data capturing the work status of a worker is acquired. Still images based on the photographic data are input into a work estimation model constructed by inputting still images of the worker's work and the work represented by the still images as training data and performing machine learning. The work represented by the still images is then estimated by inputting still images based on the photographic data. A first work estimation process is performed to create first work estimation data that chronologically arranges the work estimation results. A second work estimation process is performed to create second work estimation data by correcting the first work estimation data so that estimation results for the same work are consecutive. A work order estimation process is performed to estimate the order of work based on the second work estimation data. A determination process is performed to compare the order of work estimated by the work order estimation process with predetermined criteria for the order of work to determine whether the order of work performed by the worker is appropriate. A work time estimation process is performed to estimate the work time for each work based on the second work estimation data. If the work time for each work estimated by the work time estimation process is equal to or greater than the minimum time and equal to or less than the maximum time for each predetermined work time, the work time for the work is determined to be appropriate. According to a fourth aspect of the present invention, there is provided a method for manufacturing a semiconductor device comprising: Each process is performed by a computer A work situation analysis method is provided. Specifically, photographic data capturing the work situation of a worker is acquired. Still images based on the photographic data are input to a work estimation model constructed by inputting still images of the worker's work and the work represented by the still images as training data, and the work represented by the still images is then input to the model. The work represented by the still images is then estimated. A first work estimation process is performed to create first work estimation data that chronologically arranges the work estimation results. A second work estimation process is performed to create second work estimation data by correcting the first work estimation data so that estimation results for the same work are consecutive. A work order estimation process is performed to estimate the order of the work based on the second work estimation data. A determination process is performed to determine whether the order of the work performed by the worker is appropriate by comparing the order of the work estimated by the work order estimation process with predetermined criteria for the order of the work. If the estimation results of the work estimation model, which inputs the still images based on the acquired photographic data, cannot be classified into any of the previously learned work types, the model determines that the work corresponds to other work.

[0009] As a result, the tasks performed by the worker are estimated using the task estimation model, so the order of the worker's tasks can be determined with high accuracy. In particular, by using still images instead of videos as input, the amount of data required for machine learning and the amount of data for the task estimation model can be reduced. [Effects of the Invention]

[0010] According to the present invention, it is possible to realize a work status analysis system that can automatically determine whether a worker's work is appropriate while reducing the amount of data handled. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing the configuration of a work situation analysis system according to an embodiment of the present invention; [Figure 2] FIG. 10 is a diagram showing a process of constructing an operation estimation model for each process. [Figure 3] 10A and 10B are diagrams conceptually showing mapping results obtained by classifying training data through distance learning and mapping results obtained by classifying actual work. [Figure 4] 10A and 10B are diagrams showing a process of determining a work situation based on a video captured by a photographing device. [Figure 5] FIG. 10 is a diagram showing an example of detecting the occurrence of an omission of work. [Figure 6] FIG. 10 is a diagram showing an example of detecting that a work time has exceeded the limit. [Figure 7] FIG. 10 is a diagram showing an example of detecting whether an operation is normal. DETAILED DESCRIPTION OF THE INVENTION

[0012] Next, an embodiment of the present invention will be described with reference to the drawings. A work status analysis system 1 shown in Fig. 1 is a system that automatically determines whether or not a work performed by a worker is appropriate.

[0013] The work situation analysis system 1 can be applied to, for example, work for manufacturing or maintaining products in a factory, specifically, assembly, painting, cleaning, disassembly, application of lubricant, etc. The work situation analysis system 1 is not limited to work in a factory, but can be applied to any work performed by people, robots, machine tools, etc.

[0014] 1, the work of this embodiment is divided into three stages: process A, process B, and process C, and each process includes one or more operations. For example, if process A is applying grease to a first part, applying grease to a first location on the first part is operation A-1, and applying grease to a second location on the first part is operation A-2.

[0015] The work situation analysis system 1 includes a photographing device 11, a video management device 12, and a computer 13.

[0016] The image capturing device 11 is a video camera. The image capturing device 11 is placed so that the lens faces the work area, and captures images of workers performing work to generate video. The image capturing device 11 transmits the generated video (captured data) to the video management device 12. FIG. 1 shows one work area and one image capturing device 11. Alternatively, a plurality of image capturing devices 11 may be placed in each of a plurality of work areas. The image capturing device 11 is not limited to a video camera, and may be a camera that captures still images (captured data).

[0017] The video management device 12 is an information processing device such as a PC. The video management device 12 processes the video received from the image capture device 11. For example, the video management device 12 stores the video received from the image capture device 11 and further converts the video into multiple still images. The video management device 12 transmits the multiple still images to the computer 13.

[0018] The computer 13 is an information processing device such as a PC, a server, or a workstation. The computer 13 includes a communication device 21, a processing device 22, and a display device 23. The communication device 21 is a communication module or the like that performs wired or wireless communication. The communication device 21 is capable of communicating with external devices. For example, the communication device 21 can communicate with the video management device 12 to exchange data, an external server to exchange data, or a system administrator's terminal via a network. The processing device 22 includes a calculation device such as a CPU and a storage device such as an HDD or SSD. The calculation device executes a program stored in the storage device to perform a process of determining whether the work performed by the worker is appropriate based on still images received from the image capture device 11 or the video management device 12. The display device 23 is a display such as an LCD or organic EL display and can display information related to the analysis system 1. The information related to the analysis system 1 includes, for example, the current operating status and abnormality occurrence status of the analysis system 1.

[0019] The processing device 22 has a task estimation model. The task estimation model is a model constructed by machine learning, which receives a still image of the task situation as input and outputs the task content of the worker.

[0020] Next, the process of constructing an activity estimation model will be described with reference to FIGS.

[0021] In this embodiment, a task estimation model is constructed for each process. First, a video is generated by capturing images of the worker and workplace performing process A. It is preferable to generate multiple videos in order to learn various worker actions. The video may be generated using the image capturing device 11 or a separate image capturing device.

[0022] Next, learning data is generated based on the video. As shown in FIG. 2, the learning data is data consisting of multiple still images, and the tasks shown by the still images are associated with them. For example, by watching the video and specifying the time period during which task A-1 is being performed, learning data for task A-1 can be generated by extracting still images from the video during this time period at predetermined time intervals. Similarly, learning data for process A can be generated by specifying the time period during which task A-2 is being performed and the time periods during which other tasks are being performed. Other tasks are tasks other than task A-1 and task A-2, and are tasks that may normally occur in process A. For example, it is the task of a worker preparing for task A-2 after task A-1 is finished.

[0023] Next, the generated learning data is input into a machine learning computer to perform machine learning, thereby constructing an operation estimation model for process A. The machine learning computer may be the same as or different from computer 13. The machine learning in this embodiment is supervised learning, and the above-mentioned learning data is the training data. The format of the machine learning model is not limited, but for example, a model using a neural network with a general configuration can be used.

[0024] The task estimation model is constructed by analyzing the conditions when performing each task in process A based on the learning data, specifically the characteristics and trends related to the position and posture of the worker, the position and posture of the workpiece, and the position and posture of the tool.

[0025] More specifically, the task estimation model is constructed by performing distance learning based on the characteristics of each task. Distance learning is a technique in which learning data is plotted on virtual coordinates according to the characteristics of each category (in this embodiment, each process), and a range is determined for each category. The position of the virtual coordinates is determined according to the characteristics of the data, and data with similar characteristics are plotted in close positions on the virtual coordinates. In other words, as shown in Figure 3, data representing the same task is plotted within a certain range on the virtual coordinates.

[0026] In this way, an operation estimation model for process A can be constructed. In a similar manner, operation estimation models for processes B and C can be constructed. In this embodiment, an operation estimation model is created for each process, but an operation estimation model that encompasses multiple processes may also be constructed.

[0027] When using a task estimation model to estimate a task from unknown data (specifically, a still image whose task is unknown), the corresponding still image is first input into the task estimation model. The task estimation model analyzes the features of this still image and identifies the position of this still image on the virtual coordinate system. For example, if the still image is located at or near the center of the range of task A-1, the task estimation model outputs a very high probability of task A-1 as the estimation result. The closer the still image is to the center of the range of task A-1, the higher the probability of task A-1.

[0028] On the other hand, if the still image is between the range of task A-1 and the range of task A-2, the difference between the probability of task A-1 and the probability of task A-2 becomes small. In this way, when the estimation results output by the task estimation model indicate both the possibility of task A-1 and the possibility of task A-2, the processing device 22 determines that the task indicated by the still image is another task. Specifically, if the difference between the most probable estimation result and the second most probable estimation result is less than a threshold, the processing device 22 determines that the task is another task. This is because if the task closest in the virtual coordinate system (the task with the highest probability) is estimated, there is a possibility that the task estimation will be incorrect, resulting in an incorrect determination of the task status. Furthermore, in such cases, it is preferable to perform additional learning as needed. Additional learning is a process of updating the task estimation model by performing additional machine learning using data that is not located within the range of either task. By performing additional learning, the range indicated by each task is corrected, thereby improving the accuracy of task estimation.

[0029] Next, with reference to FIG. 4, a method for determining whether or not the work performed by a worker is appropriate using a work estimation model will be described.

[0030] The image capturing device 11 captures an image of a worker performing a task, generates a video, and transmits the video to the video management device 12. The orientation and capturing range of the image capturing device 11 are preferably the same as when the learning data was generated.

[0031] Next, the video management device 12 performs image extraction processing on the video received from the image capture device 11 to generate multiple still images. The image extraction processing is a process of cutting out the video at a predetermined time interval to generate still images. The time interval for generating still images may be a fixed value, or the time interval may be changed depending on the situation. For example, if there is little change in the worker and surrounding situation captured in the video, the time interval may be longer than usual. Furthermore, the image capture device 11 may transmit still images to the computer 13 at the time interval. In other words, in this case, the image extraction processing can be omitted.

[0032] Furthermore, the video management device 12 associates the still image with the shooting time and the process, and transmits the still image to the computer 13. For example, a still image based on a video taken while a worker is performing process A is transmitted to the computer 13 in association with the shooting time and process A.

[0033] There are various ways in which the video management device 12 identifies a process, but for example, a process can be identified as follows. For example, if only process A is performed in a certain workplace, the video management device 12 associates process A with a still image based on video captured by a camera 11 located in that workplace. Alternatively, if processes A and B are performed in a certain workplace, the video management device 12 associates process A or process B with a still image based on video, based on a predetermined schedule or a timestamp of a work instruction device. Note that if a work estimation model that encompasses all processes is created, there is no need to associate the processes.

[0034] Next, the processing device 22 of the computer 13 performs a first estimation process based on the received multiple still images and their associated shooting times and processes. The first estimation process is a process in which the received still images are input into a task estimation model to obtain an estimation result of the task indicated by the still images. The still images are also associated with shooting times and processes. The processing device 22 inputs the still images into a task estimation model corresponding to the process associated with the still images. Specifically, if the still image is associated with process A, the processing device 22 inputs the still image into the task estimation model for process A. Note that instead of the process in which the video management device 12 associates the still images with processes, the computer 13 may estimate the task related to the still images based on the task estimation model and switch the task estimation model to be applied to the still images depending on the task estimation result.

[0035] The processing device 22 creates first task estimation data shown in Fig. 4 based on the estimation results of the task estimation model and the shooting times associated with the still images. The first task estimation data is obtained by plotting the estimation results of the tasks of the workers according to the shooting times. In the example shown in Fig. 4, the number of plots per task is shown as around five to make the first task estimation data easier to understand, but in reality, it is likely that a larger number of plots will be required.

[0036] Next, the processing device 22 performs a second estimation process. The second estimation process is a process for correcting the first task estimation data. The estimation result for each still image may contain an estimation error. For example, in the first task estimation data shown in FIG. 4, the area surrounded by a dashed square is an estimation error. Generally, it is not conceivable that a worker will perform a different task for just a moment while performing a certain task.

[0037] The processing device 22 performs the following process, for example. First, the processing device 22 selects an arbitrary task and identifies a location where the number of consecutive plots for this task is equal to or less than a threshold value. For example, the processing device 22 selects task A-2 and identifies a location where the number of consecutive plots for task A-2 is 1 (the third plot). Next, the processing device 22 corrects the identified plot to a different task so that the estimation results for the same plot are consecutive. In the example shown in FIG. 4, the estimation results before and after the third plot are task A-1, so task A-2 is corrected to task A-1. The processing device 22 performs this process for all tasks. As a result, the estimation results for the same task become consecutive. Note that, after considering whether or not a correction is necessary, the processing device 22 does not perform the correction if it determines that a correction is not necessary.

[0038] Note that the above-described process is merely an example, and other processes may be performed as long as estimation results for the same task are consecutive. For example, the processing device 22 selects an arbitrary task and identifies a time period in which the proportion of this task is equal to or greater than a threshold. For example, the processing device 22 selects task A-1 and identifies a time period in which the proportion of task A-1 is equal to or greater than a threshold. In the example shown in FIG. 4, four of the first to fifth plots are task A-1, so the processing device 22 identifies the first to fifth plots. Next, the processing device 22 identifies an estimation result for the identified time period that is different from the initially selected task and corrects this estimation result to the initially selected task. In the example shown in FIG. 4, the third plot is task A-2, so it is corrected to task A-1. This results in consecutive estimation results for the same task.

[0039] Next, the processing device 22 performs a task order estimation process and a task time estimation process based on the second task estimation data. The task order estimation process is a process for estimating the order of tasks performed by the worker. Since the second task estimation data contains consecutive estimation results for the same task, the order of tasks performed by the worker can be estimated by arranging the tasks indicated by this series of estimation results.

[0040] The task time estimation process is a process for estimating the task time for each task performed by a worker. Since the second task estimation data contains consecutive estimation results for the same task, the task time for each task can be estimated based on the number of consecutive plots for that task and the interval between the capture times of the still images.

[0041] The processing device 22 performs a task order estimation process and a task time estimation process to create the judgment data shown in Fig. 4. The judgment data is data that lists the tasks performed by the worker in the order of the tasks and describes the task time for each task. Based on the judgment data, the processing device 22 judges whether the tasks performed by the worker are appropriate.

[0042] If the processing device 22 determines that the work performed by the worker is inappropriate, it notifies a higher-level control device, a manager's terminal, or the like of that fact. Furthermore, the processing device 22 enables the manager to check still images or videos taken during the time period when the work was determined to be inappropriate, so that the manager can confirm the specific work performed by the worker. Specifically, the processing device 22 may transmit still images or videos taken during the time period when the work was determined to be inappropriate to the manager's terminal, or may store these still images or videos on a server and transmit link information for accessing the server to the manager's terminal.

[0043] Next, a specific example of determining whether or not the work performed by a worker is appropriate will be described with reference to FIGS.

[0044] 5 to 7 show the second task estimation data, the reference data, the data for evaluation, and the evaluation results, respectively. The reference data is data indicating the criteria for determining whether the tasks performed by a worker are appropriate. The procedures and task details in the reference data describe the appropriate order of tasks. The minimum time is the minimum time set for each task. The maximum time is the maximum time set for each task. In other words, if each task is shorter than the minimum time or longer than the maximum time, the task performed by the worker is determined to be inappropriate. In the examples shown in FIGS. 5 to 7, there is one reference data for each task. However, if, for example, the task order is allowed to be changed, there may be multiple reference data. Specifically, in the example shown in FIG. 5, if the task order of tasks A-1 and A-2 is allowed to be changed, two reference data are used, one in which task A-1 is placed first and the other in which task A-2 is placed first. If the task performed by the worker satisfies at least one reference data, the task is determined to be appropriate.

[0045] Figure 5 shows the data for each case when an omission occurs. Steps 4, 5, and 6 of the reference data respectively list task A-1, task A-2, and others. However, in the judgment data, task A-1 is followed by others, and task A-2 in step 5 is missing. In this case, the processing device 22 determines that an omission has occurred. Note that when an omission occurs, the priority level is set to "high" because it affects product quality. When the priority level is "high," a notification is immediately sent to the higher-level control device or the manager's terminal, etc., as described above. This notification includes information that an "omission of task" has occurred, which is the reason why the worker's work was determined to be inappropriate.

[0046] FIG. 6 shows the data for each case when a time overrun occurs. Step 5 in the reference data indicates that the maximum time for task A-2 is "5 seconds." However, the judgment data indicates that the task time for step 5 is "6 seconds." In this case, the processing device 22 determines that a time overrun has occurred. Tasks A-1 and A-2 shown in FIG. 6 are tasks for which the quality of the product will not be affected even if a time overrun occurs, and therefore the priority level for the time overrun is set to "medium." Therefore, the processing device 22 does not immediately notify the higher-level control device or the manager's terminal, etc., but instead notifies them collectively at a predetermined notification timing. However, for tasks for which the quality of the product will be affected if a time overrun occurs, the priority level for the time overrun is set to "high." In this case, when a time overrun occurs, the processing device 22 immediately notifies the higher-level control device or the manager's terminal, etc. This notification includes information about the occurrence of a "time overrun," which is the reason for determining that the worker's work is inappropriate.

[0047] Figure 7 shows the data for cases where the work performed by the worker is appropriate. The order of work and the work time of the judgment data in Figure 7 both meet the reference data. Therefore, the processing device 22 judges that the work performed by the worker is appropriate.

[0048] By performing the above-described processing, it is possible to automatically determine whether the work performed by the worker is appropriate. Furthermore, if it is determined to be inappropriate, the reason for the determination (i.e., the items that do not meet the standard data) can also be notified to the administrator. Furthermore, compared to systems that learn from videos, create models, and make judgments based on the videos, the amount of data handled can be reduced.

[0049] The processing device 22 stores the judgment data in the processing device 22 or an external server, regardless of whether the work performed by the worker is appropriate or not. This allows the work of the worker to be recorded. Note that still images input to the work estimation model may also be stored in addition to the judgment data. Since still images have a smaller data volume than videos, the amount of data to be stored can be reduced. Furthermore, by storing judgment data, videos, or still images only when the work is determined to be inappropriate, the amount of data to be stored can also be reduced.

[0050] As described above, the work status analysis system 1 of this embodiment includes a communication device 21 and a processing device 22, and performs the following work status analysis method. The communication device 21 acquires photographic data capturing the work status of a worker. The processing device 22 analyzes the work status based on the photographic data. The processing device 22 has a work estimation model constructed by performing machine learning using still images of the worker's work and the work indicated by the still images as training data. The processing device 22 inputs still images based on the photographic data acquired by the communication device 21 into the work estimation model, thereby estimating the work indicated by each still image and creating first work estimation data in which the work estimation results are arranged in chronological order. The processing device 22 performs second work estimation processing to create second work estimation data by correcting the estimation results of the first work estimation data so that estimation results for the same work are consecutive. The processing device 22 performs work order estimation processing to estimate the order of work based on the second work estimation data. The processing device 22 performs a determination process to determine whether the order of work performed by the worker is appropriate by comparing the order of work estimated in the work order estimation process with predetermined criteria regarding the order of work.

[0051] As a result, the tasks performed by the worker are estimated using the task estimation model, so the order of the worker's tasks can be determined with high accuracy. In particular, by using still images instead of videos as input, the amount of data required for machine learning and the amount of data for the task estimation model can be reduced.

[0052] In the work status analysis system 1 of this embodiment, if the number of consecutive estimation results for the same work in the first work estimation data is equal to or less than a threshold, the estimation result is corrected to a different work, thereby making the estimation results for the same work consecutive.

[0053] Since it is not normal for the work performed by a worker to change for just an instant, it is possible to correct errors in work estimation.

[0054] In the work status analysis system 1 of this embodiment, the processing device 22 performs a work time estimation process to estimate the work time for each work based on the second work estimation data. The processing device 22 compares the work time for each work estimated by the work time estimation process with a predetermined work time for each work to determine whether the work time for the work performed by the worker is appropriate.

[0055] This makes it possible to determine whether the work order and the work time are appropriate. Therefore, if the work time for a certain work significantly exceeds the reference time, the cause can be identified and resolved, thereby improving work efficiency.

[0056] In the work status analysis system 1 of this embodiment, the processing device 22 determines that the work time for a task is appropriate if the work time for each task estimated by the work time estimation process is equal to or greater than the minimum time and equal to or less than the maximum time for each task that has been predetermined.

[0057] This allows for flexible determination of whether the work time is appropriate or not.

[0058] In the work status analysis system 1 of this embodiment, if the estimation result of the work estimation model, which inputs a still image based on the photographic data acquired by the communication device 21, cannot be classified into any of the previously learned works, the processing device 22 determines that it corresponds to other work.

[0059] If a still image is input and the task estimation model outputs the most probable task as the estimation result, there is a possibility of misjudgment. In such cases, additional learning can be performed as necessary to further improve the task estimation accuracy.

[0060] In the work status analysis system 1 of this embodiment, the processing device 22 determines whether the work performed by the worker is appropriate, and if it is determined to be inappropriate, notifies the worker of the reason why it is determined to be inappropriate.

[0061] This allows the administrator to check the work of the worker while taking into consideration the reason why the work was judged to be inappropriate, thereby reducing the effort required to check the work of the worker.

[0062] The preferred embodiment of the present invention has been described above, but the above configuration can be modified, for example, as follows.

[0063] The processing performed by the video management device 12 may be performed by the image capture device 11 or the computer 13 instead, and the video management device 12 may be omitted.

[0064] The imaging device 11 and the computer 13 do not have to be installed in the same factory, and the computer 13 may be installed in a facility away from the factory.

[0065] In the above embodiment, the processing device 22 performs both the task sequence estimation process and the task time estimation process, but the task time estimation process may be omitted. [Explanation of symbols]

[0066] 1. Work situation analysis system 11 Imaging equipment 12 Video management device 13. Computer 21 Communication equipment 22 Processing equipment 23 Display device

Claims

1. a communication device for acquiring photographic data of the worker's working status; a processing device that analyzes the work situation based on the photographed data; Equipped with the processing device has a task estimation model constructed by performing machine learning using a still image of a worker's task and the task shown in the still image as training data; the processing device performs a first task estimation process to estimate tasks indicated by each still image by inputting still images based on the photographic data acquired by the communication device into the task estimation model, and to create first task estimation data in which task estimation results are arranged in chronological order; the processing device performs a second task estimation process for correcting the estimation results of the first task estimation data so that estimation results of the same task are continuous, thereby creating second task estimation data; the processing device performs a task order estimation process to estimate a task order based on the second task estimation data; the processing device performs a determination process to determine whether the order of the tasks performed by the workers is appropriate by comparing the order of the tasks estimated in the task order estimation process with a predetermined standard for the order of the tasks; the processing device performs a task time estimation process to estimate a task time for each task based on the second task estimation data; The processing device determines that the work time for each task is appropriate if the work time for each task estimated by the work time estimation process is greater than or equal to the minimum time and less than or equal to the maximum time of the predetermined work time for each task.

2. a communication device for acquiring photographic data of the worker's working status; a processing device that analyzes the work situation based on the photographed data; Equipped with the processing device has a task estimation model constructed by performing machine learning using a still image of a worker's task and the task shown in the still image as training data; the processing device performs a first task estimation process to estimate tasks indicated by each still image by inputting still images based on the photographic data acquired by the communication device into the task estimation model, and to create first task estimation data in which task estimation results are arranged in chronological order; the processing device performs a second task estimation process for correcting the estimation results of the first task estimation data so that estimation results of the same task are continuous, thereby creating second task estimation data; the processing device performs a task order estimation process to estimate a task order based on the second task estimation data; the processing device performs a determination process to determine whether the order of the tasks performed by the workers is appropriate by comparing the order of the tasks estimated in the task order estimation process with a predetermined standard for the order of the tasks; The processing device is configured to determine that the estimation result of the task estimation model, which receives as input a still image based on the photographic data acquired by the communication device, falls under other tasks when the estimation result cannot be classified into any of the tasks learned in advance.

3. The work situation analysis system according to claim 1 or 2, A work status analysis system characterized in that, when the number of consecutive estimation results for the same work in the first work estimation data is below a threshold, the estimation result is corrected to a different work, thereby making the estimation results for the same work consecutive.

4. The work situation analysis system according to claim 2, the processing device performs a task time estimation process to estimate a task time for each task based on the second task estimation data; The processing device determines whether the work time for each task performed by the worker is appropriate by comparing the work time for each task estimated by the work time estimation process with a predetermined work time for each task.

5. The work situation analysis system according to any one of claims 1 to 4, The processing device determines whether the work performed by the worker is appropriate, and if it determines that the work is inappropriate, notifies the worker of the reason why it is determined to be inappropriate.

6. The computer acquires the photographic data of the worker's work situation, A still image of a worker's work and the work shown in the still image are input as training data to a work estimation model constructed by machine learning. A still image based on the photographed data is then input to the work estimation model, and the computer estimates the work shown in the still image. a computer performs a first task estimation process to create first task estimation data in which task estimation results are arranged in chronological order; a computer performs a second task estimation process to correct the first task estimation data so that estimation results of the same task are consecutive, thereby creating second task estimation data; a computer performs a task sequence estimation process to estimate a task sequence based on the second task estimation data; a computer performs a determination process to determine whether the order of the tasks performed by the worker is appropriate by comparing the order of the tasks estimated by the task order estimation process with a predetermined standard for the order of the tasks; a computer performs a task time estimation process to estimate a task time for each task based on the second task estimation data; A work status analysis method characterized in that a computer determines that the work time for a task is appropriate if the work time for each task estimated by the work time estimation process is equal to or greater than a predetermined minimum time and equal to or less than a predetermined maximum time for each task.

7. The computer acquires the photographic data of the worker's work situation, A still image of a worker's work and the work shown in the still image are input as training data to a work estimation model constructed by machine learning. A still image based on the photographed data is then input to the work estimation model, and the computer estimates the work shown in the still image. a computer performs a first task estimation process to create first task estimation data in which task estimation results are arranged in chronological order; a computer performs a second task estimation process to correct the first task estimation data so that estimation results of the same task are consecutive, thereby creating second task estimation data; a computer performs a task sequence estimation process to estimate a task sequence based on the second task estimation data; a computer performs a determination process to determine whether the order of the tasks performed by the worker is appropriate by comparing the order of the tasks estimated by the task order estimation process with a predetermined standard for the order of the tasks; a computer determining that the estimation result of the task estimation model, which receives as input a still image based on the acquired photographic data, corresponds to other tasks when the estimation result cannot be classified into any of the tasks learned in advance.

Images