Work analysis device, work analysis method, and program

Abstract

To provide a work analysis device that can appropriately analyze work. [Solution] The work analysis device 100 includes an image acquisition unit 111 that acquires a first video image in which a work area, which is at least a part of the worker K performing the work, is displayed in the imaging area; a position detection unit 112 that detects the time change in the position of the work area displayed in the first video image as movement of the work area; a frequency information generation unit 113 that generates frequency information D1 which indicates the frequency at which a predetermined movement occurs among the detected movements of the work area, in relation to the position in the imaging area where the predetermined movement occurred; and a region information generation unit 114 that identifies one or more regions in the imaging area that include a position in the frequency information D1 that satisfies a predetermined condition as a work area, and generates region information D2 which indicates one or more identified work areas.

Description

Technical Field

[0001] The present disclosure relates to a work analysis device for analyzing work, etc.

Background Art

[0002] Conventionally, a residence status analysis device has been proposed that acquires residence information regarding the residence status of a moving object within a target area, generates a heat map image in which the residence information is visualized, and displays the heat map image on a display device (see, for example, Patent Document 1). The moving object is, for example, a person. Thereby, the user can grasp the residence status of the person within the target area by looking at the heat map image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the residence status analysis device of Patent Document 1 above has a problem that it is difficult to appropriately analyze the work performed by an operator.

[0005] Therefore, the present disclosure provides a work analysis device that can appropriately analyze work, etc.

Means for Solving the Problems

[0006] A work analysis device according to one aspect of the present disclosure includes: an image acquisition unit that acquires a first moving image in which a work area, which is at least a part of a worker performing work, is shown in the imaging area; a position detection unit that detects the time change in the position of the work area shown in the first moving image as movement of the work area; a frequency information generation unit that generates frequency information indicating the frequency at which a predetermined movement occurs among the detected movements of the work area, in relation to the position in the imaging area where the predetermined movement occurred; and a region information generation unit that identifies one or more regions in the imaging area that include a position in the frequency information that satisfies a predetermined condition as a work area, and generates region information indicating one or more identified work areas.

[0007] These comprehensive or specific embodiments may be implemented as a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, and recording medium. Furthermore, the recording medium may be a non-temporary recording medium. [Effects of the Invention]

[0008] The work analysis device described herein can appropriately analyze work.

[0009] Further advantages and effects of one aspect of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by several embodiments and configurations described in the specification and drawings, but not all configurations are necessarily required. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows an example of the overall configuration of the work analysis system in the embodiment. [Figure 2] Figure 2 is a diagram illustrating the line work and imaging by multiple cameras in the embodiment. [Figure 3] Figure 3 is a diagram illustrating the work performed by the workers. [Figure 4] Figure 4 is a block diagram showing an example of the configuration of a work analysis device in an embodiment. [Figure 5] Figure 5 is a diagram illustrating the processing performed by the position detection unit in the embodiment. [Figure 6] Figure 6 is a diagram illustrating an example of the first analysis process in the embodiment. [Figure 7] Figure 7 is a diagram illustrating an example of processing by the work estimation unit included in the second analysis process in the embodiment. [Figure 8] Figure 8 is a diagram illustrating an example of other processing performed by the work estimation unit included in the second analysis process in the embodiment. [Figure 9] Figure 9 is a diagram illustrating an example of processing by the visualization analysis unit included in the third analysis process in the embodiment. [Figure 10] Figure 10 is a diagram illustrating another example of processing by the visualization analysis unit included in the third analysis process in the embodiment. [Figure 11] Figure 11 is a diagram illustrating another example of processing by the visualization analysis unit included in the third analysis process in the embodiment. [Figure 12] Figure 12 is a diagram illustrating yet another example of processing by the visualization analysis unit included in the third analysis process in the embodiment. [Figure 13] Figure 13 illustrates yet another example of processing by the visualization analysis unit included in the third analysis process in the embodiment. [Figure 14] Figure 14 is a diagram illustrating another example of frequency information in the embodiment. [Figure 15] Figure 15 is a diagram illustrating other processing performed by the frequency information generation unit in the embodiment. [Figure 16] Figure 16 is a flowchart showing an example of the processing operation of the work analysis device in the embodiment. [Figure 17]FIG. 17 is a flowchart showing an example of the first analysis process by the work analysis device in the embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of the second analysis process by the work analysis device in the embodiment. [Figure 19] FIG. 19 is a flowchart showing an example of the third analysis process by the work analysis device in the embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0011] The work analysis device according to the first aspect of the present disclosure includes an image acquisition unit that acquires a first moving image in which at least a part of a worker who is performing work, i.e., a work part, is reflected in an imaging area; a position detection unit that detects a temporal change in the position of the work part reflected in the first moving image as the movement of the work part; a frequency information generation unit that generates frequency information indicating the frequency of a predetermined movement among the detected movements of the work part in association with the position in the imaging area where the predetermined movement is performed; and an area information generation unit that specifies each of one or more areas including the position where a frequency satisfying a predetermined condition is indicated in the frequency information in the imaging area as a work area, and generates area information indicating the specified one or more work areas.

[0012] As a result, region information indicating one or more work regions in the imaging region is generated, so that it is possible to easily grasp from the region information that work corresponding to each of the one or more work regions has been performed by the worker. Therefore, the work can be appropriately analyzed. That is, in the residence status analysis device of Patent Document 1 described above, a heat map image is generated, but the above-described work region is not shown in the heat map image. Specifically, the heat map image does not show a work region indicating a work site that has been performed a frequency satisfying a predetermined condition by the work of the worker. Therefore, it is impossible to grasp from the heat map image what kind of work has been performed by the worker. On the other hand, in the first aspect of the present disclosure, it is possible to easily grasp what kind of work has been performed by the worker.

[0013] Further, in the work analysis device according to the second aspect, the frequency information generation unit may specify a stay among the detected movements of the work site as the predetermined movement. Note that the second aspect may be subordinate to the first aspect.

[0014] As a result, frequency information is generated that shows the frequency of stays at the work site in association with the position where the stays are made. Then, for example, region information is generated that shows each of one or more regions with many stays in the imaging region as a work region. As a result, work can be appropriately associated with each of the one or more work regions, and it is possible to more easily grasp from the region information that those works have been performed by the worker.

[0015] Further, in the work analysis device according to the third aspect, the frequency satisfying the predetermined condition may be a frequency equal to or higher than a predetermined threshold value. Note that the third aspect may be subordinate to the first aspect or the second aspect.

[0016] As a result, it is possible to generate region information that appropriately shows each of one or more regions with many predetermined movements such as stays as a work region.

[0017] Furthermore, in the work analysis device according to the fourth embodiment, the frequency information may be configured as a heat map. Note that the fourth embodiment may be subordinate to any one of the first to third embodiments.

[0018] This allows for easy visual identification of locations with high or low frequency of occurrences within the imaging area, based on frequency information.

[0019] Furthermore, in the work analysis device according to the fifth embodiment, the image acquisition unit further acquires a second video image in the imaging area showing the work area of ​​the worker performing the work during the estimated target period, the position detection unit further detects the time change in the position of the work area shown in the second video image as the movement of the work area, and the work analysis device further includes a work estimation unit that estimates the history of one or more tasks performed by the worker during the estimated target period based on the movement of the work area detected based on the second video image and the area information. Note that the fifth embodiment may be dependent on any one of the first to fourth embodiments.

[0020] This allows for the estimation of the history of one or more tasks during the estimated period based on the domain information, making it possible to accurately estimate when each task corresponding to a specific work domain indicated in the domain information was performed.

[0021] Furthermore, in the work analysis device according to the sixth embodiment, the work estimation unit may estimate the history of one or more of the work by estimating, for each of the one or more work areas indicated in the area information, the period during which the work part was present in the work area within the estimation target period, as the net work period during which the work corresponding to the work area was performed. Note that the sixth embodiment may be dependent on the fifth embodiment.

[0022] This allows for an accurate estimation of the net work period during which work corresponding to the work area was performed.

[0023] Furthermore, in the work analysis apparatus according to the seventh embodiment, if the work estimation unit estimates a period during the estimation target period in which no work part exists in any of the one or more work areas indicated in the area information, immediately before the net work period, the said period may be estimated as a preparation period for the work corresponding to the net work period. Note that the seventh embodiment may be dependent on the sixth embodiment.

[0024] This allows for the accurate estimation of not only the net working period but also the preparation period for the work corresponding to that net working period.

[0025] Furthermore, in the work analysis device according to the eighth embodiment, the frequency information generation unit may identify movements within a predetermined range of speeds among the detected movements of the work part as the predetermined movements. Note that the eighth embodiment may be subordinate to the first embodiment.

[0026] This generates frequency information that shows how often the work area moves within a predetermined speed range, and associates this movement with the location where it occurred. Then, for example, region information is generated that identifies one or more regions within the imaging area where movement within the predetermined speed range is frequent as work areas. As a result, it becomes possible to appropriately associate work with each of these one or more work areas, and it becomes even easier to understand from the region information that those work was performed by a worker.

[0027] Furthermore, in the work analysis device according to the ninth embodiment, the frequency information generation unit may identify the movement of the detected work part at a predetermined range of accelerations as the predetermined movement. Note that the ninth embodiment may be subordinate to the first embodiment.

[0028] This generates frequency information that shows how often the work area moves within a predetermined acceleration range, and associates this information with the location where the movement occurred. Then, for example, region information is generated that identifies one or more regions within the imaging area that frequently experience movement within the predetermined acceleration range as work areas. As a result, it becomes possible to appropriately associate work with each of these one or more work areas, and it becomes even easier to understand from the region information that those work was performed by a worker.

[0029] Furthermore, the work analysis method according to the first aspect of this disclosure is a work analysis method performed by a computer, which acquires a first video image in which a work area that is at least a part of a worker performing a work is shown in the imaging area, detects the time change of the position of the work area shown in the first video image as movement of the work area, generates frequency information that shows the frequency at which a predetermined movement occurs among the detected movements of the work area in relation to the position in the imaging area where the predetermined movement occurred, identifies one or more areas in the imaging area that include a position in the frequency information that satisfies a predetermined condition as a work area, and generates area information that indicates one or more identified work areas.

[0030] This makes it possible to achieve the same effects and advantages as the work analysis apparatus according to the first embodiment.

[0031] The embodiments will be described in detail below with reference to the drawings.

[0032] The embodiments described below are all general or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept are described as optional components. In addition, each figure is a schematic diagram and is not necessarily a strictly accurate representation. Also, the same reference numerals are used for the same components in each figure.

[0033] (Embodiment) Figure 1 shows an example of the overall configuration of the work analysis system in this embodiment.

[0034] The work analysis system 10 in this embodiment is a system that analyzes the work of multiple workers performing, for example, line work (i.e., assembly line work), and comprises a work analysis device 100, a display unit 1, an input unit 2, and multiple cameras 3.

[0035] The display unit 1 is, for example, an LCD (Liquid Crystal Display). However, the display unit 1 may also be a device other than an LCD, such as an organic light-emitting diode (electroluminescent) or a plasma display. Such a display unit 1 displays an image corresponding to the image signal output from the work analysis device 100.

[0036] The input unit 2 is configured as, for example, a keyboard, touch sensor, touchpad, or mouse, and receives user input operations and outputs an input signal corresponding to those operations to the work analysis device 100.

[0037] Each of the multiple cameras 3 captures an image of a worker performing the tasks included in the aforementioned line work, and outputs the resulting video image to the work analysis device 100.

[0038] The work analysis device 100 acquires video images from each of the multiple cameras 3 and analyzes the work performed by the worker shown in the video images. The work analysis device 100 then outputs an image signal showing the analysis results to the display unit 1.

[0039] In this embodiment, the work analysis system 10 is equipped with multiple cameras 3, but the number of cameras 3 equipped in the work analysis system 10 may be just one. For example, one camera 3 may capture images of multiple workers. Also, in this embodiment, the work analysis system 10 analyzes the work of multiple workers performing line work, but the number of workers analyzed may be just one.

[0040] Figure 2 is a diagram illustrating the assembly line process and imaging by multiple cameras 3.

[0041] An assembly line consists of n steps (where n is an integer greater than or equal to 2). The n steps are the first step, the second step, ..., the (n-1)th step, and the nth step. In each of the n steps, an operator performs one or more tasks in sequence. This set of one or more tasks performed sequentially by an operator is also called a cycle. Each task is, for example, an operation on a workpiece w placed on a workbench 4. The workpiece w may be, for example, a circuit board on which electronic components are mounted. The operation on the workpiece w may be, for example, an operation to attach material m, such as a component or screw, to the workpiece w. The first step, the second step, ..., and the nth step are performed by operators K1, K2, ..., and Kn, respectively.

[0042] Specifically, in the first step, worker K1 performs a cycle that includes attaching, for example, material m to a workpiece w placed on the workbench 4 for the first step. Then, worker K1 places the completed workpiece w obtained from the first step onto the platform 5 for the second step. In the second step, worker K2 moves the workpiece w placed on the platform 5 for the second step to the workbench 4 for the second step and performs the cycle assigned to the second step on the workpiece w that is placed there. Then, worker K2 places the completed workpiece w obtained from the second step onto the platform 5 for the third step. In this way, by performing each operation from the first to the nth step on the workpiece w, a product (for example, a mounted circuit board) containing the workpiece w is manufactured. When another new workpiece w is placed on the workbench 4 for the first step, each operation from the first to the nth step is performed on the new workpiece w in the same manner as described above. In other words, the cycle is repeatedly executed in each step.

[0043] The number of cameras 3 is, for example, n, and each of the n cameras 3 is assigned to one of the n processes. That is, one camera 3 captures an imaging area including the workbench 4 for the first process, and another camera 3 captures an imaging area including the workbench 4 for the second process.

[0044] The work analysis device 100 acquires moving images from each of the n cameras 3, and analyzes the work being performed in each process.

[0045] Figure 3 is a diagram illustrating the work performed by the workers.

[0046] As shown in Figure 3, worker K performs work on a workpiece w placed on a workbench 4. Worker K is one of the workers K1 to Kn described above. For example, worker K picks up a material m, such as a screw, placed on the workbench 4 and fastens the material m to the workpiece w. Camera 3 is fixed to a support column 6 that supports the workbench 4 and captures a predetermined imaging area. This imaging area includes the top surface of the workbench 4. As a result, camera 3 acquires a moving image showing the working area of ​​worker K, the workpiece w, and the material m. The working area is worker K's two hands, arms, etc. One or more tools may be placed in predetermined positions on the workbench 4, and worker K may use those tools to perform the work.

[0047] Figure 4 is a block diagram showing an example configuration of the work analysis device 100.

[0048] The work analysis device 100 comprises a control unit 101, an image acquisition unit 111, a position detection unit 112, a frequency information generation unit 113, a region information generation unit 114, a storage unit 115, a work estimation unit 116, a visualization analysis unit 117, and an output unit 118. This work analysis device 100 performs a first analysis process, a second analysis process, and a third analysis process. The first analysis process is the process up to generating region information D2, which will be described later. The second analysis process is the process of analyzing what kind of work worker K performed during the estimated period using the region information D2. The third analysis process is the process of improving work efficiency and visualizing work status using the results of the first and second analysis processes.

[0049] The control unit 101 acquires an input signal from the input unit 2 and controls each component of the work analysis device 100 other than the control unit 101 in accordance with the input signal. For example, the control unit 101 may selectively switch between executing the first analysis process, the second analysis process, and the third analysis process in accordance with the input signal from the input unit 2.

[0050] The image acquisition unit 111 acquires moving images from each of the multiple cameras 3, based on the images captured by each camera 3. The moving images show a work area, which is at least a part of the worker K performing the work, within the imaging area.

[0051] The position detection unit 112 detects the position of each feature point in the work area of ​​worker K shown in each frame of the video acquired by the image acquisition unit 111. Feature points are, for example, the bones of worker K's hand or arm, more specifically, the joints of the hand or arm. In other words, the position detection unit 112 detects the time change in the position of the work area shown in the video as the movement of the work area. The position detection unit 112 then outputs the time change in the detected position of each feature point, i.e., information indicating the movement of the work area, as position time-series information to the frequency information generation unit 113 or the work estimation unit 116.

[0052] When the frequency information generation unit 113 acquires position time-series information from the position detection unit 112, it generates frequency information D1 based on the time changes in the position of each feature point shown in the position time-series information. The frequency information D1 is information that indicates the frequency at which a predetermined movement occurs among the detected movements of the work area, and associates this with the position in the imaging area where the predetermined movement occurred. The frequency information generation unit 113 then stores the generated frequency information D1 in the storage unit 115. Alternatively, the frequency information generation unit 113 may output the frequency information D1 to the area information generation unit 114.

[0053] The region information generation unit 114 obtains frequency information D1 from the frequency information generation unit 113. Alternatively, the region information generation unit 114 obtains frequency information D1 stored in the storage unit 115. The region information generation unit 114 identifies one or more regions within the imaging region that include a position in the frequency information D1 that satisfies a predetermined condition as a work region, and generates region information D2 indicating the identified one or more work regions. The region information generation unit 114 then stores the generated region information D2 in the storage unit 115.

[0054] The above-described first analysis process is performed by the image acquisition unit 111, position detection unit 112, frequency information generation unit 113, and region information generation unit 114. The video image obtained by the camera 3 for this first analysis process is also referred to as the first video image. This first video image is obtained by imaging over a period in which multiple cycles are performed. This first analysis process may also be called a learning process for generating region information D2.

[0055] The storage unit 115 is a recording medium for storing frequency information D1 and area information D2. Specifically, the storage unit 115 is a hard disk drive, RAM (Random Access Memory), ROM (Read Only Memory), or semiconductor memory. The storage unit 115 may be volatile or non-volatile.

[0056] After the first analysis process, the work estimation unit 116 obtains the location time-series information of worker K from the location detection unit 112, and further obtains the area information D2 of worker K from the storage unit 115. Then, based on the location time-series information and the area information D2, the work estimation unit 116 estimates the history of one or more tasks performed by worker K during the estimation period.

[0057] Furthermore, this location time-series information is different from the location time-series information generated for the first analysis process; it is location time-series information for the second analysis process. In other words, the location time-series information acquired by the work estimation unit 116 is generated by the location detection unit 112 based on the video footage acquired by the image acquisition unit 111 after the first analysis process. The video footage acquired by the image acquisition unit 111 after the first analysis process will hereafter be referred to as the second video footage to distinguish it from the first video footage.

[0058] Specifically, the image acquisition unit 111 acquires a second video image from each camera 3 after the first analysis process, which shows the work area of ​​worker K performing work during the estimated period in the imaging area. The position detection unit 112 then detects the time change in the position of the work area shown in the second video image as the movement of that work area. This generates position time-series information indicating the detected movement of the work area and outputs it to the work estimation unit 116. The work estimation unit 116 estimates the history of one or more tasks performed by worker K during the estimated period based on this position time-series information, i.e., the movement of the work area detected based on the second video image, and the area information D2. The area information D2 has already been generated by the first analysis process, i.e., the learning process, and is stored in the storage unit 115. The work estimation unit 116 then outputs the work history information indicating the estimated history of one or more tasks to the visualization analysis unit 117.

[0059] The above-described second analysis process is performed through the processing by the image acquisition unit 111, the position detection unit 112, and the work estimation unit 116.

[0060] When the visualization analysis unit 117 obtains work history information from the work estimation unit 116, it performs a third analysis process based on that work history information. In this third analysis process, the visualization analysis unit 117 may use frequency information D1, area information D2, etc., stored in the storage unit 115. By performing this third analysis process, the visualization analysis unit 117 generates image data, such as graphs, that are intended to improve the efficiency of the work performed by worker K, and outputs this image data to the output unit 118.

[0061] When the output unit 118 acquires the image data from the visualization and analysis unit 117, it converts the image data into an image signal and outputs it to the display unit 1, thereby displaying the image shown in the image data on the display unit 1.

[0062] [First Analysis Process] Figure 5 is a diagram illustrating the processing performed by the position detection unit 112.

[0063] The position detection unit 112 detects the position of each feature point 11 in the work area of ​​worker K shown in each of the multiple frames fa included in the video image acquired by the image acquisition unit 111. For example, as shown in Figures 5(a) and (b), the position detection unit 112 detects the positions of each of the multiple joints included in the work area of ​​worker K from the frame fa showing the work area of ​​worker K, as the positions of feature points 11. For example, the work area is worker K's arm and hand. The position detection unit 112 then generates a frame fb for each of the multiple frames fa that shows the positions of the detected multiple feature points 11. This generates position time-series information consisting of multiple frames fb. Note that the position time-series information does not need to include frames fb, as long as it shows the position of the feature point 11 at each time point.

[0064] Furthermore, the imaging area of ​​the moving image acquired by the image acquisition unit 111 may be narrower than the imaging area of ​​the moving image shown in Figure 5(a), as shown in Figure 5(c). In other words, as shown in Figure 5(c), the subject displayed in frame fa included in the moving image may be enlarged compared to the subject displayed in frame fa shown in Figure 5(a). Even in this case, as shown in Figures 5(c) and (d), the position detection unit 112 detects the positions of each of the multiple joints included in the work area of ​​worker K from frame fa in which the work area of ​​worker K is displayed, as the positions of feature points 11. In this case, the work area is worker K's hand. In other words, the positions of the wrist joint and the finger joints are detected as the positions of feature points 11.

[0065] Figure 6 is a diagram illustrating an example of the first analysis process.

[0066] First, the image acquisition unit 111 acquires a video as the first video. Then, as shown in Figure 6(a), the position detection unit 112, similar to the example shown in Figure 5, generates position time-series information consisting of multiple frames fb by detecting multiple feature points 11 for each frame fa included in the first video. Each of the multiple frames fb indicates the positions of multiple feature points 11 corresponding to the time of that frame fb. In other words, the position detection unit 112 detects the movement of the worker K's work area and generates position time-series information showing the detected movement of the work area.

[0067] Next, as shown in Figure 6(b), the frequency information generation unit 113 generates frequency information D1, which is configured as a heat map, based on the position time series information. Specifically, the frequency information generation unit 113 identifies the stagnation among the detected work area movements shown in the position time series information as predetermined movements. Stagnation of a work area may be, for example, stagnation of a hand, where the hand movement maintains a speed of less than or equal to a second threshold for a time greater than or equal to a first threshold. In this case, the first threshold may be, for example, 3 seconds, 4 seconds, or 10 seconds, and the second threshold may be 0, 1 cm / second, or 2 cm / second. The frequency information generation unit 113 then generates the frequency information D1 shown in Figure 6(b) by associating the frequency at which stagnation occurred among the detected work area movements with the position in the imaging area where the stagnation occurred. Such frequency information D1 indicates that stagnation occurs frequently at positions that are displayed darkly (or where red is strongly displayed) and infrequently at positions that are displayed lightly (or where blue is strongly displayed).

[0068] The region information generation unit 114 then generates region information D2, for example, as shown in Figure 6(c), based on the frequency information D1. Specifically, the region information generation unit 114 identifies a region within the imaging region that includes a position where the frequency of satisfying a predetermined condition is indicated in the frequency information D1, as a work region. For example, the region information generation unit 114 identifies work regions A, B, and C. The region information generation unit 114 then generates region information D2 indicating the identified work regions A, B, and C. For example, the frequency that satisfies the predetermined condition is a frequency above a predetermined threshold. This predetermined threshold may be a value h (h>1) times a statistical value such as the average frequency shown in the frequency information D1. In other words, the region information generation unit 114 identifies high-frequency regions as work regions (specifically, work regions A, B, and C). Note that the shapes of work regions A, B, and C are rectangles in the example in Figure 6, but are not limited to rectangles; they may be circular or any other shape. Furthermore, the region information D2 may also indicate the order of work areas A, B, and C. For example, if the position time-series information indicates that a work part tends to enter work area A, then work area B, and then work area C, the region information generation unit 114 will include the order of work area A, work area B, and work area C in the region information D2.

[0069] Thus, in this embodiment, region information D2 is generated that indicates one or more work areas within the imaging area. Therefore, it is easy to determine from the region information D2 that the work corresponding to each of these work areas was performed by worker K. Consequently, the work can be appropriately analyzed.

[0070] Furthermore, in this embodiment, frequency information D1 is generated, which indicates the frequency of stagnation in the work area, associated with the location where the stagnation occurred. Then, region information D2 is generated, which indicates one or more regions within the imaging area that frequently experience stagnation as work areas. As a result, work can be appropriately associated with each of these one or more work areas, and it can be easily determined from the region information D2 that these works were performed by worker K. In addition, region information D2 can be generated that appropriately indicates one or more regions that frequently exhibit predetermined movements such as stagnation as work areas.

[0071] Furthermore, in this embodiment, since the frequency information D1 is configured as a heat map, it is possible to visually and easily grasp from the frequency information D1 the locations where the frequency is high or low in the imaging area.

[0072] [Second Analysis Process] Figure 7 is a diagram illustrating an example of processing performed by the work estimation unit 116 included in the second analysis process.

[0073] As shown in Figure 7, the work estimation unit 116 acquires position time-series information, including multiple frames fb, generated based on the second video image, from the position detection unit 112, and further acquires region information D2 from the storage unit 115. This position time-series information is generated based on the second video image obtained by imaging during the estimation period, and indicates the movement of the work area during the estimation period. The work estimation unit 116 then generates region history information D3 based on this position time-series information and the region information D2.

[0074] For example, the work estimation unit 116 identifies the period during which the work area of ​​worker K (i.e., feature point 11), indicated by the position time-series information, is within each work area of ​​the area information D2 as the area entry period, and generates area history information D3 that shows these identified area entry periods.

[0075] Specifically, as shown in Figure 7, if the work area was in work area A during the period from time t1 to t2, the work estimation unit 116 identifies the period from time t1 to t2 as the period of entry into work area A. Similarly, if the work area was in work area B during the period from time t3 to t4, the work estimation unit 116 identifies the period from time t3 to t4 as the period of entry into work area B. The work estimation unit 116 then generates area history information D3 indicating these identified area entry periods.

[0076] Figure 8 is a diagram illustrating an example of other processing performed by the work estimation unit 116 included in the second analysis process.

[0077] As shown in Figure 8(a), the work estimation unit 116 generates area history information D3, and then, based on that area history information D3, generates work history information D4 as shown in Figure 8(b). For example, for each of the multiple area entry periods shown in the area history information D3, the work estimation unit 116 estimates the period including that area entry period and the period immediately preceding that area entry period as the work period. The period immediately preceding the area entry period is a period that is not an area entry period, and will hereafter be called the preceding period.

[0078] Specifically, the work estimation unit 116 estimates the period including the entry period into work area A [time t1~t2] and the period immediately preceding that entry period [time t0~t1] as the work period [time t0~t2] in which work Wa corresponding to work area A was performed. In this disclosure, the period [time ta~tb] is the period from time ta to time tb (a and b immediately following t are arbitrary integers). Similarly, the work estimation unit 116 estimates the period including the entry period into work area B [time t3~t4] and the period immediately preceding that entry period [time t2~t3] as the work period [time t2~t4] in which work Wb corresponding to work area B was performed. Similarly, the work estimation unit 116 estimates the period including the entry period into work area C [time t5~t6] and the period immediately preceding that entry period [time t4~t5] as the work period [time t4~t6] in which work Wc corresponding to work area C was performed. By performing such estimation, the work estimation unit 116 generates work history information D4 that shows the estimated multiple work periods. The work history information D4 shows that worker K performed work Wa during the period t0~t2, work Wb during the period t2~t4, and work Wc during the period t4~t6.

[0079] Here, each work period includes the domain entry period and the preceding period. In other words, the work period [time t0~t2] in which work Wa is performed includes the domain entry period [time t1~t2] and the preceding period [time t0~t1]. The domain entry period [time t1~t2] is the period during which work Wa was actually performed and can be called the net work period. On the other hand, the preceding period [time t0~t1] is the period during which preparations, waiting, etc., are made for work Wa and can be called the preparation period. Similarly, the work periods of other tasks besides work Wa also include the net work period and the preparation period.

[0080] In this embodiment, the work estimation unit 116 estimates the history of one or more tasks by estimating the period of time a work part was present in a work area during the estimation target period for each of the one or more work areas shown in the area information D2, as the net work period during which the work corresponding to that work area was performed. The work estimation unit 116 generates work history information D4 that shows the history of one or more tasks estimated in this way.

[0081] Furthermore, if, within the estimated period, there is a period immediately preceding the net work period in which no work parts exist in any of the work areas indicated in the area information D2, the work estimation unit 116 estimates that period as a preparation period for the work corresponding to that net work period.

[0082] Thus, in this embodiment, since the history of one or more tasks during the estimated period is estimated based on the region information D2, it is possible to appropriately estimate when each task corresponding to a task indicated in the region information D2 was performed. Furthermore, it is possible to appropriately estimate the net work period during which the task corresponding to the work area was performed. Moreover, it is possible to appropriately estimate not only the net work period but also the preparation period for the task corresponding to that net work period.

[0083] For example, improving productivity and solving problems in assembly line work requires continuously understanding what tasks workers are performing and when. However, traditional methods such as stopwatch and VTR (Video Tape Recorder) analysis require significant effort to understand the tasks and cannot continuously monitor them. Furthermore, machine learning analysis requires manual teaching, such as labeling, which places a heavy burden on understanding the tasks.

[0084] However, in this embodiment, one or more work areas are automatically set without user specification, and the estimated target period is automatically divided into multiple work periods based on those work areas. Therefore, work history information D4 can be generated appropriately and easily without imposing a heavy burden while keeping man-hours down.

[0085] Here, as shown in Figure 8(a), the area history information D3 may indicate that the area entry period for work area A [time t7~t8] and the area entry period for work area C [time t9~t10] exist with a preparation period [time t8~t9] in between. In this case, the work estimation unit 116 may estimate the work period during which work Wd, corresponding to work areas A and C, was performed by worker K, as shown in Figure 8(c). That is, the work estimation unit 116 estimates the period including the area entry period [time t7~t8], the preceding preparation period [time t6~t7], the area entry period [time t9~t10], and the preceding preparation period [time t8~t9] as the work period during which work Wd was performed [time t6~t10]. Note that, like work Wd, the work corresponding to multiple work areas and those multiple work areas may be predetermined.

[0086] Furthermore, the work estimation unit 116 may estimate the duration of a cycle performed by worker K, as shown in Figure 8(d). For example, one cycle consists of tasks Wa, Wb, Wc, and Wd, and worker K repeatedly performs this cycle. In this case, the work estimation unit 116 estimates the duration of each cycle, such as the first cycle, second cycle, and third cycle, based on the periodicity of tasks Wa, Wb, Wc, and Wd, and the first task in the cycle (e.g., task Wa). The work estimation unit 116 then generates work history information D4 indicating the duration of those cycles.

[0087] [Third Analysis Process] Figure 9 is a diagram illustrating an example of processing performed by the visualization analysis unit 117, which is included in the third analysis process.

[0088] The visualization analysis unit 117 obtains work history information D4 for each worker K from the work estimation unit 116 and performs an analysis on that work history information D4. For example, as shown in Figure 9, the visualization analysis unit 117 generates a graph showing the cycle time for each worker K1, K2, ..., and Kn. Cycle time is the time required for one cycle. The visualization analysis unit 117 then outputs image data showing the graph to the output unit 118. As a result, the graph is displayed on the display unit 1. By looking at the graph, the user can check the cycle time for each worker K and grasp the scheduling loss. Scheduling loss is, for example, the difference between the maximum cycle time and the minimum cycle time shown in the graph, or the variation in cycle time. Scheduling loss may also be the difference between the maximum or minimum cycle time and the standard time, which is the average of the cycle times. Furthermore, by looking at the graph, the user can identify the bottleneck process among the multiple processes included in the line work, i.e., the process that requires the longest cycle time.

[0089] Figure 10 is a diagram illustrating another example of processing performed by the visualization analysis unit 117, which is included in the third analysis process.

[0090] The visualization and analysis unit 117 performs anomaly detection based on the history of multiple tasks shown in the work history information D4. Anomaly detection includes, for example, detecting missing tasks, duplicate tasks, and task substitutions.

[0091] In task omission detection, the visualization analysis unit 117 detects task omissions by determining whether or not a task is missing from a cycle. For example, as shown in Figure 10(a1), one cycle includes tasks Wa, Wb, Wc, and Wd. On the other hand, as shown in Figure 10(a2), if the work history information D4 shows a cycle containing tasks Wa, Wb, and Wd as a history, and the history of task Wc is not shown for that cycle, the visualization analysis unit 117 determines that task Wc is missing from the cycle. In other words, the visualization analysis unit 117 detects the absence of task Wc. As a result, the visualization analysis unit 117 outputs image data indicating the absence of task Wc to the output unit 118. This allows, for example, a message indicating the absence of task Wc to be displayed on the display unit 1. Alternatively, the visualization analysis unit 117 may identify each task that is considered correct to be included in a cycle, as shown in Figure 10(a1), based on one or more work areas shown in the area information D2.

[0092] In task duplication detection, the visualization analysis unit 117 detects task duplication by determining whether or not tasks are duplicated within a cycle. For example, as shown in Figure 10(b1), one cycle includes tasks Wa, Wb, Wc, and Wd. On the other hand, as shown in Figure 10(b2), the visualization analysis unit 117 determines that task Wc is duplicated if the task history information D4 shows a cycle containing two tasks Wc as part of the history. In other words, the visualization analysis unit 117 detects the duplication of task Wc. As a result, the visualization analysis unit 117 outputs image data indicating the duplication of task Wc to the output unit 118. This allows, for example, a message indicating the duplication of task Wc to be displayed on the display unit 1.

[0093] In task reorder detection, the visualization analysis unit 117 detects task reorders by determining whether the order of each task within a cycle is correct. For example, as shown in Figure 10(c1), in one cycle, four tasks are performed in the order of Wa, Wb, Wc, and Wd. On the other hand, as shown in Figure 10(c2), if the work history information D4 shows a cycle in which four tasks are performed in the order of Wa, Wb, Wd, and Wc, the visualization analysis unit 117 determines that the order of tasks Wc and Wd is incorrect. In other words, the visualization analysis unit 117 detects that the order of tasks Wc and Wd has been reordered. As a result, the visualization analysis unit 117 outputs image data indicating the reorder of tasks Wc and Wd to the output unit 118. This causes a message indicating the reorder of tasks Wc and Wd to be displayed on the display unit 1, for example.

[0094] The visualization analysis unit 117 may also identify the correct order of multiple tasks included in a single cycle based on the order of each work area shown in the area information D2. In other words, the visualization analysis unit 117 compares the order of each task shown in the work history information D4 with the order of each work area shown in the area information D2 (i.e., the order of tasks corresponding to each work area). The visualization analysis unit 117 then detects a change in the order of tasks if the orders do not match, and does not detect a change in the order of tasks if they do match.

[0095] Figure 11 illustrates another example of processing performed by the visualization analysis unit 117, which is included in the third analysis process.

[0096] The visualization analysis unit 117 may identify the variability in the time taken for each of the multiple tasks (hereinafter referred to as "work time") based on the work history information D4. Note that the work time may be the sum of the preparation period and the net work period corresponding to the task, or it may be the time of one of those periods. For example, as shown in Figure 11, the visualization analysis unit 117 identifies the variability in the work time for each of the tasks Wa, Wb, Wc, and Wd, and generates a graph showing the variability in those work times. The variability in work time may be represented by a box plot. The visualization analysis unit 117 then outputs image data showing the graph to the output unit 118. As a result, the graph is displayed on the display unit 1. By viewing the graph, the user can easily grasp the variability in the work time for each of the multiple tasks and take measures to reduce that variability.

[0097] In the example described above, the visualization analysis unit 117 identifies the variability of the work time for each of the multiple tasks, but it may also identify the variability of the cycle time for each of the multiple workers. Furthermore, the visualization analysis unit 117 may represent the variability using the maximum, minimum, variance, median, etc., of the work time or cycle time.

[0098] Figure 12 illustrates yet another example of processing performed by the visualization analysis unit 117, which is included in the third analysis process.

[0099] The visualization and analysis unit 117 may output image data showing the work history information D4 to the output unit 118. As a result, the display unit 1 displays the work history information D4. By looking at the work history information D4, the user can, for example, understand a long preparation period and work to shorten that preparation period. Specifically, the visualization and analysis unit 117 outputs image data showing the work history information D4 generated by the work estimation unit 116 based on the area history information D3 shown in Figure 12(a), i.e., the work history information D4 shown in Figure 12(b). The work history information D4 shows the preparation period [time t0~t1] and net work period [time t1~t2] of work Wa, the preparation period [time t2~t3] and net work period [time t3~t4] of work Wb, and the preparation period [time t4~t5] and net work period [time t5~t6] of work Wc. Furthermore, the work history information D4 shows the two preparation periods and the two net work periods of work Wd. The two preparation periods are preparation period [time t6~t7] and preparation period [time t8~t9]. The two net work periods are net work period [time t7~t8] and net work period [time t9~t10].

[0100] Figure 13 illustrates yet another example of processing performed by the visualization analysis unit 117, which is included in the third analysis process.

[0101] The visualization and analysis unit 117 may acquire frequency information D1 of skilled worker K from the storage unit 115 and output the frequency information D1 as image data to the output unit 118, as shown in Figures 13(a1) and (b1). The frequency information D1 shown in Figure 13(b1) is a heatmap generated based on the first video image of the skilled worker shown in Figure 13(a1). The visualization and analysis unit 117 may also acquire frequency information D1 of novice worker K from the storage unit 115 and output the frequency information D1 as image data to the output unit 118, as shown in Figures 13(a2) and (b2). The frequency information D1 shown in Figure 13(b2) is a heatmap generated based on the first video image of the novice worker shown in Figure 13(a2).

[0102] Alternatively, the visualization and analysis unit 117 may output image data containing the frequency information D1 shown in (b1) and (b2) of Figure 13 to the output unit 118, thereby displaying an image on the display unit 1 in which the two frequency information D1s are arranged in a comparative manner.

[0103] Users who view the image can easily grasp the difference between beginners and experts in the frequency of dwell time at each position within the imaging area. For example, the frequency of dwell time at a position indicated by beginner frequency information D1 is higher than the frequency of dwell time at the same position indicated by expert frequency information D1. In such a case, the user can easily understand that beginners are having more trouble with the task at that position than experts. That position may also be the position where material m is placed. In this case, the user can easily understand that beginners are having more trouble than experts with the task of grasping material m. In other words, it is possible to identify tasks in which differences appear between beginners and experts. Based on these differences, beginners can then be trained.

[0104] Furthermore, the visualization analysis unit 117 may identify the shortest path for the work area in one cycle and output image data showing that shortest path to the output unit 118, thereby displaying that shortest path on the display unit 1. In other words, the shortest path is proposed. For example, the visualization analysis unit 117 identifies multiple movement paths shown in the position time-series information and the movement time of the work area in each of those multiple movement paths for each of the multiple work areas, and selects the shortest movement path. Then, the visualization analysis unit 117 identifies the aforementioned shortest path by connecting the shortest movement paths for each of the multiple work areas.

[0105] [Other examples] Figure 14 is a diagram illustrating another example of frequency information D1.

[0106] Frequency information D1, as described above, is information that shows the frequency of a predetermined movement occurring among the detected movements of the work area, and associates it with the location in the imaging area where that predetermined movement occurred. In the example above, the predetermined movement is stagnation, but it may be other movements. For example, the predetermined movement may be the movement of a work area moving with an acceleration within a predetermined range, or the movement of a work area moving with a velocity within a predetermined range. The predetermined acceleration range may be, for example, an acceleration above a threshold (i.e., a large acceleration or high acceleration). Similarly, the predetermined velocity range may be, for example, a velocity above a threshold (i.e., a fast velocity or high velocity).

[0107] If a predetermined movement is a stagnation, the frequency information D1 indicates that the frequency of stagnation is high at the positions where the workpiece w and material m are located, as shown in Figure 14. This is because stagnation often occurs on the workpiece w when the work area, such as the worker K's hands, works on the workpiece w, or when the work area picks up the material m, stagnation often occurs on the material m.

[0108] If a predetermined movement is a high-acceleration movement, frequency information D1 indicates that the frequency of high-acceleration movements is high at the position where the mounting table 5 and material m are located. This is because high-acceleration movements often occur at the position of the mounting table 5 when worker K's hands or other working parts move from the workbench 4 side to the mounting table 5 side to pick up a workpiece w before it is in progress and return to the workbench 4 side, or when worker K picks up a completed workpiece w from the workbench 4 side to the mounting table 5 and returns to the workbench 4 side. Similarly, high-acceleration movements often occur at the position of material m when worker K's hands or other working parts move from the workpiece w side to the material m side to pick up material m and return to the workbench 4 side.

[0109] If the predetermined movement is a high-speed movement, the frequency information D1 indicates that the frequency of high-speed movements is high, for example, between the workpiece w and the mounting platform 5, and between the workpiece w and the material m. This is because high-speed movements often occur when the working parts of the operator K, such as their hands, are transporting the workpiece w and the material m.

[0110] The frequency information generation unit 113 may generate frequency information D1 based on motion at a predetermined acceleration range, motion at a predetermined velocity range, and dwell time. The region information generation unit 114 may then search for frequencies above a threshold from the frequencies at each position shown in the frequency information D1, and identify the region containing the positions where frequencies above the threshold are shown as a work region. Here, the region information generation unit 114 may identify the region where the region where frequencies above the threshold are shown in the frequency information D1 based on dwell time and the region where frequencies above the threshold are shown in the frequency information D1 based on motion at a predetermined acceleration range overlap as a work region.

[0111] Furthermore, when the area information generation unit 114 generates area information D2 from frequency information D1 based on movement within a predetermined range of speeds, the work area shown in the area information D2 can also be said to be the work movement path area. The visualization analysis unit 117 may obtain location time-series information for the estimation period from the location detection unit 112 via the work estimation unit 116, and then derive the movement time and movement distance of the work parts in the work movement path area based on that location time-series information and the area information D2 described above. If the movement time and movement distance are both outside a predetermined range, the visualization analysis unit 117 may determine that there is an abnormality or waste in the work, and may display a message indicating the determination result on the display unit 1 via the output unit 118.

[0112] Furthermore, the work estimation unit 116 may generate frequency information D1 as a heatmap based on movement at a predetermined range of speeds, based on the position time-series information. In addition, the work estimation unit 116 may estimate from the frequency information D1 areas showing a frequency above a threshold as work movement areas and generate area information D2 indicating those work movement areas.

[0113] Furthermore, when the visualization analysis unit 117 obtains location time-series information for the estimated period from the location detection unit 112 via the work estimation unit 116, it may derive the variation in velocity and acceleration of the work parts in each work area shown in the area information D2 based on that location time-series information. Alternatively, the visualization analysis unit 117 may derive the variation in velocity and acceleration of the work parts around each work area shown in the area information D2. If the variation is large, that is, if the magnitude of the variation is greater than or equal to a threshold, there is a high possibility that worker K is confused while working. Therefore, in such cases, the visualization analysis unit 117 may determine that the work is abnormal and display a message indicating the determination result on the display unit 1 via the output unit 118.

[0114] In this embodiment, the frequency information generation unit 113 identifies movements of the detected work area at a predetermined range of speeds as predetermined movements. Alternatively, the frequency information generation unit 113 identifies movements of the detected work area at a predetermined range of accelerations as predetermined movements. This generates frequency information D1, which indicates the frequency of movements of the work area at a predetermined range of speed or accelerations, associated with the location where the movement occurred. Then, region information D2 is generated, which indicates one or more regions within the imaging area where movements at a predetermined range of speed or accelerations are frequent as work areas. As a result, work can be appropriately associated with each of these one or more work areas, and it can be easily determined from the region information D2 that those work was performed by worker K.

[0115] Figure 15 is a diagram illustrating other processing performed by the frequency information generation unit 113.

[0116] The frequency information generation unit 113 may generate waveform information shown in Figure 15 based on the position time-series information and store it in the storage unit 115. The waveform information shows the time change of the position of the work area as a waveform (hereinafter also called the work waveform). The waveform information may be configured as a graph as shown in Figure 15. The horizontal axis of the graph shows time, and the vertical axis shows position. The position may be a position in one direction, such as the front-to-back direction or the left-to-right direction.

[0117] In the second analysis process, the work estimation unit 116 may acquire location time-series information for the estimation period, generate waveform information based on that location time-series information, and compare that waveform information with the waveform information stored in the storage unit 115. Based on the comparison result, the work estimation unit 116 may then estimate the work being performed by worker K.

[0118] [flowchart] Figure 16 is a flowchart showing an example of the processing operation of the work analysis device 100.

[0119] The work analysis device 100 first performs a first analysis process to generate area information D2 (step S10), then performs a second analysis process to estimate the work of worker K using the area information D2 (step S20). Then, the work analysis device 100 performs a third analysis process to analyze and visualize the work estimated by the second analysis process (step S30).

[0120] Figure 17 is a flowchart showing an example of the first analytical process performed by the work analysis device 100.

[0121] The image acquisition unit 111 of the work analysis device 100 acquires a first video image from each camera 3 (step S11). The first video image is a video showing the work area of ​​a worker performing a cycle repeatedly. Next, the following processing is performed on each of the acquired first video images. Specifically, the position detection unit 112 detects the movement of the work area based on the first video image and generates position time-series information indicating the detection result (step S12).

[0122] Then, the frequency information generation unit 113 generates frequency information D1 based on the location time series information (step S13). The region information generation unit 114 generates region information D2 based on the frequency information D1 (step S14), and stores the region information D2 in the storage unit 115 (step S15).

[0123] Figure 18 is a flowchart showing an example of the second analytical process performed by the work analysis device 100.

[0124] The image acquisition unit 111 of the work analysis device 100 acquires a second video image from each camera 3 (step S21). The second video image is a video showing the work area of ​​the worker performing the work during the estimated target period. Next, the following processing is performed on each of the acquired second video images. Specifically, the position detection unit 112 detects the movement of the work area based on the second video image and generates position time-series information indicating the detection result (step S22).

[0125] Then, the work estimation unit 116 estimates the history of one or more tasks performed by worker K during the estimation period based on the movement of the work area indicated by the position time-series information and the area information D2 stored in the storage unit 115 (step S23). Furthermore, the work estimation unit 116 generates work history information D4 showing the estimation result and outputs it to the visualization analysis unit 117 (step S24).

[0126] Figure 19 is a flowchart showing an example of the third analytical process performed by the work analysis device 100.

[0127] The visualization analysis unit 117 of the work analysis device 100 acquires work history information D4 from the work estimation unit 116 (step S31). The visualization analysis unit 117 then analyzes the organization loss and displays a graph, for example, as shown in Figure 9, on the display unit 1 via the output unit 118 (step S32). The visualization analysis unit 117 also performs anomaly detection, for example, as shown in Figure 10, and displays a message indicating the result of the anomaly detection on the display unit 1 via the output unit 118 (step S33).

[0128] Furthermore, the visualization analysis unit 117 analyzes the variability of each work time, as shown in Figure 11, for example, and displays a graph showing the analysis results on the display unit 1 via the output unit 118 (step S34). The visualization analysis unit 117 also performs an analysis to identify travel time and travel distance, as described above (step S35), and further performs an analysis to identify the shortest path (step S36). In addition, the visualization analysis unit 117 performs a human resource development analysis for training beginners by comparing the frequency information D1 of beginners and experts, as shown in Figure 13 (step S37). Note that the order of processing in steps S31 to S37 is just an example, and the processing may be performed in other order. Furthermore, the visualization analysis unit 117 does not need to perform all of the processing in steps S31 to S37, and may omit some of the processing. Furthermore, the visualization analysis unit 117 may perform other processing other than the processing in steps S31 to S37.

[0129] The work analysis system 10, work analysis device 100, and work analysis method using the work analysis device 100 described above have been explained based on the embodiments described above, but this disclosure is not limited to these embodiments. Various modifications to the above embodiments that can be conceived by a person skilled in the art may also be included in this disclosure, as long as they do not depart from the spirit of this disclosure.

[0130] For example, in the above embodiment, the position detection unit 112 detected joints as feature points, but other body parts may be detected as feature points. Alternatively, the position detection unit 112 may detect the movement of a work area by tracking the color of that work area.

[0131] Furthermore, in the above embodiment, the region information generation unit 114 identifies a region including a position in the frequency information D1 that has a frequency above a threshold as a work region, but it may also identify a region including a position in which a frequency below a threshold has a frequency as a work region. Alternatively, the region information generation unit 114 may identify a region including a position in which a frequency within a predetermined range has a frequency as a work region.

[0132] Furthermore, in the above embodiment, the work history information D4 shows multiple work periods, but it may also show only one work period.

[0133] In the above embodiment, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that implements the work analysis device 100, etc. in the above embodiment is a computer program that causes a computer to execute each step of the flowchart shown in Figures 16 to 19.

[0134] The following cases are also included in this disclosure.

[0135] (1) The at least one device described above is specifically a computer system consisting of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The at least one device described above achieves its function by the operation of the microprocessor in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate instructions to the computer in order to achieve a predetermined function.

[0136] (2) Some or all of the components constituting at least one of the above-described devices may be made up of a single system LSI (Large Scale Integration). The system LSI is a multi-functional LSI manufactured by integrating multiple components onto a single chip, and specifically, it is a computer system comprising a microprocessor, ROM, RAM, etc. The RAM stores a computer program. The system LSI achieves its function by operating the microprocessor in accordance with the computer program.

[0137] (3) Some or all of the components constituting at least one of the above-described devices may consist of an IC card or a standalone module that is detachable from the device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-described multi-function LSI. The IC card or module achieves its function by the operation of the microprocessor in accordance with a computer program. The IC card or module may be tamper-resistant.

[0138] (4) The disclosure may also be the methods described above. Alternatively, it may be a computer program that implements these methods using a computer, or a digital signal consisting of a computer program.

[0139] Furthermore, this disclosure may also refer to a computer program or digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD (Compact Disc)-ROM, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, it may refer to a digital signal recorded on such a recording medium.

[0140] Furthermore, this disclosure may also include the transmission of computer programs or digital signals via telecommunications lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc.

[0141] Alternatively, the program or digital signal may be carried out by another independent computer system by recording and transferring it on a recording medium, or by transferring the program or digital signal via a network or the like. [Industrial applicability]

[0142] The work analysis device described herein can be applied, for example, to a device or system for analyzing the work of an worker. [Explanation of Symbols]

[0143] 1 Display 2 Input section 3 cameras 4 Workbench 5. Mounting platform 6 pillars 10. Work Analysis System 11 Key Features 100 work analysis device 101 Control Unit 111 Image acquisition unit 112 Position detection unit 113 Frequency Information Generation Unit 114 Area information generation unit 115 Storage Unit 116 Work Estimation Department 117 Visualization Analysis Department 118 Output section A,B,C work area D1 Frequency Information D2 area information D3 Area History Information D4 Work History Information fa, fb frame K, K1, K2, Kn-1, Kn worker m material w work

Claims

1. An image acquisition unit acquires a first moving image in which a work area, which is at least a part of the worker performing the work, is shown in the imaging area. A position detection unit detects the time change in the position of the work area shown in the first moving image as the movement of the work area, A frequency information generation unit generates frequency information that indicates the frequency at which a predetermined movement occurs among the detected movements of the work area, in relation to the position in the imaging area where the predetermined movement occurred. A region information generation unit identifies one or more regions within the imaging region that include a position in the frequency information that satisfies a predetermined condition as a work region, and generates region information indicating one or more of the identified work regions. A work analysis device equipped with the following features.

2. The frequency information generation unit, Among the detected movements of the work area, the stagnation is identified as the predetermined movement. The work analysis apparatus according to claim 1.

3. The frequency at which the aforementioned predetermined conditions are met is greater than or equal to a predetermined threshold. The work analysis apparatus according to claim 1.

4. The frequency information is configured as a heatmap. The work analysis apparatus according to claim 1.

5. The image acquisition unit further, In the imaging area, a second video image is acquired showing the work area of ​​the worker performing the work during the estimated target period. The position detection unit further, The time change in the position of the work area shown in the second video is detected as the movement of the work area. The aforementioned work analysis device further, The system includes a work estimation unit that estimates the history of one or more tasks performed by the worker during the estimated period, based on the movement of the work area detected based on the second video image and the area information. The work analysis apparatus according to claim 1.

6. The aforementioned work estimation unit, For each of the one or more work areas indicated in the area information, the history of the one or more work is estimated by estimating the period during which the work part was present in that work area within the estimated target period as the net work period during which the work corresponding to that work area was performed. The work analysis apparatus according to claim 5.

7. The aforementioned work estimation unit, If, within the estimated period, there is a period immediately preceding the net work period in which the work area shown in the area information does not contain the work part, then that period is estimated as a preparation period for the work corresponding to the net work period. The work analysis apparatus according to claim 6.

8. The frequency information generation unit, Among the detected movements of the work area, movements within a predetermined range of speeds are identified as the predetermined movements. The work analysis apparatus according to claim 1.

9. The frequency information generation unit, Among the detected movements of the work area, movements with a predetermined range of acceleration are identified as the predetermined movements. The work analysis apparatus according to claim 1.

10. A computer-based method for analyzing work, In the imaging area, a first video image is acquired that shows a work area, which is at least a part of the worker performing the work. The time change in the position of the work area shown in the first video is detected as the movement of the work area. Frequency information is generated that shows the frequency at which a predetermined movement occurs among the detected movements of the work area, in relation to the position in the imaging area where the predetermined movement occurred. From the imaging area, one or more areas including a position that satisfies a predetermined condition in the frequency information are identified as work areas, and area information indicating one or more identified work areas is generated. Work analysis method.

11. In the imaging area, a first video image is acquired that shows a work area, which is at least a part of the worker performing the work. The time change in the position of the work area shown in the first video is detected as the movement of the work area. Frequency information is generated that shows the frequency at which a predetermined movement occurs among the detected movements of the work area, in relation to the position in the imaging area where the predetermined movement occurred. From the imaging area, one or more areas including a position that satisfies a predetermined condition in the frequency information are identified as work areas, and area information indicating one or more identified work areas is generated. A program that causes a computer to perform a task.

Images