Job classification device, job classification method, and program

Abstract

The work classification device includes: a storage unit that stores image data obtained by capturing an image of a work area; and an arithmetic circuit that classifies the action of the operator on the basis of the image data. The classification includes a first category, a second category different from the first category, a first sub-category included in the first category, and a second sub-category different from the first sub-category in the first category. The arithmetic circuit switches, on the basis of information within a work area in an image represented by image data, a first type of granularity when assigning an action of a worker to the classification.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a work classification device and / or a work classification method and program that utilize an image recognition algorithm. BACKGROUND

[0002] Patent Literature 1 discloses a device for analyzing work performed by a worker on an object. The work analysis device of Patent Literature 1 determines a position of a hand of the worker and a position of the object, calculates a distance between the hand of the worker and the object, and determines a content of a motion performed by the worker based on the calculated distance.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: JP Patent No. 7010542 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] The present disclosure provides a work classification device, a work classification method, and a program that can effectively classify a motion of a worker in correspondence with a photographing condition of image data.

[0008] MEANS FOR SOLVING THE PROBLEMS

[0009] A work classification device according to one embodiment of the present disclosure includes a storage unit that stores image data obtained by photographing a work area, and an arithmetic circuit that classifies a motion of a worker based on the image data, the classification including a first category, a second category different from the first category, a first subcategory included in the first category, and a second subcategory different from the first subcategory in the first category, the arithmetic circuit switching a granularity of the first category to which the motion of the worker is assigned when classifying the motion of the worker based on information in the work area in an image represented by the image data.

[0010] A work classification method according to one embodiment of the present disclosure includes a step in which an arithmetic circuit acquires image data obtained by photographing a work area, and a step in which the arithmetic circuit classifies a motion of a worker based on the image data, the classification including a first category, a second category different from the first category, a first subcategory included in the first category, and a second subcategory different from the first subcategory in the first category, the work classification method further including a step in which the arithmetic circuit switches a granularity of the first category to which the motion of the worker is assigned when classifying the motion of the worker based on information in the work area in an image represented by the image data.

[0011] A program according to one embodiment of the present disclosure is a program for causing an arithmetic circuit to execute the above-described work classification method.

[0012] Effects of Invention

[0013] According to the present disclosure, the worker's action can be effectively classified in correspondence with the imaging situation of the image data. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a diagram showing an outline of a work classification system according to Embodiment 1 of the present disclosure.

[0015] Figure 2 is a diagram illustrating an image represented by image data generated by a camera of Figure 1 .

[0016] Figure 3 is a block diagram showing a configuration example of a work classification apparatus of Figure 1 .

[0017] Figure 4 is a chart showing an example of a classification result of an action content of the work classification apparatus according to Embodiment 1.

[0018] Figure 5 is a flowchart illustrating an action of the work classification apparatus of Figure 1 .

[0019] Figure 6 is a detailed flowchart illustrating the work classification processing shown in Figure 5 .

[0020] Figure 7 is a table showing an example of a classification result DB of Figure 3 .

[0021] Figure 8 is a diagram showing an example of a display image showing a classification result of the work classification processing shown in Figure 5 .

[0022] Figure 9 is a diagram illustrating an image represented by image data generated by a camera in Embodiment 2.

[0023] Figure 10 is a flowchart illustrating the work classification processing according to Embodiment 2.

[0024] Figure 11 is a table showing a modification example of the classification result DB. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments will be described with reference to the accompanying drawings. Sometimes, necessary detailed descriptions are omitted. For example, detailed descriptions of well-known matters and repetitive descriptions of substantially the same structures are sometimes omitted. This is to avoid unnecessarily redundancy in the following description and to facilitate understanding by those skilled in the art.

[0026] Furthermore, the inventors have provided the accompanying drawings and the following description in order to enable those skilled in the art to fully understand this disclosure, but it is not intended to limit the subject matter described in the claims.

[0027] 1. Implementation Method 1

[0028] 1-1. Overview

[0029] Figure 1 This is a schematic diagram showing an outline of the job classification system 1 according to Embodiment 1 of this disclosure.

[0030] The job classification system 1 includes a camera 2 and a job classification device 10. The job classification system 1 is used in workplaces such as factories 6 to classify the actions of workers performing assembly line operations and other similar tasks. The job classification system 1 also includes a display device 4 for displaying the classification results of the workers' job content to users 3, such as managers or analysis supervisors of the workplace 6.

[0031] Camera 2 is configured to capture images of workers performing tasks in work area 6. Camera 2, for example, captures images of work area 6 at given intervals, generating image data representing the captured images. Additionally, in... Figure 1 The example shows one camera 2, but the number of cameras 2 included in the job classification system 1 is not limited to one; it can also be two or more. For example, camera 2 can capture dynamic images of the work site 6 and generate dynamic image data representing the captured dynamic images.

[0032] Figure 2 This is a schematic diagram illustrating image 20, represented by image data generated by camera 2. Workplace 6 is captured in image 20. In workplace 6, eight workers, 21-28, are performing their work. Figure 2 The image shows three work areas 31 to 33. For example, work areas 31 to 33 are predetermined as a given area within image 20. The position, size, etc., of work areas 31 to 33 can be arbitrarily set by the user 3. Alternatively, areas corresponding to work areas 31 to 33 on the image can be set in the actual work site 6.

[0033] Workers 21 and 22 are responsible for work area 31. The responsibility area is pre-defined as the area where the workers perform their work. At least one responsibility area is assigned to each person in charge. Workers 23 and 24 are responsible for work area 32. Workers 25-28 are responsible for work area 33. Alternatively, each work area may include multiple workers. Figure 2 The example shown is different; it establishes a one-to-one correspondence between work areas and workers.

[0034] 1-2. Structure of the work sorting device

[0035] Figure 3 This is a block diagram showing a structural example of the job sorting device 10. The job sorting device 10 includes a control unit 11, a storage unit 12, an input interface (I / F) 13, and an output interface 14.

[0036] The control unit 11 may include, for example, a processor and / or arithmetic circuitry that work with the software to achieve a given function, controlling the overall operation of the job sorting device 10. The control unit 11 reads the data and programs stored in the storage unit 12 and performs various arithmetic operations to achieve various functions. For example, the control unit 11 operates as both a detection unit 111 and a determination unit 112.

[0037] The control unit 11 can be a dedicated electronic circuit or a reconfigurable electronic circuit designed to perform a given function. The control unit 11 can also replace the arithmetic circuit or serve as the arithmetic circuit and include various semiconductor integrated circuits such as CPU, MPU, GPU, GPGPU, TPU, microcontroller, DSP, FPGA, and ASIC. Furthermore, the job classification method of this embodiment can also be executed via distributed computing.

[0038] The control unit 11 is equipped with a work detection model 113 that can detect objects and workers' work through image recognition processing.

[0039] The job detection model 113 is, for example, a learned model that has undergone learning based on neural networks such as convolutional neural networks. The job detection model 113 performs image recognition processing on the image represented by image data. For example, the job detection model 113 outputs the area in the image where a pre-defined object, such as a worker's hand, is captured as a detection result. In this embodiment, the detection object of the job detection model 113 is set to the worker's hand. The area output as a detection result is defined, for example, by the horizontal and vertical positions on the image, representing the area that encloses the detection object in a rectangular shape.

[0040] In this embodiment, when the job detection model 113 cannot identify a region of the target object in the image (i.e., cannot detect the worker's hand), it outputs a null value as the detection result, for example. The detection result may also include information indicating the time when the image was captured. The job detection model 113 can be obtained, for example, through supervised learning using teaching data that associates images of the worker's hand with labels representing correct solutions.

[0041] The completed learning model of the job detection model 113 is not limited to a neural network; it can also be other machine learning models related to image recognition. Furthermore, the job detection model 113 may not employ a model generated through machine learning, but rather an image recognition algorithm. For example, the job detection model 113 can also be configured to perform job detection through image recognition processing using a rule base.

[0042] Storage unit 12 is a recording medium that records various information, including programs and data required to realize the functions of job sorting device 10. Storage unit 12 may be implemented individually using semiconductor storage units such as flash memory, solid-state drives (SSDs), magnetic storage units such as hard disk drives (HDDs), or other recording media, or a combination thereof. Storage unit 12 is not limited to a built-in storage unit housed in the same casing as control unit 11; for example, it may be a portable type, a NAS (network-attached storage) type, or other storage unit. Storage unit 12 may also include volatile memory such as RAM.

[0043] The storage unit 12 stores image data 121 received from the camera 2, a classification result database (DB) 122 containing the classification results of the job classification device 10, and worker information 123. The worker information 123 is, for example, a database that associates identification information of multiple workers with the work area (responsibility area) where each worker is scheduled to perform a job.

[0044] Input interface 13 is an example of an input unit that connects the job sorting device 10 and the camera 2 to input information such as image data from the camera 2 into the job sorting device 10. Input interface 13 can be a communication circuit that performs data communication in accordance with existing wired or wireless communication standards.

[0045] Output interface 14 is an example of an output unit that connects the job classification device 10 to external devices such as the display device 4, providing control signals, image signals, and job classification results from the control unit 11. Output interface 14 can be a communication circuit that performs data communication according to existing wired or wireless communication standards. Output interface 14 can have the same structure as input interface 13.

[0046] Input interface 13 and output interface 14 can be as follows Figure 3 That could be implemented as a different interface, but it is not limited to that. For example, input interface 13 and output interface 14 can be constructed as a single unit.

[0047] 1-3. Actions

[0048] 1-3-1. Summary

[0049] exist Figure 2 In the image 20 shown, workers 21 and 22 in work area 31 and workers 23 and 24 in work area 32 are being filmed in positions and orientations where their hands are easily captured. Thus, the location of a worker is determined by the relationship between the position and orientation of the camera 2 and the worker, determining whether the worker's area is an area where their hands are easily captured. In this context, work areas 31 and 32 are areas where the worker's hands are easily captured. In contrast, work area 33 is an area where the worker's hands are difficult to capture.

[0050] A typical work breakdown technique (hereinafter referred to as the "typical technique") can determine the content of the actions of workers in work areas where their hands are easily visible with relatively high precision. Alternatively, in the typical technique, the time allotted for determining the content of the actions during the recording time is relatively long for workers in work areas where their hands are easily visible. However, in the typical technique, since the content of the actions performed by the worker cannot be determined unless the position of their hands is determined, the content of the actions performed by the worker in areas where their hands are difficult to capture cannot be fully determined. Therefore, in particular, the typical technique cannot evaluate what kind of work workers in areas where their hands are difficult to capture are performing, or the efficiency of the work, etc.

[0051] In order to conduct the aforementioned evaluation, one could consider configuring workers or cameras to easily capture the hands of all workers. However, this is not easily achieved due to constraints such as the construction of the work site, the configuration of workers, the conditions of the camera installation location, and the number of cameras that can be prepared.

[0052] To this end, the inventors conducted intensive research and arrived at the concept of classifying the granularity of the operator's actions according to the camera's recording conditions in the image data, thus leading to this disclosure. Here, the granularity of classification refers to the degree of detail with which the objects being classified are divided. The granularity of classification can be the depth of the hierarchical levels of the assigned categories, or it can be categories with different levels of abstraction.

[0053] For example, if an object such as a worker's hand is captured in an image represented by image data, the job classification device 10 of this embodiment classifies the worker's actions in detail based on the detection results of the object.

[0054] In contrast, even when no object is captured in the image, the inventors did not abandon the idea of ​​classifying the worker's actions, aiming for the highest possible granularity. Therefore, when no object is captured in the image, the job classification device 10 of this embodiment reduces the granularity of classification compared to when the object is captured, allowing for the classification of the worker's actions to a limited extent. Thus, compared to techniques that do not classify the worker's actions when no object is captured in the image, the job classification device 10 of this embodiment can obtain more classification information from the same amount of data. Consequently, the job classification device 10 of this embodiment reduces the amount of data required to obtain the same amount of classification information, and reduces communication traffic associated with memory, processing, and data transfer.

[0055] Figure 4 This is a diagram showing an example of the classification results of the operation content of the operation classification device 10 according to this embodiment. Figure 4 The bar chart is a chart that visualizes the classification results of the action content of the task classification device 10. The task classification device 10 determines whether the operator is located in the responsible area (in) or not located in the responsible area (out).

[0056] In this specification, the operator's actions include not only the actions performed by the operator in the area of ​​responsibility, but also the situation where the operator is present in the area of ​​responsibility and the situation where the operator is not present in the area of ​​responsibility (not in the area of ​​responsibility).

[0057] In an example where the worker's hand is captured and detectable in an image, the job classification device 10 detects whether the worker's ongoing task is a value-added task or a non-value-added task. Value-added tasks are predetermined and considered as objects of classification by the job classification device 10. Value-added tasks are an example of the target tasks of this disclosure.

[0058] Non-value-added work refers to the actions of the operator that are not related to value-added work. The operator's actions include both actions and omissions. For example, the definition of non-value-added work includes situations where the operator voluntarily performs actions other than value-added work, and situations where the operator stops performing such actions.

[0059] By performing the above processing, when objects such as the worker's hand are captured in the image, the job classification device 10 can generate... Figure 4 The task classification results are shown in the bar chart on the left.

[0060] On the other hand, even when the object is not captured in the image, the task classification device 10 can still generate [the necessary data]. Figure 4The task classification results are shown in the bar chart on the right. Thus, when the object is not captured in the image, the task classification device 10 reduces the granularity of the classification compared to when the object is captured in the image. Even when the object is not captured in the image, the task classification device 10 does not fail to classify the worker's actions, but rather reduces the granularity, yet still classifies the worker's actions within its limits. Here, the sub-category of reducing the granularity of "in" corresponds to value-added tasks or non-value-added tasks. "In" is a higher-level category than value-added or non-value-added tasks; it is a higher-level concept.

[0061] In images taken during a given period, such as a specific day (e.g., 24 hours), the daily volume of job classification results associated with a specific worker typically becomes... Figure 4 The left-hand bar chart shows the classification results for situations like slapping hands. Figure 4 The results of the classification of cases where the hand was not photographed, as shown in the bar chart on the right, are mixed.

[0062] According to the job classification device 10 of this embodiment, even if the worker designates an area where it is difficult to take a picture of their hand as their responsibility area, the device can at least determine the time the worker is in that area. In the case of periods when a picture can be taken, the job classification device 10 can further determine the time spent on value-added work and the time spent on non-value-added work during those periods.

[0063] By analyzing the job classification results or observing the results displayed on the display device 4, user 3 can obtain information about the content (actions) of the workers' work with the highest possible granularity, corresponding to the camera conditions. User 3 can then evaluate the work content of each worker based on this insight. By providing such evaluations back to the workers, the labor system, etc., the efficiency of work performed in the workplace 6 can be improved.

[0064] Furthermore, by re-examining the configuration of objects and personnel in workplace 6 based on the aforementioned insights, user 3 can improve the efficiency of operations performed in workplace 6.

[0065] 1-3-2. Overall Movement

[0066] Figure 5 This is a flowchart illustrating the operation of the job sorting device 10. The processes shown in this flowchart are, for example, executed by the control unit 11 of the job sorting device 10.

[0067] The control unit 11 obtains operator information 123 (S1). The control unit 11 can obtain operator information 123 from the outside via input interface 13, or it can obtain operator information 123 that has been pre-stored in storage unit 12.

[0068] The control unit 11 acquires image data from the camera 2 via the input interface 13 (S2). The control unit 11 stores the acquired image data 121 in the storage unit 12. It can also be connected to... Figure 5 Unlike the example, step S2 is performed before step S1.

[0069] Control unit 11 selects one operator as the target of inspection (S3). For example, control unit 11 selects one operator as the target of inspection from multiple operators in operator information 123. Control unit 11 can select an operator who should be located within the responsible area of ​​the target of inspection, such as a factory, at the time the image is acquired. In this case, operator information 123 may include information indicating the time when each operator should be located within the responsible area of ​​the target of inspection.

[0070] Next, the control unit 11 assigns identification information (S4) to the operator selected in step S3 to identify the operator. As identification information, the identification information associated with each operator in the operator information 123 can be used.

[0071] Control unit 11 performs job classification processing (S5). Details of job classification processing S5 will be described later.

[0072] Next, the control unit 11 determines whether there are other operators who are the targets of detection (S6). If there are other operators who are the targets of detection (S6 "Yes"), the control unit 11 performs steps S3 to S5 on one of the other operators. In this case, in step S3, one of the other operators is selected as the operator who is the target of detection.

[0073] In the absence of other operators being monitored (S6 "No"), the control unit 11 calculates the working time of each operator based on the results of the job classification process S5 (S7).

[0074] Next, the control unit 11 displays at least one of the results of the job classification processing S5 and the calculation results of step S7 on the display device 4 via the output interface 14 (S8).

[0075] By observing the job classification results displayed on display device 4, user 3 can understand the actions of the workers. For example, user 3 can evaluate the work content of each worker based on this insight. By feeding back such evaluations to the workers themselves, the labor system, etc., the efficiency of work performed in the workplace 6 can be improved.

[0076] 1-3-3. Homework Classification and Processing

[0077] Figure 6 This is an example Figure 5 The flowchart shown is a detailed process for job classification and processing S5.

[0078] Control unit 11 detects in the image Figure 5 Step S3 determines whether the selected operator is located within the operator's area of ​​responsibility (S11). The operator's area of ​​responsibility is predetermined as, for example, a given region within the image. Operator detection is performed, for example, using existing techniques for detecting people within an image.

[0079] If the control unit 11 detects that the operator is located in the area under its responsibility (S11 "Yes"), it determines that the operator exists in the area (existence determination) (S12).

[0080] If the control unit 11 does not detect that the operator is located within the area of ​​responsibility (S11 "No"), it determines that the operator is absent (absence determination) (S13). In this example, "operator absent" means that the operator is not located within the area of ​​responsibility.

[0081] In the next step of step S12, the control unit 11 determines whether the operator's hand has been detected (S14). For example, the control unit 11 determines in the image whether a hand has been detected in the operator's area of ​​responsibility.

[0082] In this embodiment, the worker's hand refers to the part that is further forward (farthest) than the worker's wrist. When the worker is wearing gloves, the worker's hand in this embodiment includes the gloves. That is, when the worker is wearing gloves, the control unit 11 can determine that the worker's hand has been detected when the worker's gloves are detected.

[0083] If the control unit 11 detects the operator's hand (S14 "Yes"), it detects whether the work being performed by the operator in the image is a value-added work (S15).

[0084] At least one of the processes in steps S14 and S15 is executed by, for example, the detection unit 111 of the control unit 11. The detection unit 111 executes at least one of the processes in steps S14 and S15, for example, through the operation detection model 113.

[0085] If the control unit 11 detects that the work being performed by the operator is a value-added work (S15 "Yes"), it determines that the operator is engaged in a value-added work (S16).

[0086] If the control unit 11 does not detect that the work being performed by the operator is a value-added work (S15 "No"), it determines that the operator is performing a non-value-added work (S17).

[0087] If the operator's hand is not detected in step S14 (S14 "No"), the control unit 11 determines that the operator is an unspecified operator (S18).

[0088] In this specification, the term "operator in unspecified work" means that the operator is at least present within the area of ​​responsibility. The situation of "operator in unspecified work" includes both situations where the operator is performing value-added work and situations where the operator is performing non-value-added work. The statement that the control unit 11 determines the operator to be in unspecified work means that the control unit 11 cannot detect the operator's hand (S14 "No"), and therefore cannot determine whether the operator is performing value-added work.

[0089] The steps S12, S13, S16 to S18 described above are executed, for example, by the determination unit 112 of the control unit 11.

[0090] The existence determination in step S12 is an example of classifying the operator's action into category 1. The non-existence determination in step S13 is an example of classifying the operator's action into category 2.

[0091] The value-added task determination in step S16 is an example of classifying the worker's actions into sub-category 1. The non-value-added task determination in step S17 is an example of classifying the worker's actions into sub-category 2. Sub-categories 1 and 2 are further subcategories of sub-category 1. In one example, sub-category 1 is a value-added task, and sub-category 2 is a non-value-added task.

[0092] After steps S13, S16 to S18 above, the control unit 11 records the determination result and time into the classification result DB122 (S19).

[0093] Figure 7 This is a table representing an example of the classification result DB122. In Figure 7 In the classification result DB122, the worker's identification information and the result of the task classification process S5, i.e., the action content, are linked with the time information. The time information corresponds to the time when the image, which is the object of the task classification process S5, is captured.

[0094] Figure 7 The example of the classification result DB122 indicates that worker A is not in a specific job at 10:00:01 and 10:00:02 on a specific day, and is not present at 10:00:03. Figure 7 The example of the classification result DB122 indicates that the operation B continued from 10:00:01 to 10:00:03 on the same day is an added value operation.

[0095] Control unit 11 via such Figure 7As shown, the operator's work status is recorded at a given period, and the operator's unspecified work time, value-added work time, non-value-added work time, and / or time when the operator is absent (absence time) can be totaled. In unspecified work, value-added work, and non-value-added work, since the operator is present in the area of ​​responsibility, in this specification, the unspecified work time, value-added work time, and non-value-added work time are sometimes collectively referred to as "presence time".

[0096] The presence time, absence time, value-added operation time, and non-value-added operation time are examples of the first to fourth time periods of this disclosure, respectively. Unspecified operation time is an example of the fifth time period of this disclosure.

[0097] use Figure 8 This is an example of how to display the classification results of the total number of workers' presence time, absence time, etc.

[0098] 1-4. Example Display

[0099] Figure 8 This is a schematic diagram showing an example of a display image 40 illustrating the classification results of the job classification process S5. Figure 8 The displayed image 40 Figure 5 Step S8 is shown on display device 4.

[0100] Image 40 shows a bar chart used to visualize the classification results of job classification processing S5. Figure 8 The bar chart displays the total of each worker's unspecified work time, absence time, value-added work time, and non-value-added work time on a specific day. These totals can be used... Figure 7 The classification result DB122 shown in the example is used to calculate the result.

[0101] exist Figure 8 In the example bar chart shown, the presence time (i.e., value-added work time, unspecified work time, and non-value-added work time) is accumulated upwards from the baseline 41. On the other hand, the absence time is represented by a bar chart extending downwards from the baseline 41. Thus, it becomes easier for user 3 to compare the absence time of each operator, and to compare the absence time and presence time of each operator.

[0102] Figure 8 The bar chart shown indicates that the areas of responsibility for operators A and B are, for example... Figure 2 Examples of areas 31 and 32, where it's easy to grab a hand. Therefore, in the bar charts for operators A and B, the proportion of unspecified work time relative to both value-added and non-value-added work time is relatively small.

[0103] on the other hand, Figure 8 The bar chart shown indicates that the areas of responsibility for operators C and D are, for example... Figure 2 Examples of areas like zone 33, which are difficult to photograph. Therefore, in the bar charts for operators C and D, the proportion of unspecified work time relative to value-added work time and non-value-added work time is greater than in the bar charts for operators A and B.

[0104] In addition, Figure 8 As shown in the bar chart, it can be seen that the absence time of operators A and B is the same, but the value-added operation time of operator A is longer than that of operator B.

[0105] exist Figure 8 From the bar chart shown, we can see that operator C's absence time is longer than others, and operator D's absence time is shorter than others. Furthermore, we can see that the proportion of operator C's absence time to the total of operator C's presence time (unspecified work time, value-added work time, and non-value-added work time) is also larger than others. Conversely, the proportion of operator C's absence time to the total of operator D's presence time is smaller than others.

[0106] User 3 can analyze the content of the operator's work as described above by observing the display image 40 shown on display device 4. User 3 can evaluate the work content of each operator based on the results of this analysis. By providing such evaluations to the operators, the labor system, etc., User 3 can improve the efficiency of work performed in workplace 6. Furthermore, by re-examining the equipment and personnel allocation systems in workplace 6 based on the results of the analysis described above, User 3 can improve the efficiency of work performed in workplace 6.

[0107] 1-5. Effects, etc.

[0108] As described above, the job classification device 10 according to the embodiment includes a storage unit 12 and a control unit 11, which is an example of an arithmetic circuit. The storage unit 12 stores image data obtained by photographing a job area, which is an example of a job area. The control unit 11 classifies the actions of the worker based on the image data. The classification includes a first category, a second category different from the first category, a first subcategory contained in the first category, and a second subcategory different from the first subcategory in the first category. The control unit 11 switches the granularity of assigning the worker's actions to the first category in the above classification based on information within the job area in the image 20 represented by the image data. For example, the control unit 11 switches the classification target of the worker's actions between the first and second categories and the first and second subcategories (S5).

[0109] According to this structure, the job classification device 10 can effectively classify the content of the worker's actions according to the camera conditions of the image data. The job classification device 10 also achieves at least the same effect in the following manner.

[0110] Control unit 11 can determine whether the operator is located within the work area in image 20 (S11). If control unit 11 determines that the operator is located within the work area in image 20 (S11 "Yes"), it classifies the action as Category 1 (S12). If control unit 11 determines that the operator is not located within the work area in image 20 (S11 "No"), it classifies the action as Category 2 (S13).

[0111] If the control unit 11 determines that the operator is located within the work area in image 20 (S11 "Yes"), it may also detect an example of a given object related to the work being performed by the operator, namely the operator's hand, in image 20 (S14). If the control unit 11 detects the operator's hand (S14 "Yes"), it determines whether the work is an example of a given object work, i.e., a value-added work (S15). If the control unit 11 determines that the work is a value-added work (S15 "Yes"), it classifies the action into a first sub-category (S16). If the control unit 11 determines that the work is not a value-added work (S15 "No"), it classifies the action into a second sub-category (S17).

[0112] Based on this structure, when the operator's hand is detected, the operator's actions can be further classified in detail.

[0113] The control unit 11 can also classify the action into a category that is different from either the second category, the first sub-category, or the second sub-category, even if the operator's hand is not detected (S14 "No"). According to this structure, the operator's actions can be classified with the highest possible granularity, even if the operator's hand is not detected.

[0114] The control unit 11 can also measure the first time when the operator performs an action classified as category 1 and the second time when the operator performs an action classified as category 2 in image 20 (S7). By measuring the time corresponding to each category, for example, user 3 can quantitatively grasp the content of the operator's actions.

[0115] The control unit 11 can also measure the third time when the operator performs the action of classifying into the first sub-category and the fourth time when the operator performs the action of classifying into the second sub-category in the image 20.

[0116] The job sorting device 10 may also include, for example, an output interface 14, which outputs information to the display device 4. The control unit 11 may also use the output interface 14 to display data representing the sorting results of the operation on the display device 4.

[0117] The control unit 11 can also cause the display device 4 to display information indicating the first and second times via the output interface 14. The control unit 11 can also cause the display device 4 to display information indicating the first to fourth times via the output interface 14.

[0118] User 3 can analyze the content of the workers' work by observing the data displayed on display device 4. User 3 can evaluate the work content of each worker based on the results of this analysis. By providing such evaluations to the workers themselves, the labor system, etc., User 3 can improve the efficiency of the work performed in workplace 6. Furthermore, by re-examining the equipment and personnel configuration systems in workplace 6 based on the results of the aforementioned analysis, User 3 can further improve the efficiency of the work performed in workplace 6.

[0119] 2. Implementation Method 2

[0120] In Embodiment 1, an example of the object being the worker's hand was given, but in Embodiment 2, an example of the object being a lamp was given.

[0121] Figure 9 This is a schematic diagram illustrating image 20a, represented by image data generated by camera 2 in embodiment 2. Image 20a captures a worker 21a performing work at a work site.

[0122] Three boxes 51 to 53 are provided at the work site. Components X, Y, and Z are respectively contained in boxes 51 to 53. In this embodiment, operator 21a takes out components X, Y, and Z from boxes 51 to 53 and transports the taken-out components to a given location for shipment.

[0123] exist Figure 9 Three work areas 34-36 are shown. For example, work areas 34-36 are predefined as a given region within image 20a. Figure 9 In the example, work areas 34-36 correspond to boxes 51-53 respectively. For example, the positional relationship between work area 34 and box 51 is configured such that if worker 21a stands in front of box 51, then in image 20a, worker 21a enters work area 34. The same applies to the positional relationship between work area 35 and box 52, and between work area 36 and box 53.

[0124] Lights 54 and 56 are respectively arranged in front of boxes 51-53 (between boxes 51-53 and camera 2). Lights 54-56 are configured to be lit before operator 21a performs work. If operator 21a removes part X from box 51, light 54 corresponding to box 51 will turn off. Similarly, if operator 21a removes part Y from box 52, light 55 will turn off, and if operator 21a removes part X from box 53, light 56 ​​will turn off.

[0125] In this embodiment, when the operator 21a is in front of the box with the corresponding light illuminated, the control unit 11 determines that the operator 21a is performing a value-added operation. The value-added operation envisioned in this embodiment is the operation in which the operator 21a takes a component out of the box with the corresponding light illuminated.

[0126] In this embodiment, when the operator 21a is located in front of the box where the corresponding light is off, the control unit 11 determines that the operator 21a is performing a non-value-added task. The non-value-added task envisioned in this embodiment is a task other than component removal, such as sorting.

[0127] In this embodiment, if the corresponding light is not captured in image 20a, the control unit 11 determines that the operator 21a is performing unspecified work. If the operator 21a is not located in front of the box, the control unit 11 determines that the operator 21a is absent.

[0128] Figure 10 This is a flowchart illustrating the job classification process S5a according to this embodiment. In this embodiment, the control unit 11 performs the job classification process S5a instead of that in Embodiment 1. Figure 10 The job classification and processing S5a.

[0129] If with Figure 6 Comparing the job classification processing S5 of Implementation Method 1 shown, then Figure 10 The job classification process S5a replaces step S14 and includes step S24, and replaces step S15 and includes step S25.

[0130] exist Figure 10 In the job classification processing S5a, the control unit 11 first detects whether the worker 21a is located in the work area in the image (S11). In this embodiment, the work areas 34 to 36 are the areas located in front of the boxes 51 to 53 (between the boxes 51 to 53 and the camera 2).

[0131] If the control unit 11 detects that the operator 21a is in the work area (S11 "Yes"), it determines that the operator exists in the work area (existence determination) (S12). If the control unit 11 does not detect that the operator is in the work area (S11 "No"), it determines that the operator is not present (absence determination) (S13).

[0132] In the next step of step S12, the control unit 11 determines whether a lamp is detected (S24). Figure 9 In the example, the control unit 11 determines whether any of the lamps 54 to 56 are detected.

[0133] When the control unit 11 detects a light (S24 "Yes"), it checks whether the light is on (S25). In this embodiment, checking whether the light is on is an example of detecting whether the work being performed by the worker in the image is a value-added task.

[0134] If the control unit 11 detects that the light is on (S25 "Yes"), it determines that the operator is engaged in value-added work (S16). If the control unit 11 does not detect that the light is on (S25 "No"), it determines that the operator is engaged in non-value-added work (S17).

[0135] If the control unit 11 does not detect the light in step S24 (S24 "No"), the control unit 11 determines that the operator is an unspecified operator (S18).

[0136] According to this embodiment, the job classification device 10 can also effectively classify the content of the operator's actions according to the camera conditions of the image data.

[0137] 3. Other implementation methods

[0138] As described above, embodiments have been illustrated as examples of the technology in this disclosure. However, the technology in this disclosure is not limited to this and can be adapted to embodiments with modifications, substitutions, additions, omissions, etc. Furthermore, new embodiments can be made by combining the constituent elements described in the above embodiments. Therefore, other embodiments are illustrated below.

[0139] In Embodiment 1, an example was described where the object is the worker's hand and the task involves hand movements. This disclosure is not limited to this; the object can also be the worker's foot. In this case, the task may also include foot movements. The object can be any part of the worker's body, or it can be a tool used by the worker in the task.

[0140] In Implementation Method 1 Figure 7The example illustrates the classification result DB122, which includes only unspecified work time, value-added work time, and non-value-added work time as the content of the operator's actions, but the classification result DB is not limited to this. Figure 11 This is a table that illustrates a variation of the classification result DB, example 122a.

[0141] exist Figure 11 In the classification result DB122a, the content of the operator's actions is classified as either "in" (Category 1) or "not in" (Category 2) as the primary classification. If the primary classification is "in" (Category 1), the content of the operator's actions is further classified as "in value-added work" (Subcategory 1), "in non-value-added work" (Subcategory 2), or "in unspecified work" as a subcategory (subcategory).

[0142] In Embodiment 1, an example is described of the control unit 11 performing step S3, which involves selecting one worker as the detection target from the image represented by image data 121. However, this disclosure is not limited to this. For example, a specific work area and a specific worker may be pre-established as a correspondence. In this case, step S3 may be omitted. For example, if the control unit 11 determines the work area, it can determine the worker corresponding to that work area. In this case, even if no worker corresponding to that work area is captured within the work area in the image represented by image data 121, the worker corresponding to that work area can still be determined.

[0143] When a specific work area and a specific worker are pre-assigned as described above, the control unit 11 can identify the worker corresponding to that work area once the work area is determined. Therefore, in this case, the process can be omitted. Figure 5 Step S4.

[0144] 4. Example of a method

[0145] The following illustrates the manner in which this disclosure is made.

[0146] <Method 1>

[0147] A job classification device includes: a storage unit that stores image data obtained by photographing a job area; and a processing circuit that classifies the actions of a worker based on the image data, the classification including a first category, a second category different from the first category, a first subcategory contained in the first category, and a second subcategory different from the first subcategory in the first category, the processing circuit switching the granularity of the first category when assigning the worker's actions to the above classification based on information within the job area in the image represented by the image data.

[0148] <Method 2>

[0149] In the job classification device described in Method 1, the arithmetic circuit performs the following processing: determining whether the worker is located in the work area in the image; if the worker is located in the work area in the image, classifying the action into a first category; if the worker is not located in the work area in the image, classifying the action into a second category.

[0150] <Method 3>

[0151] In the job classification device described in Method 2, the arithmetic circuit performs the following processing: when it is determined in the image that the worker is located in the job area, a given object related to the job performed by the worker is detected in the image; when the object is detected, it is determined whether the job is a given object job; when the job is determined to be the object job, the action is classified into the first sub-category; when the job is determined not to be the object job, the action is classified into the second sub-category.

[0152] <Method 4>

[0153] In the job classification device described in Method 3, the arithmetic circuit classifies the action into a category that is different from any of the second category, the first sub-category, and the second sub-category when the object is not detected.

[0154] <Method 5>

[0155] In any of the work classification devices described in methods 1 to 4, the computing circuit measures the first time when the worker performs the action of classifying the action into the first category in the image, and the second time when the worker performs the action of classifying the action into the second category in the image.

[0156] <Method 6>

[0157] In the job classification device described in Method 5, the computing circuit measures the third time when the worker performs the action of classifying the job into the first sub-category in the image, and the fourth time when the worker performs the action of classifying the job into the second sub-category in the image.

[0158] <Method 7>

[0159] The job classification device described in any of the methods 1 to 6 further includes: an output unit that outputs information to a display device, wherein the arithmetic circuit causes the display device to display data representing the classification result of the action via the output unit.

[0160] <Method 8>

[0161] The job classification device described in method 5 or 6 further includes an output unit that outputs information to a display device, wherein the arithmetic circuit causes the display device to display information indicating the first time and the second time via the output unit.

[0162] <Method 9>

[0163] The job classification device described in Method 6 further includes an output unit that outputs information to a display device, wherein the arithmetic circuit causes the display device to display information representing the first to fourth times via the output unit.

[0164] <Method 10>

[0165] In the job classification device described in mode 3 or 4, the object is the operator's hand, and the job involves the action of the hand.

[0166] <Method 11>

[0167] In the job classification device described in mode 3 or 4, the object is the worker's foot, and the object job includes the movement of the foot.

[0168] <Method 12>

[0169] In the job classification device described in method 3 or 4, the object is a light installed in the job area, and the computing circuit detects whether the light is lit based on the image data. If the light is detected to be lit, the job is determined to be the object job.

[0170] <Method 13>

[0171] A job classification method for classifying worker actions includes: a step of a computing circuit acquiring image data obtained by photographing a work area; and a step of the computing circuit classifying the worker's actions based on the image data, wherein the classification includes a first category, a second category different from the first category, a first subcategory contained in the first category, and a second subcategory different from the first subcategory in the first category. The job classification method further includes: a step of the computing circuit switching the granularity of the first category when assigning the worker's actions to the classification based on information within the work area in the image represented by the image data.

[0172] <Method 14>

[0173] A program for causing an arithmetic circuit to execute the job classification method described in mode 13.

[0174] Industrial availability

[0175] This disclosure relates to an apparatus that can be used to analyze the work of workers in workplaces such as factories.

[0176] Explanation of reference numerals in the attached figures

[0177] 1. Homework Classification System

[0178] 2 cameras

[0179] 3 users

[0180] 4 Display devices

[0181] 6. Workplace

[0182] 10. Job sorting device

[0183] 11 Control Department

[0184] 12 Storage Department

[0185] 13 Input Interface

[0186] 14 Output Interfaces

[0187] 20 images

[0188] Operators 21-28

[0189] Work areas 31-36

[0190] 40 Display Image

[0191] 41 Reference Axis

[0192] Boxes 51-53

[0193] 54-56 lamps

[0194] 111 Detection Department

[0195] 112 Judgment Department

[0196] 113 Operation Detection Model

[0197] 121 Image Data

[0198] 122 Classification Results Database

[0199] 123 Operator Information.

Claims

1. A job sorting device, comprising: The storage unit stores image data obtained by photographing the work area; and The processing circuit classifies the operator's actions based on the image data. The classification includes a first category, a second category different from the first category, a first subcategory contained in the first category, and a second subcategory in the first category different from the first subcategory. The computing circuit switches the granularity of assigning the operator's actions to the first category during classification based on information within the work area in the image represented by the image data.

2. The job sorting device according to claim 1, wherein, The arithmetic circuit performs the following processing: Determine whether the worker is located within the work area in the image. If the worker is determined to be located within the work area in the image, the action is classified as Category 1. If it is determined in the image that the worker is not located within the work area, the action is classified as Category 2.

3. The job sorting device according to claim 2, wherein, The arithmetic circuit performs the following processing: If it is determined in the image that the worker is located within the work area, then a given object related to the work being performed by the worker is detected in the image. If the object is detected, determine whether the task is a given object task. If the task is determined to be the target task, the action is classified into the first sub-category. If it is determined that the task is not the target task, the action is classified into the second sub-category.

4. The job sorting device according to claim 3, wherein, If the object is not detected, the computing circuit classifies the action into a category that is different from any of the second category, the first sub-category, and the second sub-category.

5. The job sorting device according to claim 1, wherein, The computing circuit measures the first time when the operator performs an action classified into the first category in the image, and the second time when the operator performs an action classified into the second category in the image.

6. The job sorting device according to claim 5, wherein, The computing circuit measures the third time when the operator performs the action of classifying the action into the first sub-category in the image, and the fourth time when the operator performs the action of classifying the action into the second sub-category in the image.

7. The job sorting device according to claim 1, wherein, The job sorting device also includes: The output section outputs information to the display device. The arithmetic circuit, via the output section, causes the display device to display data representing the classification results of the action.

8. The job sorting device according to claim 5, wherein, The job sorting device also includes: The output section outputs information to the display device. The arithmetic circuit, via the output section, causes the display device to display information indicating the first time and the second time.

9. The job sorting device according to claim 6, wherein, The job sorting device also includes: The output section outputs information to the display device. The arithmetic circuit causes the display device to display information representing the first to fourth times via the output section.

10. The job sorting device according to claim 3, wherein, The object being described is the worker's hand. The object task includes the hand movements.

11. The job sorting device according to claim 3, wherein, The object being referred to is the worker's feet. The object operation includes the movement of the foot.

12. The job sorting device according to claim 3, wherein, The object being referred to is a light installed in the work area. The arithmetic circuit performs the following processing: The system detects whether the light is on based on the image data. If the light is detected to be lit, the task is determined to be the target task.

13. A job classification method for classifying the actions of a worker, the job classification method comprising: The steps of the arithmetic circuit acquiring image data obtained by photographing the work area; and The step of the computing circuit classifying the operator's actions based on the image data. The classification includes a first category, a second category different from the first category, a first subcategory contained in the first category, and a second subcategory in the first category different from the first subcategory. The job classification method also includes: The computing circuit switches the step of assigning the operator's actions to the granularity of the first category when classifying them based on information within the work area in the image represented by the image data.

14. A program for causing an arithmetic circuit to perform the job classification method of claim 13.

Images