Work support system and work support method

Abstract

To provide a work support system that can identify variations in operation in repetitive works and support the work of an operator without reducing a work efficiently and a work accuracy.SOLUTION: A work support system 1000 includes: a section division unit 100 that outputs comparison work data in association with work sequence information using sensor data of a predetermined site of an operator and the work sequence information in which a process or a procedure of a work is defined; an identification area selection unit 200 that selects an identification area of the operator whose proficiency level is determined based on reference area information in which a position of a work target and the position of the operator are defined and imaging data, and associates the operator in the identification area and the comparison work data of the predetermined site of the operator with each other; and a variation calculation unit 300 that determines the proficiency level of the operator based on a difference of variation of repetitive works in the process or the procedure of the work using reference sensor data for determining the proficiency level of the operator and the work sequence information and presents the result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a task assistance system and a task assistance method. [Background technology]

[0002] Conventionally, there are various techniques for supporting the work of workers on work objects such as workpieces. For example, Patent Document 1 discloses a technique in which "the work time proficiency is calculated by comparing the actual work time, from which invalid work not caused by the worker is removed based on data obtained from an image capture device 11 and a work video analysis device 21, with the standard work time previously stored in a standard work time DB 22. The quality proficiency is calculated from the results of inputting the number of quality defects caused by the worker that occurred in the assembly and inspection processes during manufacturing in an assembly process quality defect input unit 31 and inputting the number of quality defects caused by the worker that occurred in the final inspection process in a final process quality defect input unit 34. Furthermore, the overall work proficiency is calculated from the sum of the work time proficiency and the quality proficiency."

[0003] Furthermore, for example, Patent Document 2 discloses a technology in which "the variance calculation unit 103 of the proficiency assessment device 100 divides three-dimensional posture data indicating the three-dimensional posture of the trainee when the trainee attempts a movement multiple times into movement phases based on changes in the position of a predetermined body part, normalizes the duration of the phase in the three-dimensional posture data of the multiple trials for each phase, and resamples the normalized three-dimensional posture data at predetermined intervals. The variance calculation unit 103 calculates a value quantitatively representing the variance of the movement among the multiple trials from one or more of the variance of the duration for each phase in the three-dimensional posture data of the multiple trials, the variance of the trajectory for each phase of the predetermined body part obtained from the resampled three-dimensional posture data, and the variance of the posture obtained from a posture matrix in which the three-dimensional posture data resampled for each phase are arranged. The proficiency calculation unit 104 calculates the proficiency based on the value representing the variance of the movement." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2004-086322 A [Patent Document 2] JP 2015-150226 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the above Patent Document 1, the proficiency level is judged from the working time and the number of defects, but there is no mention of judging the proficiency level of a worker from the variation in the motion of the part of the body that acts on the work object. Also, in the above Patent Document 2, the proficiency level is judged from the time series distribution of the three-dimensional data, but there is no disclosure of considering the variation in the motion when the worker repeats the motion of the part of the body that acts on the work object.

[0006] For example, consider a case where a worker repeatedly picks up a tool from a shelf or table used for work setup and manually screws a part onto a steel plate, which is the workpiece to be worked on. An expert worker repeatedly performs a series of actions, such as picking up the tool and screwing the part, at the same time and in the same position (e.g., the same tempo and the same hand position), whereas an unexperienced worker often exhibits temporal and spatial variations in the actions. Such variations in the actions during repetitive tasks lead to a decrease in work efficiency and accuracy. For this reason, it is important to understand the variations in the actions during repetitive tasks and support the worker's work so as not to lead to a decrease in work efficiency and accuracy.

[0007] The present invention aims to provide a work support system and a work support method that are capable of understanding variations in movements during repetitive work and supporting the work of a worker without causing a decrease in work efficiency or work accuracy. [Means for solving the problem]

[0008] In order to solve the above problems and achieve the object, the work support system of the present invention includes a section division unit that uses sensor data of a predetermined part of a worker obtained by analyzing image data of a worker performing work on a work object, including repetitive movements, and work sequence information that defines a process or procedure of the work, to output comparative work data in which the sensor data is associated with the process or procedure of the work, and a section division unit that selects an identification area of ​​a worker to be subjected to proficiency assessment based on reference area information that defines a work position, which is the position of the work object, or a work position where the worker performs work, and the image data, and selects an identification area of ​​the worker to be subjected to proficiency assessment based on the image data and reference area information that defines a work position, which is the position of the work object, or a work position where the worker performs work, and selects an identification area of ​​the worker to be subjected to proficiency assessment based on the image data and the selected identification area. an identification area selection unit which matches a worker in the selected identification area with the comparative work data of a specified part of the worker; reference work data which is determined using reference sensor data that is a standard for judging the worker's proficiency and the work sequence information, and which matches the reference sensor data with a work process or procedure; and a variance calculation unit which uses the comparative work data of a worker in the selected identification area and the work sequence information to judge the worker's proficiency based on a difference in variance in repetitive actions in the work process or procedure, and presents the result of the judgment. Effect of the Invention

[0009] According to the present invention, it is possible to grasp the variation in movements during repetitive work and to support the work of a worker so as not to cause a decrease in work efficiency or work accuracy. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a work support system. [Diagram 2] FIG. 1 is a diagram showing an outline of a computer. [Diagram 3] FIG. 2 is a diagram showing an example of imaging data including images and videos captured by a sensor. [Figure 4] FIG. 11 is a diagram illustrating an example of sensor data. [Diagram 5] FIG. 11 is a diagram showing an example of comparison task data. [Figure 6A] FIG. 11 is a diagram showing an example of work sequence information (work process). [Figure 6B] FIG. 11 is a diagram showing an example of work sequence information (work procedure). [Figure 7] FIG. 11 is a diagram showing an example of reference area information. [Figure 8] FIG. 11 is a diagram showing an example of specific reference area information. [Figure 9] FIG. 13 is a diagram showing an example of time comparison variation data. [Figure 10] FIG. 13 is a diagram showing an example of joint position comparison variation data. [Figure 11] FIG. 13 is a diagram showing an example of a skill determination result screen. [Figure 12] FIG. 13 is a diagram showing an example of a variation determination result details screen. [Figure 13] 13 is a flowchart showing a processing procedure of a skill level determination process. [Figure 14] FIG. 11 is a diagram illustrating a configuration example of a work support system (embodiment 2). [Figure 15] FIG. 11 is a diagram showing an example of comparison operation data (Example 2). [Figure 16] FIG. 11 is a diagram showing an example of work sequence information (Example 2). [Figure 17] FIG. 13 is a diagram showing an example of a skill assessment result screen (Example 2). [Figure 18] 11 is a flowchart showing a procedure of a skill level determination process (Example 2). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0012] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0013] In the following explanation, various information may be described using expressions such as "database," "table," and "list," but the various information may be expressed in other data structures. To indicate independence of data structure, "XX table," "XX list," and the like may be referred to as "XX information." When describing identification information, when expressions such as "identification information," "identifier," "name," "ID," and "number" are used, these are interchangeable.

[0014] When there are multiple components having the same or similar functions, they may be described by using the same reference numerals with different subscripts. However, when there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0015] In addition, in the following description, processing performed by executing a program may be described, but the program is executed by a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) to perform a defined process using storage resources (e.g., a memory) and / or an interface device (e.g., a communication port) as appropriate, so the subject of the processing may be the processor. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing a program may be a calculation unit, and may include a dedicated circuit that performs specific processing (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

[0016] A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing a program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs. EXAMPLES

[0017] Fig. 1 is a diagram showing an example of the configuration of a work support system 1000 in this embodiment. As shown in Fig. 1, the work support system 1000 uses work sequence information 601, which is data related to the work process and work procedure of the work performed by the worker, and includes a section division unit 100 that outputs sensor data of the part (for example, joint position) where the worker acts on the work object, which is included in the image data output from a sensor S such as a camera, as comparison work data 501, which is time-series work data divided into work processes or work procedures, and uses reference area information 701 that defines the work position, which is the position of the work object and the work position where the worker performs the work, for each work process and work procedure, to filter the part where the worker acts on the work object and identify the worker. The work support system 1000 has an identification area selection unit 200 which selects an identification area for the work support system 1000, a variation calculation unit 300 which calculates the variation in work time and the variation in parts acting on the work target in the repetitive motion of each work process or each work procedure based on divided reference work data 501A obtained by dividing reference work data, which is past time-series work data output from the sensor S, into work processes or work procedures and the comparison work data 501, and outputs the proficiency and improvement points of the work based on the variation to a display device D such as a display, for each work process or each work procedure, and a work sequence controller 400 which controls each of these units. The specific processing performed by each of these units in the work support system 1000 and the specific configuration of the data used will be described later.

[0018] The sensor S and the work support system 1000 are connected via a general communication network. The display device D and the work support system 1000 are connected via a general communication network or by various cables of a predetermined standard. In the following description, it is assumed that one work process is made up of one or more work procedures, but one work procedure may be made up of one or more work procedures.

[0019] The work support system 1000 shown in FIG. 1 can be realized, for example, by a general computer 1600 as shown in FIG. 2 (schematic diagram of a computer) that includes a CPU 1601, a memory 1602, an external storage device 1603 such as a HDD (Hard Disk Drive), a reading / writing device 1607 for reading and writing information from a portable storage medium 1608 such as a CD (Compact Disk) or a USB memory, an input device 1606 such as a keyboard or a mouse, an output device 1605 such as a display, a communication device 1604 such as a NIC (Network Interface Card) for connecting to a communication network, and an internal communication line (called a system bus) 1609 such as a system bus that connects these together.

[0020] Various data stored in the above system or used for processing can be realized by the CPU 1601 reading and using the data from the memory 1602 or the external storage device 1603. Also, each functional unit of the above system (e.g., the section division unit 100, the identification area selection unit 200, the variation calculation unit 300, and the work sequence controller 400) can be realized by the CPU 1601 loading a predetermined program stored in the external storage device 1603 into the memory 1602 and executing it.

[0021] The above-mentioned predetermined program may be stored (downloaded) in the external storage device 1603 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and then loaded onto the memory 1602 and executed by the CPU 1601. Alternatively, the program may be directly loaded onto the memory 1602 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and executed by the CPU 1601.

[0022] In the following, an example is given in which the work support system 1000 is configured by a single computer, but similar functions may be realized by distributing all or part of these functions across one or more computers, such as a cloud, and communicating with each other via a network.

[0023] FIG. 3 is a diagram showing an example of imaging data including images and videos captured by a sensor S. As shown in FIG. 3, the imaging data 301 records a state in which a worker works on a workpiece, which is a work object. For example, in FIG. 3, multiple workers W1, W2, and W3 are working within the imaging range of the sensor S. Among them, the worker W2 works on a workpiece WK, which is a work object placed on a work table T1. In addition, in the imaging data 301, a tool TO used for the work on the workpiece WK is placed on a setup table T2. The worker W2 selects a tool suitable for the work process from among the multiple tools TO and works on the work. The workers W1 and W3 in the imaging data 301 are workers who are not involved in the work on the workpiece WK. The section division unit 100 obtains sensor data on various parts of the worker acting on the work object included in the imaging data 301 by using various techniques such as image analysis that have been conventionally known from the imaging data 301.

[0024] Fig. 4 is a diagram showing an example of sensor data. The sensor data is data before comparison task data 501, which will be described later, is divided into task processes or task steps. As shown in Fig. 4, the sensor data 401 is output as time-series task data including sensor values ​​of various parts (e.g., hand joints, elbow joints) acting on a task object. In this embodiment, a joint part is exemplified as a part of a worker acting on a task object, but this is not limiting, and other parts such as an upper arm, a forearm, a thigh, or a lower leg may be used as a part of a worker acting on a task object.

[0025] 4 shows sensor data when, for example, the worker W2 shown in FIG. 3 moves the joints of the hand, elbow, and shoulder to perform work on the workpiece WK, which is the work target. In this case, hand sensor data 401a, elbow sensor data 401b, and shoulder sensor data 401c are plotted on a graph with the horizontal axis representing time t and the vertical axis representing the sensor value. In this way, the sensor data stores the sensor values ​​of the parts acting on the work target in chronological order. The sensor data is output for each worker included in the captured image data 301.

[0026] FIG. 5 is a diagram showing an example of the comparison work data. The comparison work data is data obtained after dividing the above-mentioned sensor data 401 into work processes or work procedures. As shown in FIG. 5, the comparison work data 501 is stored with the sensor data 401 and work sequence information 601 associated with each other. In FIG. 5, for example, the sensor data 401 divided into work processes P1 to P9 is stored as the comparison work data 501. In this way, the comparison work data 501 is stored with the sensor data 401 and the above-mentioned work sequence information 601 associated with each other.

[0027] 6A and 6B are diagrams showing an example of work sequence information 601. The work sequence information 601 includes work process sequence information in which a standard work time is defined for each work process, and work procedure sequence information in which a standard procedure time is defined for procedures included in each work process.

[0028] As shown in FIG. 6A, in the work process sequence information 601A, a standard work time and a work content are defined for each work process. The section division unit 100 can detect the transition of the sensor value in each work process corresponding to the standard work time by overlapping the work process sequence information 601A with the sensor data 401. In FIG. 6, for example, the standard work time of work process 0 is 18 minutes (the end time of the work finish 18 minutes after the start time of the work 0:00, which is 0:18), and the transition of the sensor value corresponding to the standard work process or standard operating procedure can be detected. In addition, as shown in FIG. 6A, the work content indicating what kind of work is to be performed is stored in association with the standard work time. For example, it is indicated that work a0 and b0 are performed in work process 0.

[0029] Also, as shown in FIG. 6B, in the work procedure sequence information 601B, a standard procedure time is defined for each work procedure for each work process. For example, the section division unit 100 can detect the transition of the sensor value in the standard work procedure for each work procedure constituting each work process by overlapping the work procedure sequence information 601B and the sensor data 401. In FIG. 6, for example, the procedures constituting the work process 0 are procedure A to procedure K, and the standard procedure time of procedure A is 2 minutes (the end time of the work end 2 minutes after 0:00 from the start time of the work start), and the transition of the sensor value corresponding to the standard procedure section can be detected. Also, as shown in FIG. 6B, the procedure contents indicating what kind of work is to be performed are stored in association with the standard work time. For example, it is indicated that procedures a0 and b0 are performed in the work procedure A.

[0030] As described later, the identification area selection unit 200 acquires the comparison task data 501 and the reference area information 701, and selects an identification area for identifying a worker. More specifically, the identification area selection unit 200 acquires the comparison task data 501 for each of the workers W1, W2, and W3 included in the captured image data 301, which was created by the section division unit 100. Furthermore, the identification area selection unit 200 compares the captured image data 301 with the reference area information 701, selects an overlapping range between the two as an identification area, and sets the comparison task data 501 for the worker included in the selected identification area (worker W2 in this example) as data to be used for proficiency assessment for the worker.

[0031] Furthermore, the identification area selection unit 200 acquires the divided reference task data 501A stored in advance as a reference and the reference area information 701, and selects an identification area for identifying the worker. More specifically, the identification area selection unit 200 acquires the reference task data 501A of the expert included in the past captured image data 301 created by the section division unit 100. Furthermore, the identification area selection unit 200 compares the past captured image data 301 with the reference area information 701, selects an overlapping range between the two as an identification area, and sets the reference task data 501A of the expert included in the selected identification area as data to be used for proficiency assessment for the expert.

[0032] The divided reference task data 501A is data in which reference task data obtained from the past comparative task data 501 of a predetermined expert is associated with a task process or a task procedure. The divided reference task data 501A has the same structure as the comparative task data 501 divided into task processes or task procedures, so a detailed description is omitted here.

[0033] In this embodiment, the divided reference task data 501A of the expert is used, but instead, the divided reference task data 501A of the worker himself may be used. In this case, when the past comparison task data 501 of the same worker is used as the divided reference task data 501A, it becomes possible to confirm, on the proficiency assessment result screen, how much the worker to be assessed for proficiency has grown compared to the past task data and whether the variation in the repetitive actions has disappeared. The user may specify whether to use the divided reference task data 501A of the expert or the divided reference task data 501A of the worker himself.

[0034] Here, we will explain repetitive motion. Repetitive motion refers to a motion that is repeated in time and / or space, such as a motion in which the same work procedure is performed multiple times in succession in a certain work process, or a motion in which the same work is performed multiple times in a certain work procedure. For example, consider a case in which a series of actions is repeated, such as picking up a tool from a shelf or table for setting up a work and screwing a part. In this case, the action of picking up a tool and screwing a part is performed every few seconds in the space between the position of the worker, the position of the tool, and the position of the part, so that the action can be said to be repetitive motion.

[0035] The work sequence information 601 shown in Figs. 6A and 6B includes the repetitive actions described above. For example, if a procedure is one in which a worker repeatedly picks up a tool from a shelf or table for setting up the work and manually screws up a part on a steel plate, which is a workpiece to be worked on, the procedure is performed multiple times in succession (for example, two procedures, Procedure E and Procedure F, shown in Fig. 6B). In such a procedure, a skilled worker repeatedly performs a series of actions, such as picking up a tool and screwing up a part, at the same time and in the same position (for example, at the same tempo and in the same hand position), so that the temporal and spatial variations in each action are often below a certain standard. On the other hand, a non-skilled worker performs unnecessary actions and the like, which causes temporal and spatial variations in the actions compared to a skilled worker. The greater the variation in the repetitive actions in the work, the more the work efficiency and work accuracy decrease.

[0036] In this embodiment, the system not only grasps the variation in the worker's actions in a certain work process or work procedure, but also the variation in such repetitive actions, and supports the worker's work so as not to cause a decrease in work efficiency and work accuracy. What actions are considered to be repetitive actions is determined in advance. For example, in FIG. 6B, work procedures E and F are the same procedure and are represented as being repeated twice. Although the following is an example of a work procedure, the same can be considered when a work process is repeated.

[0037] FIG. 7 is a diagram showing an example of the reference area information 701. As shown in FIG. 7, in the reference area information 701, the work position, which is the position of the work target, and the work position where the worker performs the work are predefined for each work process and work procedure. FIG. 7 shows, as an example, that the range where the worker performs the work in the work process 0 is the range (x, y)=(300-450, 150-400) on the coordinates defined in the imaging range of certain imaging data. The identification area selection unit 200 compares the imaging data 301 shown in FIG. 3 with the above-mentioned reference area information 701 defined for each of the work processes and work procedures, and identifies the worker included in the reference area information 701 as the worker to be the target of the proficiency assessment. The identification area selection unit 200 associates the part (e.g., joint position) corresponding to the sensor data acquired by the section division unit 100 with the worker W2 included in the reference area information 701 for the identified worker. In FIG. 7, each part of the worker W2 acting on the work object is associated with each part, including the top of the head, both hands, elbows, shoulders, waist, knees, and ankles.

[0038] In this way, the identification area selection unit 200 selects, as the identification area for performing work of each work process, the range in which the workers included in the imaging range of the imaging data 301 perform work of each work process defined as the above-mentioned reference area information 701. In Fig. 7, it can be seen that, of the workers W1, W2, and W3, the range on the coordinate system (x, y) = (300-450, 150-400) in which worker W2 is located performs work of work process 0, is selected as the identification area, and an image of that range is output.

[0039] Furthermore, the identification area selection unit 200 specifies, among the parts that act on the work target of the worker and are included in the identification area, the above-mentioned parts that have a predetermined positional relationship with the workpiece WK and the work tables T1, T2 (for example, a relationship within 1 meter, which is a certain distance from the worker) as the parts that directly act on the workpiece WK. This will be specifically described below.

[0040] Fig. 8 is a diagram showing an example of specific reference area information 801 after each of the parts of the body 702 shown in Fig. 7, namely the left and right hands, elbows, shoulders, and waist, which are parts that are at a certain distance from the workpiece WK or the worktable T1, is identified as a part that directly acts on the workpiece WK. Fig. 8 shows that, of the parts that act on the work target of the worker, the parts that are at least a certain distance from the workpiece WK and the worktables T1 and T2 are identified as a part that directly acts on the workpiece WK.

[0041] The above-mentioned parts may be identified by, for example, specifying the parts that directly act on the workpiece WK when the parts operated by the worker W2 when performing work change by a certain amount or more. For example, in Fig. 8, it is assumed that the worker W2 performs the movements M1 and M2 to perform the work of the work process 0. In this case, if the movements of the left and right hands, elbows, shoulders, and waist are larger than those of the top of the head, knees, ankles, etc., and the amount of movement is equal to or greater than a certain threshold, the parts can be identified as the parts that directly act on the workpiece WK.

[0042] 7 and 8, the workers who are included in the image capture data 301 and the identification area have been described, but the case of an expert can be considered in the same way. That is, as described above, the identification area selection unit 200 identifies an expert who is included in the past image capture data 301 and the identification area as a part that directly acts on the workpiece WK.

[0043] The reference area information 701 shown in Fig. 7 and the specific reference area information 801 shown in Fig. 8 are stored in association with the comparison task data 501 and the divided reference task data 501A shown in Fig. 5. Such association and storage are performed by the identification area selection unit 200. In this way, the identification area selection unit 200 reads out the reference area information 701 and selects an identification area where a worker will perform work for each work process from the image data, and identifies, for the worker performing work in the selected identification area, a part that directly acts on the work as a part to be subjected to proficiency judgment. For the skilled worker included in the reference task data 501A, the identification area selection unit 200 selects the above identification area and identifies the above part to be subjected to proficiency judgment in the same manner as for the worker.

[0044] In this way, the identification area selection unit 200 selects an identification area including a worker to be subjected to the variance calculation, and further specifies the part of the joint part of the worker included in the identification area that directly acts on the work object as the part to be subjected to the proficiency judgment. Therefore, the variance calculation unit 300 described later can reduce noise when calculating the variance using various conventionally known algorithms including DTW (Dynamic Time Warping). In other words, when comparing the whole body of a worker with the whole body of a skilled worker, the motion of the part that is not related to the work is included, so there is a lot of noise and the accuracy is reduced. Therefore, in this embodiment, the accuracy is improved by limiting the parts for the proficiency judgment as described above. That is, in the conventional image analysis, the whole body of the worker is imaged, so that the accuracy is reduced due to the occurrence of points that should always be focused on and points that should not be focused on, but this system can solve such a problem. For example, there are points to focus on in a work procedure manual, such as pre-setup actions, actions to retrieve parts, crimping actions, and post-setup actions, and by evaluating the deviations around those points, it is possible to prevent a deterioration in accuracy caused by incorporating variations in body parts of the worker (for example, head, waist, elbows, etc., that are unrelated to the work) into the proficiency evaluation.

[0045] As described later, the variance calculation unit 300 uses the comparison task data 501 for a worker included in the selected identification area (worker W2 in this example) and the divided reference task data 501A for an expert worker, together with the task sequence information 601, to compare the variance in the repetitive actions in each task process and each task procedure, thereby determining the proficiency of the worker.

[0046] Fig. 9 is a diagram showing an example of the variation in work time (time comparison variation data) calculated by the variation calculation unit 300. In Fig. 9, the description of the work process or work procedure shown in Fig. 5 is omitted.

[0047] In FIG. 9, the time comparison variation data 901 shows that there is a time variation in the positional fluctuation (in this case, the positional fluctuation of the shoulder) when performing work including the repetitive motion of the part 401c shown in FIG. 4. Specifically, in the comparative work data 901c (graph at the bottom of the drawing) of the worker's part 401c, the time required to perform the repetitive motion in the work from steps E, F to step G in a certain process is the work time TW1. On the other hand, in the comparative work data 901c' (graph at the top of the drawing) of the skilled worker's part 401c, the time required to perform the repetitive motion in the work from steps E, F to step G in a certain process is recorded as the work time TW2 in the divided reference work data 501A. Therefore, it is shown that there is a difference ΔT1 in the time variation due to the comparison of the work times between the two. This shows that, for example, the comparative work data 901c takes longer to perform the repetitive motion, and there is a possibility that each of the repetitive motions is wasteful. The variation calculation unit 300 scores the difference in the above-mentioned time variation by expanding or contracting the time comparison variation data in the time axis direction using various conventionally known algorithms such as DTW. In Fig. 9, the time comparison variation data is calculated for a certain work procedure, but the same can be considered for a work process.

[0048] In this way, the variance calculation unit 300 calculates the variance (variance in the time direction) of the work time of each action in the repetitive action performed by the worker in the comparative work data of the worker, and calculates the variance (variance in the time direction) of each action in the repetitive action performed by the expert in the divided reference work data of the expert. Then, the variance calculation unit 300 calculates the difference in the time variance between the two as a score. By calculating such a score and presenting it to the user on the screen as described below, the user can infer the reason for the difference in the time variance, such as the worker's insufficient preparation, trial and error due to an undefined order of the actions, or a small mistake in the middle of the action leading to a retry.

[0049] Fig. 10 is a diagram showing an example of the variation (joint position comparison variation data) of the parts acting on the work target calculated by the variation calculation unit 300. In Fig. 10, similar to the case described in Fig. 9, the description of the work process or work procedure is omitted.

[0050] In FIG. 10, the joint part ratio variation data 1001 shows that there is spatial variation in the positional fluctuation (in this case, the positional fluctuation of the shoulder) when performing work including the repetitive motion of the part 401c shown in FIG. 4. Specifically, the joint position fluctuation is compared in the comparative work data 1001c (the graph at the bottom of the drawing) of the part 401c and the comparative work data 1001c' (the graph at the top of the drawing) at a time width where the waveforms in a certain process match. In FIG. 10, a waveform TW3 of a certain time width in the comparative work data 1001c matches a waveform TW4 of a certain time width in the comparative work data 1001c'. The time width is, for example, the same time width as the time width of steps E and F, which are the repetitive motions shown in FIG. 6B, so it can be seen that steps E and F, which are the repetitive motions, are performed at TW3 and TW4. That is, in the joint part comparison variation data 1001, when the work positions of both are compared using the joint positions in the procedures E and F, which are repeated actions recorded with the same time width, it is shown that there is a position shift of the part 401c for a time width ΔT2 at a certain point in time, that is, a spatial variation. Therefore, in this example, when the repeated actions of procedures E and F are performed, it can be said that the repeating action of the worker W2 has a position shift of ΔT2 compared to that of the skilled worker. As in the case shown in FIG. 9, the variation calculation unit 300 scores the difference in the spatial variation of the joint position comparison variation data using various conventionally known algorithms including DTW. In FIG. 10, the joint position comparison variation data is calculated for a certain work procedure, but as in the case of FIG. 9, the same can be considered for the work process.

[0051] In this way, the variance calculation unit 300 calculates the variance (spatial variance) of the movement of the worker and the worker's body part in each of the repetitive movements performed by the worker in the comparative work data of the worker, and calculates the variance (spatial variance) of the movement of the worker and the worker's body part in each of the repetitive movements performed by the skilled worker in the divided reference work data of the skilled worker. Then, the variance calculation unit 300 calculates the difference in spatial variance between the two as a score. By calculating such a score and presenting it to the user on the screen as described later, it is possible to infer the reason for the difference in spatial variance, such as the worker's insufficient preparation, trial and error due to an undefined order of movements, or a small mistake during the movement that causes a manufacturing operation to be performed differently than usual.

[0052] The idea of ​​finding the difference in the spatial variation is, for example, to calculate the difference in the coordinate positions of the joint angles between waveforms matched by DTW, and calculate the coordinate deviation from the matching results. The advantage of calculating the coordinate deviation from the matching results in this way is that even if there are differences in the phase of the waveforms of both data, the start coordinates of the comparison range, or the work speed, if the work is the same, the time difference can be absorbed to generate the joint position deviation.

[0053] As described in FIGS. 9 and 10, the variance calculation unit 300 calculates time comparison variance data and joint position comparison variance data that indicate the variance when a task including a repetitive motion in a certain task process or procedure is performed, and displays the proficiency judgment result obtained by scoring the difference in variance between the worker and the expert on a display device such as a display. The variance calculation unit 300 refers to the task sequence information 601 shown in FIGS. 6A and 6B, and outputs the task process or procedure in which the repetitive motion is performed in association with the proficiency judgment result. As described below, by presenting the proficiency judgment result to the user, the user can select and indicate the points in the task process or procedure that should be focused on to improve the task by picking up the points from the task contents in the task procedure manual.

[0054] Fig. 11 is a diagram showing an example of a proficiency judgment result screen displayed on a display device such as a display by the variance calculation unit 300. The proficiency judgment result screen 1101 shown in Fig. 11 includes a time variation judgment result display area 1102 that displays the proficiency judgment result for the time comparison variance data 901 shown in Fig. 9, and a joint part variance judgment result display area 1103 that displays the proficiency judgment result for the joint part comparison variance data 1001 shown in Fig. 10.

[0055] More specifically, in the time variation judgment result display area 1102, the judgment result of the proficiency level based on the difference in the time variation for each of the work processes P1 to P9 performed by the worker is displayed. In this example, the greater the difference in the variation in the work time between the skilled worker and the worker (i.e., the more inexperienced), the darker the color, and the smaller the difference in the variation (i.e., the more skilled), the lighter the color. For example, in the work process P1, the difference in the variation in the work time between the skilled worker and the worker is smaller than the difference in the variation in the work time between the skilled worker and the worker in the work process P6, so it can be seen that the lighter color is displayed. Therefore, at a glance, the user can determine that the proficiency level of the worker in the work process P6 is low. Also, for example, in FIG. 11, since the judgment result of the proficiency level is displayed in chronological order, for example, if the later work processes are displayed darker, it can be inferred that the cause of the difference in the variation is an increase in the work delay caused by the worker's fatigue.

[0056] Similarly, in the joint part variation judgment result display area 1103, the judgment result of the proficiency level based on the difference in spatial variation is displayed for each of the work steps P1 to P9 of the work performed by the worker. In this example, the greater the difference in the variation in the joint position between the skilled worker and the worker (i.e., the less skilled the worker), the darker the color, and the smaller the difference in the variation (i.e., the more skilled the worker), the lighter the color. For example, in the work step P1, the difference in the variation in the joint position between the skilled worker and the worker is smaller than the difference in the variation in the joint position between the skilled worker and the worker in the work step P2, so it can be seen that the lighter color is displayed. At a glance, the user can determine that the proficiency level of the worker in the work step P2 is low. Also, for example, in FIG. 11, since the judgment result of the proficiency level is displayed in chronological order, for example, if the later work steps are displayed darker, it can be inferred that the cause of the difference in the variation is a decrease in the work accuracy caused by the worker's fatigue.

[0057] Furthermore, the user can check the relationship between the time variation judgment result display area 1102 and the joint part variation judgment result display area 1103 to perform various analyses. For example, in the work process P2 displayed in the time variation judgment result display area 1102, the difference in the variation in the work time with the skilled worker is small, while the difference in the variation in the joint position with the skilled worker is large in the joint part variation judgment result display area 1103 of the same work process P2, so that it is understood that the accuracy of the work is low. In addition, in the work process P6 displayed in the time variation judgment result display area 1102, the difference in the variation in the work time with the skilled worker is large, while the difference in the variation in the joint position with the skilled worker is small in the joint part variation judgment result display area 1103 of the same work process P6, so that it is understood that the accuracy of the work is high. In this way, the correlation between the work delay and the work accuracy in each work process or work procedure can be easily grasped. Furthermore, the details of the work process displayed in each area displayed on the proficiency judgment result screen shown in FIG. 11 can be confirmed. The screen for confirming the details will be described later.

[0058] Furthermore, the variation calculation unit 300 displays repetitive action display areas 1104 and 1105, which indicate that a repetitive action is included in a work process or a work procedure, in association with the work process or the work procedure. In FIG. 11, the variation calculation unit 300 displays "RP2" in the repetitive action display areas 1104 and 1105, which indicates that the work process P2 includes a work procedure including a repetitive action, and that the number of times is two. In this way, the variation calculation unit 300 displays information indicating that a repetitive action is included in a work process or a work procedure in the repetitive action display areas 1104 and 1105, so that the user can infer the reason for the variation at a glance. The variation calculation unit 300 may determine which work process or work procedure includes a repetitive action by, for example, referring to the work sequence information 601 shown in FIG. 6A and FIG. 6B.

[0059] Fig. 12 is a diagram showing an example of a variation judgment result details screen showing details of the proficiency judgment result screen shown in Fig. 11. When a user selects a work process or work procedure displayed in each display area of ​​the proficiency judgment result screen 1101, the variation calculation unit 300 displays a variation judgment result details screen on a display device such as a display. In the following, as an example, a case where an image of specific reference area information 801 is displayed is illustrated, but reference area information 701 may also be displayed.

[0060] In the variation judgment result detail screen 1201 shown in FIG. 12, as an example, a detailed screen of the proficiency judgment result in the work process P6 displayed in the time variation judgment result display area 1102 and the joint part variation judgment result display area 1103 is illustrated. In FIG. 12, a reference specific reference area information display area 1202 is displayed, which displays the image or video of the specific reference area information 801 stored in association with the divided reference work data 501A. The slide bar B1 is an adjustment bar that can change the time when the image or video was captured. For example, when the user clicks the right end arrow of the slide bar B1, the variation calculation unit 300 plays the image or video of the skilled worker working in the future. This allows the user to visually check the image or video of the skilled worker working, which changes from moment to moment in the future. When the user clicks the left end arrow of the slide bar B1, the variation calculation unit 300 plays the image or video of the skilled worker working in the past. This allows the user to visually check images and videos of the skilled worker performing work that change from moment to moment over the past. In particular, for repetitive movements, the same movement is performed repeatedly with the same tempo and the same hand position, so by playing back the images and videos, the user can easily understand the variations in each movement in the repetitive movements.

[0061] In addition, in FIG. 12, a specific reference area information display area 1203 is displayed, which displays an image or video of the specific reference area information 801 stored in association with the comparison task data 501. The slide bar B2 is an adjustment bar that can change the time when the image or video was captured. For example, when the user clicks the right end arrow of the slide bar B2, the variation calculation unit 300 plays an image or video of the worker working in the future. This allows the user to visually check the image or video of the worker working, which changes from moment to moment, in the future. When the user clicks the left end arrow of the slide bar B2, the variation calculation unit 300 plays an image or video of the worker working in the past. This allows the user to visually check the image or video of the worker working, which changes from moment to moment, in the past. In particular, for repetitive actions, as in the case of the reference specific reference area information display area 1202, it is easy to understand what kind of variation there is in each action in the repetitive action. The user can then compare the reference specific reference area information display area 1202 and the specific reference area information display area 1203 to confirm the difference in the variations.

[0062] Moreover, the variation calculation unit 300 displays, in the reference area information display area 1203, a difference range D indicating which range on the image is different, by comparing it with the image in the reference specific reference area information display area 1202. Furthermore, the variation calculation unit 300 displays a marking part M in which a joint part included in the difference range D is marked as a joint part whose proficiency is lower than that of the expert by a certain threshold or more. For example, the variation calculation unit 300 compares the positions of the same joint part (e.g., hand, elbow) on the image displayed in the reference area information display area 1203 and the image displayed in the reference specific reference area information display area 1202 at the same time to specify the difference range D. Then, among the joint parts included in the difference range D, a part whose proficiency is lower by a certain threshold or more (e.g., a part whose position has a large difference in variance between the expert and the worker by a certain threshold or more) is displayed as the marking part M. It can be seen from FIG. 12 that the joint parts of the hand and elbow of the worker W2 are highlighted as the marking part M.

[0063] In this way, on the variation judgment result details screen 1201, the reference specific reference area information display area 1202 and the reference area information display area 1203 are displayed on one screen, and the difference between the two images is displayed as the marking part M. Therefore, the user can grasp at a glance which work process or work procedure the worker's work differs from that of the skilled worker. In particular, for repetitive actions, the part that causes the variation in each action in the repetitive action can be easily grasped.

[0064] Furthermore, the variation judgment result details screen 1201 displays a delay status graph display area 1204 that shows the extent to which the comparison task data 501 is delayed (or accelerated) in time relative to the divided reference task data 501A. As shown in the delay status graph display area 1204, the horizontal axis is time and the vertical axis is the difference, which is the delay status, and the change in the temporal difference between the comparison task data 501 and the divided reference task data 501A is displayed for each time. In this example, it is shown that the task is continuously delayed during a certain time period t1-t2 of a certain task procedure or task. In this way, the variation calculation unit 300 calculates the temporal difference between the comparison task data 501 and the divided reference task data 501A and displays the calculated difference in the delay status graph display area 1204 as the task delay status, so that the user can grasp at a glance the progress of the delay of the task of the worker relative to the skilled worker.

[0065] In addition, the variation judgment result details screen 1201 displays a variation status graph display area 1205 that shows the degree of variation in the position of the part of the compared task data 501 relative to the divided reference task data 501A in a time series. As shown in the variation status graph display area 1205, the horizontal axis is time and the vertical axis is the difference in joint position, and the change in the spatial difference between the compared task data 501 and the divided reference task data 501A is displayed for each time. In this example, it is shown that the variation of the joint position of a certain joint position (finger joint) becomes large at certain times t3 and t4. In this way, the variation calculation unit 300 calculates the spatial difference between the compared task data 501 and the divided reference task data 501A for each time, and displays the calculated difference in the variation status graph display area 1205 as the degree of variation in the task position, so that the user can grasp at a glance the transition of the variation in the position of the part of the worker relative to the skilled worker.

[0066] In this way, the variation judgment result details screen 1201 displays the delay status graph display area 1204 and the variation status graph display area 1205 on one screen. Therefore, the user can grasp the difference in time variation and the difference in spatial variation in a certain work process or work procedure at a glance. Furthermore, by displaying the delay status graph display area 1204 and the variation status graph display area 1205 on one screen together with the above-mentioned reference specific reference area information display area 1202 and reference area information display area 1203, the user can easily grasp the correlation between them.

[0067] Furthermore, when a repetitive action is included in a worker's work process or work procedure, the user can easily understand what kind of variation has occurred in the repetitive action by comparing the reference specific reference area information display area 1202 and the reference area information display area 1203 on the variation judgment result details screen 120. For example, when two repetitive actions are included in the work process P2, the user can check in detail the delay status of the first and second actions by the worker W2, the transition of the variation of the parts in the first and second actions, and the like.

[0068] Next, the processing performed in the task support system 1000 will be described. Fig. 13 is a flowchart showing the procedure of processing (proficiency assessment processing) performed in the task support system 1000. In the skill assessment processing, first, the section division unit 100 receives imaging data 301 output from a sensor S such as a camera, and acquires sensor data 401 about the worker included in the received imaging data 301 (S1301). The section division unit 100 outputs comparison task data 501 in which the acquired sensor data 401 is associated with the task sequence information 601 shown in Figs. 6A and 6B (S1302).

[0069] Next, the identification area selection unit 200 selects an identification area for identifying a worker using the comparison task data 501 and the reference area information 701, and also selects an identification area for identifying an expert using the divided reference task data 501A and the reference area information 701 (S1303). The identification area selection unit 200 specifies the parts of the worker and expert who perform the task in the selected identification area that directly act on the work as the parts to be subjected to proficiency judgment (S1304).

[0070] The variation calculation unit 300 uses the comparison task data 501 for the worker included in the selected identification area, the divided reference task data 501A for the expert included in the selected identification area, and the task sequence information 601 to determine the proficiency of the worker relative to the expert based on the difference in variance in each task step and task procedure and the difference in variance in the repetitive motions included therein (S1305), and displays the proficiency determination result screen shown in Fig. 11 (S1306). That is, the identification area selection unit 200 specifies, among the predetermined parts of the worker in the selected identification area, parts that have a predetermined positional relationship with the task object as parts that directly act on the task object, and the variation calculation unit 300 determines the proficiency of the worker based on the difference in variance in the repetitive motions for the specified parts that directly act on the worker, and presents the result of the determination.

[0071] In this manner, in this embodiment, the section division unit 100 uses sensor data 401 of a predetermined part (for example, a joint part) of the worker obtained by analyzing image data 301 capturing an image of a worker performing work on a work object, including a repetitive motion, and work sequence information 601 defining the steps or steps of the work, to output comparison work data 501 in which the sensor data 401 is associated with the steps or steps of the work, and selects an identification area for a worker to be subjected to proficiency judgment based on the image data 301 and reference area information 701 defining the work position, which is the position of the work object, and the work position where the worker performs the work, and then determines the worker in the selected identification area and the location of the worker. The system includes an identification area selection unit 200 that matches the comparison work data of a fixed portion with reference sensor data (e.g., sensor data 301 of an expert) that is a standard for determining the proficiency of the worker, reference work data (e.g., divided reference work data 501A of an expert) that is determined using reference sensor data (e.g., sensor data 301 of an expert) that is a standard for determining the proficiency of the worker and the work sequence information 601, and that matches the reference sensor data with the process or procedure of the work, and a variance calculation unit 300 that uses the comparison work data 501 of a worker in the selected identification area and the work sequence information 601 to determine the proficiency of the worker based on the difference in variance of repetitive actions in the process or procedure of the work, and presents the result of the determination.

[0072] Therefore, the system can grasp not only the difference in the variation in the motion of the worker in each work process or work procedure compared to the skilled worker, but also the difference in the variation in the motion in the repetitive work, and can support the work of the worker so as not to cause a decrease in the work efficiency and work accuracy. That is, based on the image data 301 acquired by the imaging unit and the work sequence information 601 indicating the work process or work procedure, the identification area of ​​the worker is selected, the proficiency is judged from the difference in the degree of variation in the joint positions related to the repetitive motion of the worker in the identification area, and the judgment result is output as an improvement point after identifying the work defined in the work sequence information, so that the above-mentioned support is possible. Conventionally, the proficiency of the worker was not judged in consideration of the relationship between the positions of parts and products related to the work process or work procedure such as an assembly drawing and the motion of the worker, that is, the relationship between the position of the work object on which the worker performs the work according to the work process or work procedure and the motion of the worker. However, according to the present system, as described above, it is possible to support the work by considering the variation in the motion in the work process or work procedure, including the repetitive work. EXAMPLES

[0073] In the first embodiment, the work process and the work procedures included in the work process are determined in advance, and the difference in the variation of the motions and repetitive motions in the work process and the work procedures is calculated and scored to determine the proficiency level, and the result is presented to the user. However, the variation in motion is not necessarily limited to those explicitly included in the work process and the work procedures. For example, when repeating a series of motions such as picking up a tool and screwing a part, the difference in the motions during preparation before picking up the tool (for example, the motion of placing multiple tools in the same direction before work) between an expert and an unexperienced worker may cause variation in the subsequent motions. Therefore, in the following, the proficiency level is determined not only in the motions included in the work process and the work procedures, but also in the approach motion, which is a motion for preparation performed between the work process and the work procedures, so that the user can be presented with more objective information for determining the variation in motion. As described above, the approach motion is a motion during preparation for starting work, but the approach motion may include a repetitive work, as in the case of the first embodiment. For example, when the action of placing a plurality of tools in the same direction is repeatedly performed before a task, the action can be said to be an approach action that includes a repetitive action.

[0074] The work support system 2000 in the second embodiment differs from the work support system 1000 in the first embodiment in that it has comparison work data 2001, divided reference work data 2001A, and work sequence information 2002 which are different from the comparison work data 501, divided reference work data 501A, and work sequence information 601 in the first embodiment. Therefore, in the following, these points which are different from the first embodiment will be mainly described, and the other components will be given the same reference numerals and their description will be omitted.

[0075] Fig. 14 is a diagram showing an example of the configuration of a task support system 2000 in this embodiment. As shown in Fig. 14, the task support system 2000 has comparison task data 2001, divided reference task data 2001A, and task sequence information 2002, which are different from those in the first embodiment.

[0076] FIG. 15 is a diagram showing an example of the comparative work data 2001 divided into work processes or work steps. As shown in FIG. 15, the comparative work data 2001 is data in which the sensor data 401 is associated with the work sequence information 2002, which is data on the work process and work time of the work performed by the worker. The comparative work data 2001 in FIG. 15, unlike the case of the first embodiment, includes a predetermined spare time between a certain work process or work step and the next work process or work step. For example, FIG. 15 shows that the spare time P12 is included between the work processes P1 and P2, and the spare time P56 is included between the work processes P5 and P6. In this way, the sensor data 401 divided into the sections of the work processes P1 to P9, including the spare time P12 and the spare time P56, is stored as the comparative work data 2001, and the comparative work data 2001 stores the sensor data 401 and the above-mentioned work sequence information 2002 in association with each other. As in the first embodiment, the divided reference task data 2001A has the same structure as the comparison task data 2001 divided into task processes or task steps, and therefore a detailed description thereof is omitted here. Although Fig. 15 shows an example in which there is spare time between task processes, the same applies to cases in which there is spare time between task steps.

[0077] 16 is a diagram showing an example of the work sequence information 2002. The work sequence information 2002 includes work process sequence information in which a standard work time is defined for each work process, and work procedure sequence information in which a standard procedure time is defined for procedures included in each work process.

[0078] As shown in FIG. 16, in the work process sequence information 2002A, a standard work time is defined for each work process. The section division unit 100 can detect the transition of the sensor value in each work process corresponding to the standard work time by overlapping the work process sequence information 2002A and the sensor data 401. Unlike the case of the first embodiment, FIG. 16 shows that there is a two-minute spare time P12 between work process 1 and work process 2, and there is a two-minute spare time P56 between work process 5 and work process 6. In other words, it shows that the spare time is the difference between the end time of the previous work process and the start time of the next process. The section division unit 100 detects the transition of the sensor value in the spare time as in each work process. During such spare time, for example, an action for preparation before starting work, such as arranging multiple tools in the same direction before work, is performed. Although not illustrated below, the spare time between work procedures may be considered in the same way for the work procedure sequence information 2002B as for the work procedure sequence information 2002A.

[0079] Fig. 17 is a diagram showing an example of a proficiency judgment result screen displayed on a display device such as a display by the variance calculation unit 300. The proficiency judgment result screen 1701 shown in Fig. 17 includes a time variation judgment result display area 1702 similar to the time variation judgment result display area 1102 for the time comparison variance data 901 shown in Fig. 9, and a joint part variance judgment result display area 1703 similar to the joint part variance judgment result display area 1103 for the joint part comparison variance data 1001 shown in Fig. 10.

[0080] In FIG. 17, unlike the first embodiment, the difference in time variation in the reserve time P12 and the reserve time P56, and the difference in joint part variation are displayed. It can be seen that the reserve time P12 and the reserve time P56 are both displayed in dark colors because the difference in variation is large. In this way, in this embodiment, the judgment results of the proficiency are displayed for the approach motion, which is a motion performed in the reserve time between the work steps P1 and P2 and between the work steps P5 and P6, and for the repeat motion if the approach motion includes a repetitive motion. Therefore, the proficiency is judged for these motions performed in the reserve time for which the process content or procedure content is not explicitly defined in the work step sequence information 601, and the results are displayed. In FIG. 17, the variation calculation unit 300 displays the approach motion (AP12) in the same way as the repeat motion (RP2) so that it can be visually recognized as an approach motion.

[0081] The variation calculation unit 300 displays the approach action display areas 1704 and 1705, which indicate that the approach action is included in the reserve time between the work steps or work procedures, in association with the reserve time. In FIG. 17, the variation calculation unit 300 displays "AP12" in the approach action display areas 1704 and 1705, which indicates that the approach action is included in the reserve time between the work steps P1 and P2. The approach action "AP56", which is the reserve time between the work steps P5 and P6, is similarly displayed. If the approach action includes a repetitive action, these can be displayed side by side. In this way, the variation calculation unit 300 displays information indicating that the approach action performed between the work steps or work procedures is included in the approach action display areas 1704 and 1705, so that the user can see at a glance that there is a difference in the variation in the action in the reserve time where no specific work content is specified. Although not shown in the following description, the variation judgment result details screen 1201 as shown in FIG. 12 is also displayed for the approach movement in the above-mentioned spare time, similarly to the first embodiment.

[0082] FIG. 18 is a flowchart showing the procedure of the process (proficiency assessment process) performed by the work support system 2000. In the proficiency assessment process in this embodiment, the processes from S1301 to S1304 are performed in the same manner as in the first embodiment. After that, the variance calculation unit 300 uses the comparison task data 501 for the worker included in the selected identification area, the divided reference task data 501A for the skilled worker included in the selected identification area, and the task sequence information 2002 to determine whether or not there is spare time between task steps or task procedures (S1801). For example, the variance calculation unit 300 refers to the task sequence information 2002 and determines whether or not there is discontinuous time between each task step or task procedure, thereby determining whether or not there is spare time between those task steps or task procedures.

[0083] When it is determined that there is a discontinuous time between each work process or each work procedure and that there is spare time between those work processes or work procedures (S1801; Yes), the variance calculation unit 300 determines the proficiency of the worker relative to that of an expert based on the difference in variance in the approach movement of the spare time, the difference in variance in the repetitive movements included in the approach movement, the difference in variance in each work process or work procedure, and the difference in variance in the repetitive movements included therein (S1802).

[0084] On the other hand, if the variance calculation unit 300 determines that there is no discontinuous time between each work process or each work procedure and that there is no spare time between those work processes or work procedures (S1801; No), it performs processing similar to S1305 in the first embodiment.

[0085] The variation calculation section 300 displays the result of the determination made in S1802 or S1305 as a skill determination result screen shown in FIG. 17 (S1803).

[0086] That is, the identification area selection unit 200 identifies, among the specified body parts of the worker in the selected identification area, those parts that have a specified positional relationship with the work object as parts that directly act on the work object, and the variance calculation unit 300 judges the proficiency of the worker based on the difference in variance in the approach movement for the identified directly acting parts, and presents the result of the judgment.

[0087] As described above, in this embodiment, the section division unit 100 outputs the comparison work data 2001 using the work sequence information 2002 including the spare time between the work steps or procedures and the sensor data 401, and the identification area selection unit 200 uses the reference work data 2001A determined using reference sensor data (e.g., sensor data 301 of an expert) serving as a standard for determining the worker's proficiency and the work sequence information 2002 including the spare time to determine the worker's proficiency based on the difference in variation in the preparatory approach movements performed by the worker during the spare time between the work steps or procedures, and presents the result of the determination.

[0088] Therefore, even if there is an approach motion performed between work steps or work procedures, the judgment result of the proficiency of the approach motion can be presented. For example, in the case of repeating a series of actions of picking up a tool and screwing a part, if the step of "screwing a part" is defined in the work sequence information, but there is a time that is discontinuous with the previous work step or work procedure, that time is determined to be preparation time, and the proficiency of the worker is judged for the approach motion at that time (for example, an action in preparation before picking up a tool, such as placing multiple tools in the same direction before work) and further for the repeated motion, and the judgment result is output. Therefore, it is possible to grasp the proficiency of the worker and the difference in motion from an expert for the approach motion during the preparation time for performing a work defined as a work step or work procedure.

[0089] Although each embodiment has been described above, various modifications are possible as shown below. For example, in the above embodiment, the divided reference task data 501A of the expert is used as reference data for determining the proficiency of the worker. However, since the body structure and skeleton are different between the expert and the worker, in order to improve accuracy, the divided reference task data 501A of the expert may be compared with the comparative task data 501 of the worker after skeleton correction. For example, the section division unit 100 compares the skeleton obtained by image analysis of the worker included in the imaging data 301 with the skeleton obtained by image analysis of the expert included in the past imaging data 301, and calculates the degree of enlargement or reduction of the image or video according to the magnitude of the difference in the same joint position. The area selection unit 200 corrects the position of the joint part of the worker included in the identification area according to the calculated degree of enlargement or reduction, and further specifies the part that directly acts on the work object among the joint parts at the corrected position as the part to be subjected to proficiency judgment. That is, the section division unit 100 corrects the position of a predetermined part of the expert included in the image data 301 that is the analysis source of the reference sensor data (e.g., the sensor data 301 of the expert) that is the standard for judging the proficiency of the worker to the position of the predetermined part of the worker obtained by analyzing the image data 301, and the variation calculation unit 300 judges the proficiency of the worker based on the difference in the variation of the repetitive motion in the process or procedure of the work using the reference work data 2001A that is determined using the reference sensor data after the correction and the work sequence information 2002, and presents the result of the judgment. In this way, the part for proficiency judgment can be specified taking into consideration the difference in the skeleton between the expert and the worker, and the judgment result of the proficiency can be presented.

[0090] Furthermore, in each of the above-mentioned embodiments, the divided reference task data 501A of the skilled worker is assumed to be the task data of the skilled worker at a certain point in the past. However, the same skilled worker will not necessarily be better than the worker in the future. Therefore, when the proficiency level of the worker determined in S1305 of FIG. 13 or S1802 or S1305 of FIG. 18 is superior to that of the worker (for example, the score of the difference in variance is negative), the variation calculation unit 300 may register the comparative task data 501 of the worker as the divided reference task data 501A of the new skilled worker. That is, when the proficiency level of the worker is superior to that of the skilled worker as a result of determining the proficiency level of the worker based on the difference in variance, the variation calculation unit 300 registers the comparative task data 501 of the worker determined to be superior as the new reference task data 501A. As a result, when the task level of the worker exceeds that of the skilled worker, the comparative task data of the worker can be automatically registered as reference data.

[0091] In addition, in each of the above-mentioned embodiments, the divided reference task data 501A of the expert is assumed to be the task data of the expert at a certain point in the past. However, one expert is not necessarily good at all work steps or work procedures, and there are cases where expert A is good at one task and expert B is good at another task. Therefore, the variation calculation unit 300 determines which work steps or work procedures the worker is better at (for example, the score of the difference in variation is negative) from the proficiency determined in S1305 of Fig. 13 and S1802 or S1305 of Fig. 18, and updates the data of the work steps or work procedures determined to be excellent in the divided reference task data 501A using the data of the work steps or work procedures determined to be excellent. When the proficiency of the worker is judged based on the difference in the variance and the worker is judged to be more skilled than the expert, the variance calculation unit 300 uses the data on the work process or work procedure judged to be superior (for example, the data on work process P2 included in FIG. 5 or FIG. 15) to update the data on the work process or work procedure judged to be superior among the reference work data (for example, the data corresponding to work process P2). As a result, when the work level of the worker exceeds the work level of the expert, the data is updated for each work process or work procedure, and for example, new reference data that minimizes the work time can be automatically registered.

[0092] Furthermore, the variation calculation unit 300 may register the comparative task data 501 of the worker as the divided reference task data 501A of the skilled worker when the task time of the task process or task procedure corresponding to the comparative task data 501 of the worker and the task time of the task process or task procedure corresponding to the divided reference task data 501A of the skilled worker are within a predetermined range (for example, ±20%) with respect to the MOD value calculated by the MODAPTS (modular arrangement of predetermined timestandards) method. That is, the variation calculation unit 300 may register the comparative task data 501 of the worker as the reference task data 501A of the skilled worker when the task time of the task process or task procedure corresponding to the comparative task data 501 of the worker calculated by the MODAPTS method is within a predetermined range of the MOD value as a result of judging the proficiency of the worker based on the difference in the variation. In this case, the divided reference task data 501A of the skilled worker can be asymptotically approached to the task time data calculated by the PTS method.

[0093] In the above embodiments, examples have been given of cases in which the positions of the work table and the work are fixed, but the same ideas as in the above embodiments can also be applied in cases in which the range of these positions is somewhat fixed, such as in line production, where the range is determined to some extent, such as being located within a predetermined range (for example, 3 meters square). [Explanation of symbols]

[0094] 1000, 2000 Work Support System 100 Section division 200 Identification area selection unit 300 Variation Calculation Section 400 Work sequence controller 501, 2001 Comparison work data 501A, 2001A Divided Reference Work Data 601, 2002 Work sequence information 701 Reference Area Information

Claims

1. a section division unit that uses sensor data of a predetermined part of a worker obtained by analyzing image data of a worker performing a task on a work object, including a repetitive motion, and task sequence information that defines a process or procedure of the task, to output comparative task data in which the sensor data is associated with the process or procedure of the task; an identification area selection unit that selects an identification area of ​​a worker whose proficiency level is to be determined based on the image data and reference area information that defines a work position, which is the position of the work object, and a work position where the worker performs work, and associates the worker in the selected identification area with the comparative work data of a predetermined part of the worker; a variance calculation unit that uses reference work data, which is determined using reference sensor data that serves as a standard for judging the proficiency of the worker and the work sequence information, to associate the reference sensor data with a process or procedure of the work, the comparative work data of a worker in the selected identification area, and the work sequence information, to judge the proficiency of the worker based on a difference in variance of repetitive actions in the process or procedure of the work, and presents the result of the judgment; A work support system comprising:

2. The section dividing unit outputs the comparison work data using the work sequence information including spare times between the steps or procedures of the work and the sensor data, the identification area selection unit uses the reference task data determined using the reference sensor data and the task sequence information including the spare time to determine the proficiency of the worker based on a difference in variation in a preparatory approach motion performed by the worker during the spare time between the steps or procedures of the task, and presents a result of the determination.

2. The work support system according to claim 1 .

3. the identification area selection unit identifies, among predetermined body parts of the worker in the selected identification area, a body part that has a predetermined positional relationship with the work object as a body part that directly acts on the work object; The variation calculation unit determines a proficiency level of the worker based on a difference in variation of the repetitive motion for the specified directly acting part, and presents a result of the determination.

2. The work support system according to claim 1 .

4. the identification area selection unit identifies, among predetermined body parts of the worker in the selected identification area, a body part that has a predetermined positional relationship with the work object as a body part that directly acts on the work object; The variation calculation unit determines a proficiency level of the worker based on a difference in variation of the approach motion for the specified directly acting part, and presents a result of the determination.

3. The work support system according to claim 2.

5. the section dividing unit corrects a position of a predetermined part of the expert included in the imaging data on which the reference sensor data is analyzed to a position of the predetermined part of the expert obtained by analyzing the imaging data; the variation calculation unit uses the reference work data determined using the corrected standard sensor data and the work sequence information to determine the proficiency of the worker based on a difference in variation of a repetitive motion in a process or procedure of the work, and presents a result of the determination.

2. The work support system according to claim 1 .

6. When the proficiency of the worker is superior to that of an expert as a result of judging the proficiency of the worker based on the difference in the variance, the variation calculation unit updates the data of the work process or work procedure judged to be superior among the reference work data using the data of the work process or work procedure judged to be superior.

2. The work support system according to claim 1 .

7. When the proficiency of the worker is superior to that of the expert as a result of judging the proficiency of the worker based on the difference in the variance, the variation calculation unit registers the comparative task data of the worker judged to be superior as new reference task data.

2. The work support system according to claim 1 .

8. The variance calculation unit, as a result of judging the proficiency of the worker based on the difference in the variance, registers the comparative work data of the worker as reference work data of a skilled worker when the work time of the work process or work procedure corresponding to the comparative work data of the worker, calculated by the MODAPTS method, falls within a predetermined range of MOD values.

2. The work support system according to claim 1 .

9. A work assistance method for assisting a worker in a work, which is carried out by a computer, comprising the steps of: the section dividing unit uses sensor data of a predetermined part of the worker obtained by analyzing image data of a worker performing a task on a work object, including a repetitive motion, and task sequence information that defines a process or procedure of the task, to output comparative task data in which the sensor data is associated with the process or procedure of the task; an identification area selection unit selects an identification area of ​​a worker whose proficiency is to be judged based on reference area information that defines a work position, which is the position of the work object, and a work position where the worker performs work, and the image data, and associates the worker in the selected identification area with the comparison work data of a predetermined part of the worker; a variation calculation unit uses reference work data, which is determined using reference sensor data serving as a standard for determining the proficiency of the worker and the work sequence information, and which associates the reference sensor data with the work process or procedure, the comparative work data of the worker in the selected identification area, and the work sequence information to determine the proficiency of the worker based on a difference in variation in repetitive actions in the work process or procedure, and presents the result of the determination; A work support method comprising the steps of:

Images