Work evaluation apparatus

Abstract

To provide a work evaluation apparatus which can evaluate work of a worker with high accuracy.SOLUTION: A work evaluation apparatus includes: a detection unit which detects coordinates of a skeleton of a first worker in a first image captured by an imaging apparatus and coordinates of a skeleton of a second worker in a second image captured by the imaging apparatus; an evaluation unit which evaluates the work of a second worker based on temporal change of the coordinates of the skeleton of the working second worker in the second image, using a learning model which has been trained by machine learning, in time series, with the coordinates of the skeleton of the working first worker in the first image; a correction unit which corrects the coordinates of the skeleton of the first worker with which the learning model is trained, or the coordinates of the skeleton of the second worker detected by the detection unit, using correction values; and a calculation unit which calculates the correction values from differences of coordinates of the skeletons due to a difference in physique between the first worker in the first image and the second worker in the second image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an activity evaluation device. [Background technology]

[0002] For example, Patent Document 1 describes a service that recognizes human behavior from camera footage based on skeletal information, using a learning model in which basic movements have already been learned according to machine learning algorithms such as deep learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-3434 Summary of the Invention [Problem to be solved by the invention]

[0004] However, for example, when evaluating work using a learning model such as that described above based on the skeleton of a worker in an image, there is a risk that the worker's movements will not be evaluated accurately due to differences in physique between the worker and the person who performed the basic movements that are the subject of the learning model.

[0005] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a task evaluation device that can evaluate the tasks of a worker with high accuracy. [Means for solving the problem]

[0006] The work evaluation device of the present invention has a detection unit that detects skeletal coordinates of a first worker in a first image captured by an imaging device and skeletal coordinates of a second worker in a second image captured by the imaging device; an evaluation unit that uses a learning model that has been machine-learned to learn the skeletal coordinates of the first worker while working in the first image according to a time series, and evaluates the work of the second worker based on changes over time in the skeletal coordinates of the second worker while working in the second image; a correction unit that corrects the skeletal coordinates of the first worker learned by the learning model or the skeletal coordinates of the second worker detected by the detection unit with a correction value; and a calculation unit that calculates the correction value from the difference in skeletal coordinates due to a difference in physique between the first worker in the first image and the second worker in the second image.

[0007] In the above-mentioned work evaluation device, the calculation unit may calculate the correction value from the difference between the center coordinates of a line connecting both shoulders of the first worker in the first image and the center coordinates of a line connecting both shoulders of the second worker in the second image.

[0008] In the above task evaluation device, the calculation section may vary the correction value for each skeleton depending on a type of task performed by the first worker.

[0009] The above-mentioned work evaluation device has a judgment unit that calculates a height difference between the first worker and the second worker from the first image and the second image, and judges whether or not correction by the correction unit is necessary based on the height difference, and the correction unit does not need to correct the skeletal coordinates when the judgment unit judges that correction is unnecessary.

[0010] The above-mentioned work evaluation device has a judgment unit that calculates a difference in shoulder width between the first worker and the second worker from the first image and the second image, and judges whether or not correction by the correction unit is necessary based on the difference in shoulder width, and the correction unit does not need to correct the skeletal coordinates when the judgment unit judges that correction is unnecessary. Effect of the Invention

[0011] According to the present invention, the work of a worker can be evaluated with high accuracy. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a configuration diagram showing an example of a task evaluation system. [Diagram 2] FIG. 2 is a diagram showing an example of correction of skeleton coordinates. [Diagram 3] FIG. 3 is a diagram illustrating an example of calculation of the correction value. [Figure 4] FIG. 4 is a flowchart showing an example of the correction determination process. [Diagram 5] FIG. 5 is a flowchart showing an example of the task evaluation process. [Figure 6] FIG. 6 is a flowchart showing another example of the task evaluation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Example of the configuration of a work evaluation system) 1 is a configuration diagram showing an example of a work evaluation system 9. As an example, the work evaluation system 9 is installed in an assembly plant for automobiles or the like.

[0014] The work evaluation system 9 includes a work evaluation server 1, a camera device 2, and a terminal 3 which are capable of communicating with each other via a LAN (Local Area Network) 90. The work evaluation server 1 is an example of a work evaluation device. The work evaluation server 1 evaluates the work actions of a worker. The camera device 2 is an example of an imaging device, and captures images of people such as workers. The terminal 3 is carried, for example, by the worker or a related person, and receives the work evaluation results from the work evaluation server 1 and notifies the worker, etc.

[0015] The work evaluation server 1 has a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, a HDD (Hard Disk Drive) 13, and a communication port 14. The CPU 10 is electrically connected to the ROM 11, the RAM 12, the HDD 13, and the communication port 14 via a bus 19 so as to be able to input and output signals among them.

[0016] The ROM 11 stores a program that drives the CPU 10. The RAM 12 functions as a working memory for the CPU 10. The communication port 14 is, for example, a wireless LAN card, and processes communication of the CPU 10 via the LAN 90.

[0017] When the CPU 10 reads a program from the ROM 11, it forms, as software functions, a device control unit 100, a skeleton detection unit 101, a correction necessity determination unit 102, a correction value calculation unit 103, a skeleton data correction unit 104, and an operation evaluation unit 105. In addition, the HDD 13 stores detected skeleton data 130, learned skeleton data 131, evaluation data 132, and a learning model 133.

[0018] The device control unit 100 instructs various operations to the skeleton detection unit 101, the correction necessity determination unit 102, the correction value calculation unit 103, the skeleton data correction unit 104, and the task evaluation unit 105 in accordance with a sequence defined in a program. Note that the device control unit 100, the skeleton detection unit 101, the correction necessity determination unit 102, the correction value calculation unit 103, the skeleton data correction unit 104, and the task evaluation unit 105 are not limited to being realized by software, and may be realized by hardware such as an ASIC (Application Specified Integrated Circuit).

[0019] The skeleton detection unit 101 is an example of a detection unit. The skeleton detection unit 101 detects the coordinates of the skeleton of a person in an image captured by the camera device 2. The skeleton detection unit 101 receives image data from the camera device 2 via the LAN 90. The skeleton detection unit 101 detects the coordinates of the skeleton of the person by image analysis using, for example, a general-purpose AI (Artificial Intelligence).

[0020] When the task evaluation unit 105 evaluates a worker to be evaluated (hereinafter referred to as the evaluation worker), the camera device 2 captures an image of the evaluation worker. In addition, the camera device 2 captures an image of an exemplary worker (hereinafter referred to as the exemplary worker) in order to allow the learning model 133 used by the task evaluation unit 105 for evaluation to learn the coordinates of the skeleton of the evaluation standard. Note that the exemplary worker is an example of a first worker, and the evaluation worker is an example of a second worker. Also, the image captured of the exemplary worker is an example of the first image, and the image captured of the evaluation worker is an example of the second image.

[0021] The skeleton detection unit 101 detects the coordinates of the evaluation worker's skeleton from the image of the evaluation worker and stores them in the HDD 13 as detected skeleton data 130. The detected skeleton data 130 indicates, for example, the coordinates of the evaluation worker's nose, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, and right hip in the image in chronological order for each frame number of the image. The coordinates are indicated by the X coordinate and the Y coordinate (x, y) when the horizontal and vertical directions of the rectangular frame of the image are defined as the X axis and the Y axis, respectively. There is no limitation on the position of the origin.

[0022] The work evaluation unit 105 is an example of an evaluation unit. The work evaluation unit 105 uses a learning model 133 that has been machine-learned according to a time series of the skeletal coordinates of the model worker working in the image, and evaluates the work of the evaluation worker based on the time change in the skeletal coordinates of the evaluation worker working in the image. The work evaluation unit 105 determines whether the time change in the skeletal coordinates of the evaluation worker recorded in the detected skeleton data 130 is similar to the time change in the skeletal coordinates of the evaluation worker recorded in the learned skeleton data 131, thereby determining whether the evaluation worker is working in accordance with a predetermined standard indicated by the model worker.

[0023] The learning model 133 is stored in the HDD 13 as a set of various calculation modules and parameters. In the learning process of the learning model 133, the evaluation result of the work is annotated using the time change in the coordinates of the skeleton of the model worker during the work as teacher data. The learning model 133 is a neural network that mathematically models the function of the human brain, and determines the weight coefficient of the activation function of the part corresponding to the neuron based on machine learning. In this way, the work evaluation unit 105 can output the evaluation result with high accuracy by using the learning model 133 that has been machine-learned by supervised learning.

[0024] The coordinates of the skeleton of the model worker learned by the learning model 133 are stored in the HDD 13 as learned skeleton data 131. Similar to the detected skeleton data 130, the learned skeleton data 131 indicates the coordinates of the nose, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, and right hip of the evaluation worker in the image in chronological order for each frame number of the image. The learning model 133 outputs an evaluation result based on the degree of difference between the learned skeleton data 131 and the detected skeleton data 130.

[0025] In this way, the work evaluation unit 105 determines the evaluation result as "good" if the evaluator's work complies with the predetermined criteria, and determines the evaluation result as "fail" if the evaluator's work does not complies with the predetermined criteria. The work evaluation unit 105 saves the evaluation result as evaluation data 132 in the HDD 13. The work evaluation unit 105 transmits the evaluation data 132 to the terminal 3 via the LAN 90. This notifies the evaluator of the evaluation result of the work, and the evaluator's work is improved.

[0026] The skeleton data correction unit 104 is an example of a correction unit. The skeleton data correction unit 104 corrects the skeleton coordinates of the model worker learned by the learning model 133 using a correction value. For example, the skeleton data correction unit 104 corrects the skeleton coordinates of the learning skeleton data 131. At this time, the skeleton data correction unit 104 uses individual correction values ​​for each coordinate of the nose, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, and right hip of the learning skeleton data 131. Note that, as described later, the skeleton data correction unit 104 may correct the skeleton coordinates of the detected skeleton data 130 instead of the learning skeleton data 131.

[0027] The correction value calculation unit 103 is an example of a calculation unit. The correction value calculation unit 103 calculates a correction value for the learning skeleton data 131 from the difference in skeleton coordinates due to the difference in physique between the model worker and the evaluation worker in the image. For example, the correction value calculation unit 103 calculates a correction value from the difference in skeleton coordinates between the model worker and the evaluation worker who are in the same position and posture. The correction value calculation unit 103 calculates individual correction values ​​for the coordinates of the nose, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, and right hip.

[0028] This reduces the effect of differences in physique between the evaluating worker and the model worker on the evaluation results of the work, allowing the work evaluation section 105 to evaluate the work with high accuracy.

[0029] The correction necessity determination unit 102 is an example of a determination unit. The correction necessity determination unit 102 calculates the height difference between the evaluation worker and the model worker from an image of the evaluation worker captured by the camera device 2 and an image of the model worker, and determines whether correction of the learned skeletal data 131 by the skeletal data correction unit 104 is necessary based on the height difference. For example, the correction necessity determination unit 102 calculates the height difference from the skeletal coordinates of the evaluation worker in a specific posture among the skeletal coordinates included in the detected skeletal data 130 and the skeletal coordinates of the evaluation worker in a specific posture among the skeletal coordinates included in the learned skeletal data 131.

[0030] If the height difference is equal to or greater than the threshold, the correction necessity determination unit 102 determines that correction is necessary, and if the height difference is less than the threshold, determines that correction of the learned skeletal data 131 is unnecessary. If the correction necessity determination unit 102 determines that correction is necessary, the skeletal data correction unit 104 corrects the skeletal coordinates of the learned skeletal data 131, and if the correction necessity determination unit 102 determines that correction is unnecessary, the skeletal data correction unit 104 does not correct the skeletal coordinates of the learned skeletal data 131.

[0031] Therefore, the skeleton data correction unit 104 can make an appropriate correction according to the height difference between the evaluation worker and the model worker. The correction necessity determination unit 102 may calculate the difference in shoulder width between the evaluation worker and the model worker in addition to or instead of the height difference, and determine that correction of the learning skeleton data 131 is unnecessary if the difference in shoulder width is less than a threshold value. In this case, the skeleton data correction unit 104 can make an appropriate correction according to the difference in shoulder width between the evaluation worker and the model worker.

[0032] (Correction example) Fig. 2 is a diagram showing an example of correction of skeletal coordinates. Fig. 2 shows an image G1a of a model worker Ha performing an operation of assembling a part C to a workpiece W, and an image G2a of an evaluation worker Hb performing the same operation as the model worker Ha. The image G1a is captured when the learning model 133 learns the skeletal coordinates, and the image G2a is captured when the operation evaluation unit 105 evaluates the operation.

[0033] The images G1a and G2a are captured by a camera device 2 installed in front of the model worker Ha and the evaluation worker Hb in the workplace. In the workplace, for example, a workbench 80 on which work is performed, a parts box 81, and a transport conveyor 83 are installed. For example, the model worker Ha and the evaluation worker Hb take out a part C from the parts box 81 on the right hand side and assemble it on a workpiece W on the workbench 80. The workpiece W is transported by the transport conveyor 83.

[0034] The skeleton detection unit 101 detects the coordinates of the skeletons of the model worker Ha and the evaluation worker Hb from the images G1a and G2a by image analysis using a general-purpose AI. Examples of the skeleton coordinates include the coordinates of the nose position P0, the left shoulder position P23, the right shoulder position P13, the left elbow position P22, the right elbow position P12, the left hand position P21, the right hand position P11, the left hip position P32, and the right hip position P31 of the model worker Ha and the evaluation worker Hb. In the images G1a and G2a, the horizontal direction is the X direction, and the vertical direction is the Y direction.

[0035] Before evaluation, the task evaluation unit 105 has the learning model 133 learn the skeletal coordinates in the image G1a to generate learned skeleton data 131. During evaluation, the task evaluation unit 105 evaluates the task based on the skeletal coordinates in the image G2a using the learning model 133 that has already learned the skeletal coordinates of the image G1a. At this time, the skeletal coordinates in the image G1a and the skeletal coordinates in the image G2a are compared with each other, and an evaluation result according to the comparison result is obtained.

[0036] However, due to differences in physique between the model worker Ha and the evaluation worker Hb, their skeletal coordinates differ even when they are performing the same assembly work at the same position. The height of the evaluation worker Hb is shorter than that of the model worker Ha. For this reason, even if the evaluation worker Hb performs the work correctly, the work evaluation unit 105 may output an evaluation result of "Fail" due to the difference in skeletal coordinates.

[0037] Therefore, the skeleton data correction unit 104 corrects the skeleton coordinates of the model worker Ha as indicated by the dotted arrows. The white circles indicate the corrected skeleton positions P0, P11-P13, P21-P23, P31, and P32. The corrected skeleton coordinates of the model worker Ha are close to the skeleton coordinates in the image G2a. Therefore, when the evaluation worker Hb performs the work correctly, the risk of the work evaluation unit 105 outputting an evaluation result of "fail" is reduced.

[0038] The skeleton data correcting unit 104 does not necessarily need to correct the coordinates of all skeletons. In the assembly work of this example, the model worker Ha and the evaluation worker Hb hold down the center of the left edge of the workpiece W with their left hand. The holding position on the workpiece W is substantially constant regardless of the physique of the model worker Ha and the evaluation worker Hb. Therefore, the skeleton data correcting unit 104 does not need to correct the position P21 of the left hand.

[0039] Also, unlike this example, when the model worker Ha and the evaluation worker Hb perform the task of picking up the part C from the parts box 81, the position P11 of the right hand is substantially constant regardless of their respective physiques. Therefore, in the evaluation of the task of picking up the part C, the skeletal data correction unit 104 does not need to correct the position P11 of the right hand.

[0040] Therefore, the correction value calculation unit 103 can set different correction values ​​for each skeleton depending on the type of work performed by the evaluation worker Hb. This allows the skeleton coordinates to be more appropriately corrected depending on the type of work, allowing the work evaluation unit 105 to evaluate the work with higher accuracy.

[0041] (Example of calculation of correction value) Fig. 3 is a diagram showing an example of calculation of a correction value. Fig. 3 shows images G1b, G2b of a model worker Ha and an evaluation worker Hb taking a predetermined basic posture in front of a workbench 80. In Fig. 3, the same components as those in Fig. 2 are given the same reference numerals and their explanations are omitted.

[0042] The correction value calculation unit 103 calculates a correction value from the model worker Ha in the basic posture and the evaluation worker Hb. The camera device 2 captures an image G1b of the model worker Ha in the basic posture when learning the learning model 133. At this time, for example, the task evaluation unit 105 stores the coordinates of the skeleton detected from the image G2a by the skeleton detection unit 101 as part of the learned skeleton data 131. Note that the coordinates of the skeleton of the model worker Ha in the basic posture are not limited to this, and may be stored in advance in the HDD 13 as data separate from the learned skeleton data 131.

[0043] The camera device 2 also captures an image G2b of the evaluation worker Hb in a basic posture before the evaluation. At this time, the correction value calculation unit 103 calculates a correction value by comparing the coordinates of the skeleton detected from the image G2b by the skeleton detection unit 101 with the coordinates of the skeleton of the model worker Ha in the basic posture. As an example, the basic posture is a state in which the upper arms are stretched horizontally to the left and right and the forearms are stretched vertically upward and facing forward so that the difference in physique between the model worker Ha and the evaluation worker Hb is clear, but is not limited to this. The correction value calculation unit 103 calculates a correction value from the difference in skeletal coordinates due to the difference in physique between the model worker Ha and the evaluation worker Hb.

[0044] When calculating the correction value, the correction value calculation unit 103 determines the correction reference points Pa and Pb at the center of the straight lines Na and Nb that connect the shoulders of the model worker Ha and the evaluation worker Hb in the basic posture. Here, if the coordinates of the reference point Pa are (Xa, Ya) and the coordinates of the reference point Pb are (Xb, Yb), then Xa=Xb and Ya>Yb are established. The correction value calculation unit 103 calculates the distances Da and Db from the measurement point Po set at the corner of the parts box 81 to the reference points Pa and Pb, for example. The correction value calculation unit 103 calculates the ratio (Db / Da) of Db to the distance Da as the correction value of the X coordinate. The correction value calculation unit 103 also calculates the difference ΔY (=Ya-Yb) between the Y coordinates of the reference points Pa and Pb as the correction value of the Y coordinate.

[0045] The skeleton data correction unit 104 corrects the coordinates of the right shoulder, left shoulder, right hip, and left hip (P13, P23, P31, P32) using the above correction values. An example of correction of the left shoulder coordinate will be described below.

[0046] For example, when the coordinates of the measurement point Po are (0,0), the coordinates of the reference point Pa are (100,110), and the coordinates of the reference point Pb are (100,100). The coordinates of the left shoulder of the model worker Ha are (154,110). At this time, the distance Da between the measurement point Po and the reference point Pa is 148.7, and the distance Db between the measurement point Po and the reference point Pb is 141.4, so the correction value of the X coordinate is calculated as 0.95 (=141.4 / 148.7). The correction value of the Y coordinate is calculated as 10 (=110-100).

[0047] The skeleton data correction unit 104 corrects the coordinate (154, 110) of the left shoulder of the model worker Ha using each correction value. The X coordinate of the left shoulder after correction is calculated as 146 (=154×0.95) by multiplying the X coordinate (154) before correction by the correction value of the X coordinate. The Y coordinate of the left shoulder after correction is calculated as 100 (=110−10) by subtracting the Y coordinate correction value from the Y coordinate (110) before correction. Therefore, the coordinate of the left shoulder after correction is (146, 100). This coordinate substantially matches the coordinate of the left shoulder in the image G2b. The coordinates of the right shoulder, right hip, and left hip are corrected in the same way as the left shoulder. The correction value calculation unit 103 is not limited to this, and may calculate the correction value from the difference in relative positions between the coordinates of the left shoulder, right shoulder, right hip, and left hip in each image G1b, G2b.

[0048] Furthermore, the skeletal data correction unit 104 corrects the coordinates of the left elbow, right elbow, left hand, and right hand (P22, P12, P21, P11) using, for example, the upper arm length U and forearm length L of the image G2b. For example, the correction value calculation unit 103 calculates the distance between the coordinates of the left shoulder and left elbow in image G2b as the upper arm length U of the evaluation worker Hb, and calculates the distance between the coordinates of the left elbow and left hand in image G2b as the forearm length L of the evaluation worker Hb.

[0049] The correction value calculation unit 103 calculates the upper arm length U as a correction value for the coordinates of the left elbow and right elbow, and calculates the forearm length L as a correction value for the coordinates of the left hand and right hand. The skeleton data correction unit 104 corrects the coordinate of the left elbow of the learning skeleton data 131 from the corrected coordinate of the left shoulder, the length of the upper arm U, and the angle of the left upper arm with respect to the vertical direction of the learning skeleton data 131. Similarly, the coordinate of the right elbow is corrected from the corrected coordinate of the right shoulder, the length of the upper arm U, and the angle of the right upper arm with respect to the vertical direction.

[0050] Furthermore, the skeleton data correction unit 104 corrects the coordinates of the left hand in the learning skeleton data 131 from the corrected coordinates of the left elbow, the length L of the forearm, and the angle of the left forearm with respect to the vertical direction in the learning skeleton data 131. Similarly, the coordinates of the right hand are corrected from the corrected coordinates of the right elbow, the length L of the forearm, and the angle of the right forearm with respect to the vertical direction. Note that the above correction method is an example and is not limited to this.

[0051] Further, the correction value calculation unit 103 sets the difference ΔY between the Y coordinates of the reference points Pa and Pb as the correction value of the coordinate of the nose. The skeleton data correction unit 104 corrects the Y coordinate of the nose by subtracting the difference ΔY from the Y coordinate of the nose in the learned skeleton data 131. Note that the X coordinate of the nose is not corrected.

[0052] In this way, the correction value calculation unit 103 calculates the correction value from the difference between the coordinates (Xa, Ya) of the reference point Pa at the center of the line Na connecting the shoulders of the model worker Ha in the image G1b and the coordinates (Xb, Yb) of the reference point Pb at the center of the line Nb connecting the shoulders of the evaluation worker Hb in the image G2b. Therefore, the skeleton data correction unit 104 can correct the skeleton coordinates with high accuracy regardless of the posture of the evaluation worker Hb during work. Note that, although the present example shows a case where the model worker Ha is taller than the evaluation worker Hb, even if the evaluation worker Hb is taller than the model worker Ha, it is possible to correct the learned skeleton coordinates by a method similar to the above method.

[0053] In addition, when the heights of the model worker Ha and the evaluation worker Hb are substantially the same but the shoulder widths are different, the skeletal coordinates of the learning skeletal data 131 can be corrected, for example, from the ratio of the distances in the X direction from the reference points Pa, Pb to the coordinates of each skeletal point.

[0054] The correction necessity determination unit 102 determines whether correction is necessary based on the physique difference between the model worker Ha and the evaluation worker Hb before calculating the correction value. The correction necessity determination unit 102 calculates the height difference between the model worker Ha and the evaluation worker Hb, and determines that correction is necessary if the height difference is equal to or greater than a threshold, and determines that correction is not necessary if the height difference is less than the threshold. For example, the correction necessity determination unit 102 uses the difference (|Ta-Tb|) between the Y coordinates Ta and Tb of the nose as the height for the determination.

[0055] The correction necessity determination unit 102 may also determine whether correction is necessary by comparing the difference (|Wa-Wb|) between the shoulder widths Wa and Wb of the model worker Ha and the evaluated worker Hb with a threshold. The shoulder width Wa of the model worker Ha is calculated from the distance between the coordinates of the right shoulder and the left shoulder of the learning skeleton data 131, and the shoulder width Wa of the evaluated worker Hb is calculated from the distance between the coordinates of the right shoulder and the left shoulder of the image G2b.

[0056] In this way, the correction necessity determination unit 102 determines whether correction of skeletal coordinates is necessary based on the difference in height and shoulder width between the model worker Ha and the evaluation worker Hb. Therefore, if there is substantially no difference in physique between the model worker Ha and the evaluation worker Hb, correction can be omitted. Note that the correction necessity determination unit 102 may determine whether correction is necessary based on only one of the difference in height and shoulder width between the model worker Ha and the evaluation worker Hb.

[0057] (Correction judgment process) 4 is a flowchart showing an example of the correction judgment process. This process is executed before the task evaluation process. Note that the skeleton coordinates of the model worker Ha in the basic posture as shown in the image G1b are included in the learning skeleton data 131 in advance.

[0058] First, the skeleton detection unit 101 acquires an image of the evaluation worker from the camera device 2 via the LAN 90 (step St1). Next, the skeleton detection unit 101 detects the coordinates of the evaluation worker's skeleton (step St2). Next, the work evaluation unit 105 judges whether the evaluation worker is in a basic posture by comparing the coordinates of the skeleton with a predetermined pattern (step St3). If the evaluation worker is not in a basic posture (No in step St3), the processing from step St1 onwards is performed again.

[0059] If the evaluation worker is in the basic posture (Yes in step St3), the correction necessity determination unit 102 reads out the skeletal coordinates of the model worker in the basic posture from the learning skeletal data 131 (step St4). Next, the correction necessity determination unit 102 calculates the height difference and shoulder width difference between the model worker in the basic posture and the evaluation worker from the skeletal coordinates of each of them (step St5). The correction necessity determination unit 102 compares the height difference with a threshold value THa (step St6). For example, the threshold value THa is determined from the relationship between the height difference and the desired evaluation accuracy.

[0060] If the height difference is less than the threshold THa (No in step St6), the correction necessity determination unit 102 compares the shoulder width difference with a threshold THb (step St9). For example, the threshold THb is determined based on the relationship between the shoulder width difference and the desired evaluation accuracy. If the shoulder width difference is less than the threshold THb (No in step St9), the correction necessity determination unit 102 determines that correction processing is unnecessary (step St10).

[0061] If the height difference is equal to or greater than the threshold value THa (Yes in step St6), or if the shoulder width difference is equal to or greater than the threshold value THb (Yes in step St9), the correction necessity determination unit 102 determines that correction processing is necessary (step St7). Next, the correction value calculation unit 103 calculates a correction value from the coordinates of the skeletons of the model worker and the evaluation worker in the basic posture (step St8). In this manner, the correction determination processing is performed.

[0062] (Evaluation process) 5 is a flow chart showing an example of the work evaluation process. This process is executed after the correction determination process. First, when the correction necessity determination unit 102 determines that the correction process is necessary (Yes in step St21), the skeleton data correction unit 104 reads out the skeleton coordinates of the model worker working in the learning skeleton data 131 (step St22). Next, the skeleton data correction unit 104 corrects the skeleton coordinates of the learning skeleton data 131 with the correction value (step St23). Also, when it is determined that the correction process is unnecessary (No in step St21), steps St22 and St23 are not executed.

[0063] Next, the skeleton detection unit 101 acquires an image of the evaluator from the camera device 2 via the LAN 90 (step St24). Next, the skeleton detection unit 101 detects the coordinates of the skeleton of the evaluator (step St25). Next, the skeleton detection unit 101 stores the coordinates of the skeleton in chronological order as detected skeleton data 130 (step St26).

[0064] Next, the task evaluation unit 105 compares the skeleton coordinates of the corrected learning skeleton data 131 with the skeleton coordinates of the detected skeleton data 130 (step St27). Next, the task evaluation unit 105 uses the learning model 133 to determine from the comparison result of the skeleton coordinates whether or not the movement of the model worker according to the learning skeleton data 131 and the movement of the evaluation worker according to the detected skeleton data 130 are similar to each other (step St28).

[0065] If the motion of the model worker and the motion of the evaluation worker are similar to each other (Yes in step St28), the task evaluation unit 105 evaluates the task as "good" (step St29), and if the motion of the model worker and the motion of the evaluation worker are not similar to each other (No in step St28), the task evaluation unit 105 evaluates the task as "unacceptable" (step St31). Next, the task evaluation unit 105 saves the evaluation result as evaluation data 132 (step St30). The evaluation process is performed in this manner.

[0066] (Other Examples) The skeleton data correction unit 104 may correct the skeleton coordinates of the detected skeleton data 130 with the correction value, instead of the skeleton coordinates of the learned skeleton data 131.

[0067] Fig. 6 is a flow chart showing another example of the task evaluation process. In Fig. 6, the same reference numerals are used for the processes common to Fig. 5, and the description thereof will be omitted. After detecting and storing the coordinates of the skeleton of the evaluating worker (St24 to St26), if the correction necessity determination unit 102 determines that correction processing is necessary (Yes in step St26a), the skeleton data correction unit 104 corrects the skeleton coordinates of the detected skeleton data 130 with a correction value (step St26b), and if the correction necessity determination unit 102 determines that correction processing is unnecessary (No in step St26a), correction is not performed. Thereafter, the task evaluation unit 105 evaluates the task as described above (St27 to St31).

[0068] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this embodiment and can be modified in various ways without departing from the scope of the present invention. [Explanation of symbols]

[0069] 1 Work evaluation server (work evaluation device), 10 CPU, 101 skeleton detection unit (detection unit), 102 correction necessity determination unit (determination unit), 103 correction value calculation unit (calculation unit), 104 skeleton data correction unit (correction unit), 105 work evaluation unit (evaluation unit), 130 detected skeleton data, 131 learning skeleton data, 133 learning model

Claims

1. a detection unit that detects coordinates of a skeleton of a first worker in a first image captured by an imaging device and coordinates of a skeleton of a second worker in a second image captured by the imaging device; an evaluation unit that uses a learning model that has been machine-learned on a time series basis of skeletal coordinates of the first worker while performing the work in the first image, and evaluates the work of the second worker based on a time change in skeletal coordinates of the second worker while performing the work in the second image; a correction unit that corrects, with a correction value, the skeletal coordinates of the first worker learned by the learning model or the skeletal coordinates of the second worker detected by the detection unit; a calculation unit that calculates the correction value from a difference in coordinates of a skeleton due to a difference in physique between the first worker in the first image and the second worker in the second image, Work evaluation device.

2. the calculation unit calculates the correction value from a difference between a center coordinate of a line connecting both shoulders of the first worker in the first image and a center coordinate of a line connecting both shoulders of the second worker in the second image. The task evaluation device according to claim 1 .

3. The calculation unit varies the correction value for each skeleton according to a type of work performed by the first worker. The task evaluation device according to claim 1 or 2.

4. a determination unit that calculates a height difference between the first worker and the second worker from the first image and the second image, and determines whether or not correction by the correction unit is necessary based on the height difference; The correction unit does not correct the coordinates of the skeleton when the determination unit determines that correction is unnecessary. The task evaluation device according to claim 1 or 2.

5. a determination unit that calculates a difference in shoulder width between the first worker and the second worker from the first image and the second image, and determines whether or not correction by the correction unit is necessary based on the difference in shoulder width; The correction unit does not correct the coordinates of the skeleton when the determination unit determines that correction is unnecessary. The task evaluation device according to claim 1 or 2.

Images