Work element analysis device and work element analysis method

Abstract

The following technology is known: in a production plant, the work elements constituting a series of work of an operator are inferred and the omission of work or the work time is analyzed. At this time, when the work elements are inferred using only the position information of the operator performing the work, sometimes the work elements different from the actual work elements are inferred. According to the present invention, the occurrence probability of the work elements in the work is obtained in advance, and when the work elements are inferred from the work information, the occurrence probability is used for correction, so that the work elements with higher accuracy can be obtained and analyzed.

Description

Technical Field

[0001] The present invention relates to a work element analysis device and a work element analysis method for analyzing work elements of an operator. Background Art

[0002] In a production factory, in order to grasp the omission of work or the working time of each worker, there is a known technology that uses an image of the worker performing the work or the position information of the worker to analyze the work elements constituting the work. In Patent Document 1, the work elements of the worker are inferred based on the position information of the body parts of the worker or the object at a certain moment.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: International Publication No. 2018 / 087844 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] However, when work elements are estimated based only on position information of a body or an object acquired at a certain time as in Patent Document 1, work elements different from actual work elements may be estimated if, for example, noise appears in the position information.

[0008] The present invention is completed in order to solve the above-mentioned problems, and its purpose is to obtain a work element analysis device with better accuracy.

[0009] Means used to solve problems

[0010] The work element analysis device of the present invention comprises: a work information acquisition unit, which acquires work information generated as the operator performs the work; a work element inference unit, which infers the work elements constituting the work based on the work information acquired by the work information acquisition unit; a generation probability acquisition unit, which acquires the generation probability of the work elements in the work, and the generation probability is calculated using the total value of each of the work elements inferred by the work element inference unit plus an adjustment value; and a correction unit, which corrects the work elements inferred by the work element inference unit based on the generation probability acquired by the generation probability acquisition unit.

[0011] Effects of the Invention

[0012] According to the present invention, since the occurrence probability of the work element in the work is also taken into consideration to correct the work element estimated from the work information, it is possible to analyze the work element with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 1 is a structural diagram showing the structure of the work factor analysis device 200 according to the first embodiment.

[0014] Figure 2 This is an explanatory diagram showing an example in which the work element output probability calculation unit 241 of the work element analysis device 200 according to the first embodiment aggregates the work elements.

[0015] Figure 3 This is an explanatory diagram showing an example in which the work element output probability calculation unit 241 of the work element analysis device 200 according to the first embodiment adds the adjustment value to the total value.

[0016] Figure 4 This is an explanatory diagram showing an example in which the occurrence probability acquisition unit 240 of the work element analysis device 200 according to the first embodiment calculates the work element occurrence probability.

[0017] Figure 5 This is an explanatory diagram showing an example in which the work element transition probability calculation unit 243 of the work element analysis device 200 according to the first embodiment aggregates the work elements.

[0018] Figure 6 This is an explanatory diagram showing an example in which the work element transition probability calculation unit 243 of the work element analysis device 200 according to the first embodiment adds an adjustment value.

[0019] Figure 7 This is an explanatory diagram showing an example in which the occurrence probability acquisition unit 240 of the work element analysis device 200 according to the first embodiment calculates the work element transition probability.

[0020] Figure 8 This is a flowchart showing the operation of the work element analysis device 200 according to the first embodiment.

[0021] Fig. 9 This is a flowchart showing the operation of the work element analysis device 200 according to the second embodiment.

[0022] Fig.10 This is a flowchart of calculation of the likelihood of each node by the correction unit 250 of the work factor analysis device 200 according to the second embodiment.

[0023] Fig.11 This is a flowchart showing the operation of the work factor analysis device 200 according to the second embodiment.

[0024] Fig.12 This is a flowchart showing the operation of the correction unit 250 of the work factor analysis device 200 according to the second embodiment to calculate the likelihood.

[0025] Fig.13This is an explanatory diagram showing an example of a specific processing code when the correction unit 250 of the work factor analysis device 200 according to the second embodiment calculates likelihood.

[0026] Fig.14 This is a flowchart showing the operation of extracting the maximum likelihood work history by the correction unit 250 of the work factor analysis device 200 according to the second embodiment.

[0027] Fig.15 It is a hardware configuration diagram of the work factor analysis device 200 according to the present embodiment. DETAILED DESCRIPTION

[0028] Implementation Method 1

[0029] The following describes the work element analysis device according to Embodiment 1. The work element analysis device 200 according to Embodiment 1 analyzes a series of work performed by a worker at a manufacturing site such as a production plant, and detects "work elements" which are elements constituting the series of work.

[0030] For example, in a manufacturing site where an operation such as assembling an item is performed, when an operator performs such actions as arranging components, tightening screws, and confirming whether the item is fixed, "assembly" is an "operation", and "component arrangement", "screw tightening", and "confirmation" are "operation elements". Here, although the actions such as "component arrangement", "screw tightening", and "confirmation" are set as operation elements, these actions can also be further subdivided, and the user can decide which set of actions to define as operation elements according to the summary method considered by the user. In this embodiment, the operation elements are divided into 4, and the operation element numbers "0", "1", "2", and "3" are marked as marks to distinguish the divided operation elements for explanation.

[0031] Figure 1 1 is a block diagram showing the structure of the work element analysis device 200 of the first embodiment. The work element analysis device 200 analyzes the work elements performed by the operator based on the signals from the camera 101, the microphone 102, and the production equipment 103, and outputs the analysis results to the analysis result storage unit 300. The work element analysis device 200 of the present embodiment is configured to include: a work information acquisition unit 210 that acquires work information generated as the operator performs the work; a work element inference unit 220 that infers the work elements constituting the work based on the work information acquired by the work information acquisition unit 210; a generation probability acquisition unit 240 that acquires the generation probability of the work elements in the work; and a correction unit 250 that corrects the work elements inferred by the work element inference unit 220 based on the generation probability acquired by the generation probability acquisition unit 240.

[0032] In the work element analysis device 200 of such a structure, first, the work element inference unit 220 infers the work elements constituting the work based on the work information acquired by the work information acquisition unit 210. There are many inference methods. The work element analysis device 200 of this embodiment is provided with a generation probability acquisition unit 240 and a correction unit 250, so that when the work element inference unit 220 infers a work element number different from the appropriate work element number due to noise in the work information acquired by the work information acquisition unit 210 or the operator accidentally performs the same action as other work elements, the error is corrected, and the work element analysis with higher accuracy can be performed. In the work element analysis device 200, the generation probability acquisition unit 240 obtains the generation probability of each work element that may be generated in the work, and based on the generation probability, the correction unit 250 corrects the work element inferred by the work element inference unit 220. As a result, for example, when a work element with a low probability of being generated in a certain work is inferred, the possibility that the inference is an erroneous inference is high, so it is corrected, and the work element analysis with higher accuracy can be performed.

[0033] The structure of each part will be described in detail below.

[0034] The camera 101 captures the operator and his surroundings during the operation, converts the captured images into image signals, and outputs the captured images. The microphone 102 is installed at the work site where the operator is working or at the operator, and converts the sounds generated by the operator during the operation or the operator's voice into electrical signals, and outputs the audio signals. The production equipment 103 is, for example, an electric actuator or other production equipment used by the operator during the operation, and outputs status signals indicating the driving, stopping, torque, etc. of the electric motor. The image signals, audio signals, and status signals outputted by the camera 101, the microphone 102, and the production equipment 103, respectively, are inputted into the operation information acquisition unit 210 of the operation factor analysis device 200.

[0035] The signal input to the operation information acquisition unit 210 may be one or more signals including information related to the operation performed by the operator, and may be not limited to the camera 101, the microphone 102, and the production equipment 103, and may be a signal including information detected by an acceleration sensor or a depth sensor, etc. In addition, the signal input to the operation information acquisition unit 210 may be a signal pre-stored in an external recording device such as a recorder.

[0036] In the operation information acquisition unit 210, the skeleton information extraction unit 211, the image feature extraction unit 212, the sound feature extraction unit 213 and the production equipment feature extraction unit 214 are used to extract the operation information generated as the operator performs the operation from the signals input from the camera 101, the microphone 102 and the production equipment 103, and the operation information integration unit 215 integrates the extracted operation information for each frame and outputs it to the operation element inference unit.

[0037] The image signal input from the camera 101 to the work information acquisition unit 210 of the work element analysis device 200 is input to the skeleton information extraction unit 211 and the image feature quantity extraction unit 212 of the work information acquisition unit 210 .

[0038] In the skeleton information extraction unit 211, the image signal obtained from the camera 101 is processed to extract the skeleton information of the operator as the work information, for example, for each frame of the image. Here, the skeleton information is information representing the coordinates of the representative joint positions of the human body such as the shoulder, elbow, and wrist of the operator in the image coordinate system, and is information representing the working posture of the operator. As a method for extracting the skeleton information of the operator from the image, in addition to the method of extracting the joint position from the marker position installed on the operator, there is also a method of extracting the joint position using a pre-learned Convolutional-Neural-Network (CNN). The skeleton information extracted by the skeleton information extraction unit 211 is input to the work information integration unit 215.

[0039] The image feature extraction unit 212 extracts image-related feature quantities such as Histograms-of-Oriented-Gradients (HOG) feature quantities for each frame from the image signal obtained from the camera 101. The image feature quantity information extracted by the image feature extraction unit 212 is input to the work information integration unit 215.

[0040] The sound signal input from the microphone 102 to the work information acquisition unit 210 of the work element analysis device 200 is input to the sound feature quantity extraction unit 213 of the work information acquisition unit 210 .

[0041] The sound feature extraction unit 213 extracts sound features as work information by, for example, performing spectrum analysis on the sound of the sound signal acquired from the microphone 102 . The sound feature information extracted by the sound feature extraction unit 213 is input to the work information integration unit 215 .

[0042] The state signal input from the production equipment 103 to the operation information acquisition unit 210 of the operation factor analysis device 200 is input to the production equipment feature quantity extraction unit 214 of the operation information acquisition unit 210 .

[0043] The production equipment feature extraction unit 214 processes the state signal obtained from the production equipment 103, for example, extracting information on the operating state generated when the operator turns on the switch of the electric actuator, information on the label read by the barcode reader, etc. as the feature of the production equipment. The production equipment feature information extracted by the production equipment feature extraction unit 214 is input to the operation information integration unit 215.

[0044] The operation information integration unit 215 integrates the skeleton information, image feature information, sound feature information, and production equipment feature information as operation information with the time information into one operation information for each frame at the same time. The operation information integrated by the operation information integration unit 215 is input to the operation element inference unit 220.

[0045] The work element inference unit 220 includes a learned model generation unit 221 as a learning unit, a learned model storage unit 222, and an inference unit 223. In the work element inference unit 220, the learned model generated by the learned model generation unit 221 is pre-stored in the learned model storage unit 222, and the work elements constituting the work are inferred for each frame based on the work information acquired by the work information acquisition unit 210 and using the learned model in the learned model storage unit 222.

[0046] In addition, there have always been various methods for the work element inference unit 220 to infer work elements, which are not limited to the above-mentioned methods. The work elements can be inferred based on statistical data, or the user can set set values ​​for information such as feature quantities of the work information to infer the work elements.

[0047] The work element teaching unit 230 obtains teaching data as information of real work elements corresponding to work information through user input, and outputs the teaching data to the learning model generating unit 221 of the work element inferring unit 220. The real work element refers to the work element that the work element analyzing device 200 should obtain based on the work information, and the teaching data is obtained by, for example, the user determining the work element number for the past work information through the work element teaching unit 230.

[0048] Furthermore, the work element teaching unit 230 also outputs teaching data to the work element output probability calculation unit 241 and the work element transition probability calculation unit 243 of the generation probability acquisition unit 240 .

[0049] The learned model generation unit 221 generates a learned model that performs machine learning on the correlation between the work elements and the work information in order to infer the work elements constituting the process of the work from the work information, based on the work information input from the work information acquisition unit 210 and the teaching data input from the work element teaching unit 230, and outputs the learned model to the learned model storage unit 222. By using the learned model, the inference unit infers the work element number for each frame of work information.

[0050] In addition, although the learned model is generated by the learned model generation unit 221, it is not necessary to obtain the work information from the work information acquisition unit 210 to generate the learned model, and it can also be directly input to the learned model storage unit 222. In this case, the learned model generation unit 221 outputs the learned model input to the work element teaching unit 230 directly to the learned model storage unit 222.

[0051] Furthermore, the learned model may be generated and stored outside the work factor analysis device 200, and the learned model may be acquired by using a network or the like when the inference unit 223 performs inference.

[0052] The learned model storage unit 222 stores the learned model output from the learned model generation unit 221. By storing the learned model in the learned model storage unit 222 in advance, the inference unit 223 can use the learned model to infer the work element based on the work information. The learned model storage unit 222 outputs the learned model to the inference unit 223.

[0053] After receiving the operation information acquired by the operation information acquisition unit 210, the inference unit 223 infers the operation element by using the learned model output from the learned model storage unit 222. The operation element infers the probability value of each operation element for the operation information of one frame, and outputs the operation element with the highest probability as the operation element inference information to the generation probability acquisition unit 240 and the correction unit 250.

[0054] The generation probability acquisition unit 240 includes a work element output probability calculation unit 241, a work element output probability information storage unit 242, a work element transition probability calculation unit 243, and a work element transition probability information storage unit 244. In the generation probability acquisition unit 240, the work element output probability and the work element transition probability, which are the generation probabilities of the work elements in the work, are acquired and output to the correction unit 250. In the present embodiment, the generation probability of the work element in the work is calculated based on the real work element for the past work information, but it is not limited to this as long as it is the generation probability related to the work element.

[0055] The operation element output probability calculation unit 241 calculates the operation element output probability, which is calculated based on the statistics of the real operation elements for the past operation information and the operation elements inferred by the operation element inference unit 220 based on the same past operation information. The information of the real operation elements for the past operation information is obtained based on the teaching data output from the operation element teaching unit 230. When calculating the operation element output probability, the operation element output probability calculation unit 241 adds an adjustment value to the number of the operation element so that the operation element output probability does not become 0. The specific adjustment method is described later. By adjusting the operation element generation probability in this way, it is possible to take into account the following situation. The case is that the operation element that does not occur in the past operation information used when calculating the operation element output probability is inferred based on the abnormality of the operation element inference unit 220. The operation element output probability calculation unit 241 stores the information of the operation element output probability in the operation element output probability information storage unit 242. The operation element output probability information storage unit 242 outputs the information of the operation element output probability to the correction unit 250.

[0056] The operation element transition probability calculation unit 243 calculates the operation element transition probability, which is calculated based on the statistics of the past transition from the operation element number to the next operation element number. The information of the past transition from the operation element number to the next operation element number is obtained based on the teaching data output from the operation element teaching unit 230. When calculating the operation element transition probability, the operation element transition probability calculation unit 243 adjusts the probability of the operation element by adding an adjustment value to the probability of the operation element so that the operation element transition probability does not become 0. The specific adjustment method is described later. By adjusting the operation element transition probability in this way, it is possible to consider the following situation for correction, that is, the transition of the operation element that did not transition in the past operation elements used when calculating the operation element transition probability occurs. The operation element transition probability calculation unit 243 stores the information of the operation element transition probability in the operation element transition probability information storage unit 244. The operation element transition probability information storage unit 244 outputs the information of the operation element transition probability to the correction unit 250.

[0057] In addition, the time when the learning completion model generation unit 221 of the work element inference unit 220 generates the work information when the learning completion model is generated can also be different from the time when the work element output probability calculation unit 241 of the generation probability acquisition unit 240 calculates the probability and the time when the work element transition probability calculation unit 243 calculates the probability of past work element inference information.

[0058] In addition, for example, when the user understands the trend related to the work element and corrects the generation probability obtained from the statistical result, the generation probability obtained from the statistical result can be corrected. In this case, in this embodiment, the work element output probability information or the work element transition probability information stored in the work element output probability information storage unit 242 and the work element transition probability information storage unit 244 is rewritten.

[0059] After obtaining the operation element inference information from the operation element inference unit 220, the correction unit 250 obtains the operation element output probability information and the operation element transition probability information from the operation element output probability information storage unit 242 and the operation element transition probability information storage unit 244. Then, based on the operation element output probability and the operation element transition probability obtained as the generation probability by the generation probability acquisition unit 240, the operation element obtained from the operation element inference unit 220 is corrected using the Viterbi algorithm, and the corrected operation element correction information is output to the analysis unit 260.

[0060] The correction unit 250 can consider the performance of the work element estimated by the work element estimation unit 220 by using the work element output probability for correction, and can consider the probability of the work element changing when the work information advances by one frame by using the work element transition probability for correction. By correcting the work element using the generation rate of the work element estimated by the work element estimation unit 220 in this way, it is possible to obtain and analyze work elements with higher accuracy.

[0061] In the present embodiment, two probabilities are acquired by the occurrence probability acquisition unit 240 , but the probability acquired by the occurrence probability acquisition unit 240 may be one, and in this case, the correction is performed by the correction unit 250 based on the one probability.

[0062] The analysis unit 260 includes the continuous intervals of the same operation element as the time required to perform the operation element for the operation element correction information output from the correction unit 250, and analyzes the time consumed for one cycle of the repeated operation. At this time, in addition to the analysis of detecting the omission of the operation or the misorder of the operation by evaluating the generation order of the operation elements, the analysis is also performed based on the upper limit value of the operation element time to extract abnormal information, and the analysis result is output to the analysis result storage unit 300. The user refers to the analysis result storage unit 300 to understand the operation time of each operator or the occurrence of operation errors.

[0063] In addition, the analysis unit 260 and the work element teaching unit 230 are not necessarily included in the work element analysis device 200. The work element analysis device 200 only needs to infer and correct the work element based on the work information.

[0064] Next, the operation of the work factor analysis device 200 according to the first embodiment will be described.

[0065] Hereinafter, an example will be described in which a work process of one cycle of a repetitive work is composed of four work elements. Each work element is assigned a work element number of 0 to 3.

[0066] First, a learned model that has been machine-learned on the association between work elements and work information is stored in the learned model storage unit 222 in advance, so that the work element inference unit 220 can infer the work elements based on the work information obtained from the work information acquisition unit 210. The learned model is generated and output by the learned model generation unit 221 based on the past work information input from the work information acquisition unit 210 to the learned model generation unit 221 and the information of the real work elements for the same past work information input from the work element teaching unit 230, and is stored in the learned model storage unit 222.

[0067] In addition, the corrected work element output probability and work element transition probability are stored in advance in the work element output probability information storage unit 242 and the work element transition probability information storage unit 244 of the generation probability acquisition unit 240, so that the correction unit 250 can correct the work element inferred by the work element inference unit 220 based on the generation probability. The calculation method of the work element output probability and the work element transition probability is described below.

[0068] Operation element output probability b ik The work element output probability calculation unit 241 calculates the probability as follows and stores it in the work element output probability information storage unit 242 .

[0069] First, the work element information inferred by the work element inference unit 220 based on the past work information and the real work element information for the past work information input by the user to the work element teaching unit 230 are input to the work element output probability calculation unit 241. Then, the number of work element numbers k inferred by the work element inference unit 220 for the real work element numbers i in the work information of the same frame is summed up (hereinafter, the sum value c b_ik ). Figure 2The diagram is an explanatory diagram showing an example of totaling work elements by the work element output probability calculation unit 241 of the work element analysis device 200 as Embodiment 1. In the example shown here, for example, when the actual work element number is 0, the total value inferred as work element number 0 is 43, the total value inferred as work element number 1 is 8, the total value inferred as work element number 2 is 3, and the total value inferred as work element number 3 is 6, and the number of work elements inferred as not being actual work elements when inference is made only based on work information is 17.

[0070] Next, the work element output probability calculation unit 241 calculates the total value c b_ik As shown in the following formula 1, the adjustment value corresponding to the total number of work element number i and the total value c b_ik Add, and calculate the correction value c' b_ik .

[0071] [Formula 1]

[0072] c′ b_ik =c b_ik +w b ∑ m c b_im

[0073] Figure 3 1 is an explanatory diagram showing an example in which the work element output probability calculation unit 241 of the work element analysis device 200 according to the first embodiment adds the adjustment value to the total value. Figure 3 10% (W ) of the total value of the number of items estimated by the work element estimation unit 220 for the real work element number i b ) is added as an adjustment value. For example, the total value of the actual operation element number 0 is 60, so an adjustment value of 6, which is 10% of it, is added to each operation element number inferred by the operation element inference unit. By adding the adjustment value in this way, the operation element output probability b is prevented from being ik When the correction unit 250 makes a correction by setting the error value to 0, the correction unit 250 can make a correction by taking into account the case where an abnormal work element that does not occur in the work information used to calculate the work element output probability is estimated.

[0074] Finally, the work element output probability b of the work element number k in the work element estimation unit 220 in the real work element number i is calculated using the value adjusted by the following equation 2: ik .

[0075] [Formula 2]

[0076]

[0077] Figure 4 The operation factor analysis device 200 of the first embodiment calculates the operation factor output probability b by the generation probability acquisition unit 240. ik For example, the probability that the actual operation element number 0 is determined by the operation element inference unit to be the operation element number 0 is expressed by Figure 3 The corrected value 49 for the actual work element number 0 is divided by the corrected value 84 for the entire work element number 0 to obtain 58.3%. ik It is stored in the work element output probability information storage unit 242.

[0078] Operation element transition probability a ij The work element transition probability calculation unit 243 calculates the probability as follows and stores it in the work element transition probability information storage unit 244 .

[0079] First, the information of the real work elements of the consecutive frames input by the user from the work element teaching unit 230 is input to the work element transition probability calculation unit 243. The number of work element numbers of the next frame of the frame with the real work element number i is respectively counted (hereinafter referred to as the total value c a_ij ). Figure 5 The diagram is an explanatory diagram showing an example in which the work element transition probability calculation unit 243 of the work element analysis device 200 as Embodiment 1 aggregates the work elements. In the example shown here, for example, when the work element number of the transition source is 0, the aggregate value of the work element number 0 of the transition target becomes 78, the aggregate value of the work element number 1 becomes 3, the aggregate value of the work element number 2 becomes 0, and the aggregate value of the work element number 3 becomes 0, which is a state in which there is no data in the past in which the work element number is transitioned from 0 to 2 and 3.

[0080] Next, the work element transition probability calculation unit 243 calculates the total value c a_ij As shown in the following formula 3, the adjustment value corresponding to the total number of work element number i and the total value c a_ij Add, and calculate the correction value c' a_ij .

[0081] [Formula 3]

[0082]

[0083] Figure 6 1 is an explanatory diagram showing an example in which the work element transition probability calculation unit 243 of the work element analysis device 200 according to the first embodiment adds an adjustment value. Figure 61% (W) of the total value of the number of items estimated by the work element estimation unit 220 for the real work element number i a ) is added as an adjustment value. For example, Figure 5 In the example, the total value of the work element number 0 of the transition source is 81, so an adjustment value of 0.81 is added as 1% of it. By adding the adjustment value in this way, Figure 5 The total value c of 0 in the total table a_ij exist Figure 6 In this way, by preventing the operating element from changing probability a ij When the error value becomes 0, the correction unit 250 can make correction by taking into account the fact that an abnormal transition of the work element that does not occur in the work information used to calculate the work element transition probability occurs.

[0084] Finally, the work element transition probability a of the work element number j that is transitioned from the real work element number i is calculated using the value adjusted by the following equation 4: ij .

[0085] [Formula 4]

[0086]

[0087] Figure 7 1 is an explanatory diagram showing an example in which the occurrence probability acquisition unit 240 of the work element analysis device 200 of the first embodiment calculates the work element transition probability. For example, the probability of the work element numbered 0 at the transition source to the work element numbered 0 at the transition destination is expressed as Figure 6 The corrected value of 78.81 for the work element number 0 as the transition source is divided by the corrected value of 84.24 for the entire work element number 0 as the transition source to obtain 93.6%. ij It is stored in the work element transition probability information storage unit 244 .

[0088] In this way, the work element analysis device 200 is used in a state where the learned model, work element output probability, and work element transition probability are stored in the learned model storage unit 222, the work element output probability information storage unit 242, and the work element transition probability information storage unit 244.

[0089] Next, the operation of the work element analysis device 200 when analyzing the work elements will be described. Figure 8 This is a flowchart showing the operation of the work element analysis device 200 according to the first embodiment.

[0090] In step S001, the operation information acquisition unit 210 acquires operation information. Specifically, when information is input from the camera 101, the microphone 102, and the production equipment 103, the operation information generated by the operation performed by the operator, namely, the skeleton information, the image feature information, the sound feature information, and the production equipment feature information, is extracted by the skeleton information extraction unit 211, the image feature extraction unit 212, the sound feature extraction unit 213, and the production equipment feature extraction unit 214. Then, the operation information integration unit 215 integrates these operation information into one frame at each moment, and outputs the integrated operation information to the operation element inference unit 220.

[0091] In step S002, the work element inference unit 220 infers the work elements constituting the work based on the work information output from the work information acquisition unit 210. Specifically, when the work information is input from the work information acquisition unit 210 to the inference unit 223 of the work element inference unit 220, the inference unit 223 infers the probability value of each work element for the work information of one frame by using the learned model storage unit 222, and outputs the work element with the highest probability as the work element inference information to the correction unit 250 in a time series order.

[0092] In step S003, the correction unit 250 corrects the work element inferred by the work element inference unit 220 based on the generation probability acquired by the generation probability acquisition unit 240. Specifically, when the correction unit 250 acquires the work element inference information from the work element inference unit 220, the correction unit 250 uses the work element output probability stored in the work element output probability information storage unit 242 of the generation probability acquisition unit 240 and the work element transition probability stored in the work element transition probability information storage unit 244, and applies the Viterbi algorithm to correct the work element inferred by the work element inference unit 220. The corrected work element is output to the analysis unit 260.

[0093] Here, the effect when the generation probability acquisition unit 240 performs adjustment by adding an adjustment value to the generation probability of a work element having a generation probability value of 0 is described using a specific example of the work element transition probability.

[0094] For example, suppose that a worker performs a job in 15 frames, with the job element numbers of each frame being 000002222223333 respectively. The job element inference unit 220 makes an inference based on the job information corresponding to these jobs, and outputs 000002222223133 as the job element number of the 15 frames as an inference result. In this case, at the timing of the 13th frame, noise appears in the job information or the worker accidentally performs the same action as job element number 1. Therefore, the job element number of the 13th frame is inferred to be 1 by the inference unit 223. Such an inference error is corrected in the correction unit 250 based on the job element transition probability. Here, it is assumed that in the case of Figure 5 When the total value of the transition of the operation element number from 0 to 2 remains unchanged at 0 without adjustment as shown, the correction unit 250 does not allow the transition from the operation element number 0 to 2, and for example, the operation element number is corrected to 0000122222223333 so that the operation element number 1 is forced to appear at the boundary between the operation element numbers 0 and 2. When corrected in this way, the order of the operation element numbers is changed to 0, 1, 2, and 3, so the analysis unit 260 cannot perform the analysis for detecting the omission of the operation. By adjusting the generation probability of the operation element whose generation probability value is 0 by adding the adjustment value as in the present embodiment, the correction unit 250 can obtain a result of 0000022222223333 as the operation element number, and the analysis unit in the next step can perform the analysis for detecting the omission of the operation element number 1.

[0095] In step S004, the analysis unit 260 analyzes the corrected work element output from the correction unit 250. Specifically, the continuous interval of the same work element is counted as the time required to perform the work element, the time consumed by the work elements 1 to 4 as one cycle is analyzed, or the analysis is performed to detect work omissions based on the transition of the work elements. The analysis unit 260 outputs the analysis results to the analysis result storage unit 300.

[0096] As described above, the work element analysis device 200 of embodiment 1 corrects the work element inferred by the work element inference unit 220 based not only on the work information but also on the occurrence probability of the work element in the work, thereby being able to analyze the work element with better accuracy.

[0097] Implementation Method 2

[0098] Next, the work element analysis device 200 of Embodiment 2 is described. The difference from Embodiment 1 lies in the work element inference information output from the inference unit 223 of the work element inference unit 220 and the processing method of the correction unit 250, but the rest is the same as Embodiment 1.

[0099] When the work information acquired by the work information acquisition unit 210 is input, the inference unit 223 of the work element inference unit 220 obtains the probability value p of each work element for one frame of work information by using the learned model output from the learned model storage unit 222. t i (0.0≦p t i ≦1.0,Σ i p t i = 1.0, i = 0, 1, ..., N-1, N, N: total number of operation elements, t = 0, 1, ..., T-1, T, T: total number of frames). Then, the obtained probability value p is inferred. t i The probability value p with the highest probability among t i The work element inference unit 220 calculates the probability value p of each work element. t i The inferred work elements are output to the correction unit 250 as work element inference information.

[0100] The correction unit 250 calculates the probability of occurrence obtained by the generation probability obtaining unit 240 and the probability value p of each work element. t i , the work element estimated by the work element estimation unit 220 is corrected. Then, the work element with the highest probability is output to the analysis unit 260 as work element correction information.

[0101] Next, the correction unit 250 performs a correction based on the occurrence probability acquired by the occurrence probability acquisition unit 240 and the probability value p of each work element. t i The operation of the work element with the highest probability is described below. For the purpose of description, variables V, layer, node, and bestPath are defined according to the following formula 5.

[0102] [Formula 5]

[0103] V={layer t}(t=0, 1, ..., T-1), (T: total number of frames)

[0104] layer t ={node t ik}(i=0, 1, .., N-1, k=0, 1, .., N-1)

[0105] node t ik = {likelihood tik ,prvIdx t ik ,nxtIdx t ik}

[0106] bestPath={bestPath t}(t=0, 1, ..., T-1)

[0107] Here, prvIdx t ik Can store at most 1 indication layer t-1 The nodes contained in node t-1 IK A variable with index (I, K) (I≠i, K≠k, I=0, 1, ..., N-1, K=0, 1, ..., N-1).

[0108] nxtId t ik Can store at most 1 indication layer t+1 The nodes contained in node t+1 IK The index (I, K) of the variable.

[0109] likelihood t ik is the node node at index (i, k) at time t t ik The scalar quantity to which the default value is assigned is 0, which is the likelihood when the maximum likelihood operation history is tracked from time 0 to time t.

[0110] BestPath is a variable storing the operation history having the maximum likelihood from time 0 to time T-1, and is information finally output by the correction unit 250.

[0111] The operation of the correction unit 250 will be described based on the above-mentioned definitions of the variables. Fig. 9 This is a flowchart showing the operation of the work element analysis device 200 according to the second embodiment.

[0112] In step S101, the correction unit 250 calculates the likelihood of each node corresponding to the frame at time 0. 0 ik . Fig.10 The correction unit 250 of the work factor analysis device 200 according to the embodiment calculates the likelihood 0 ik The following formula 6 shows the calculation formula.

[0113] [Formula 6]

[0114] likelihood 0 ik =p t i ×b ik

[0115] In step S102, the correction unit 250 extracts the node t-1 IK The node of the best transformation target. Fig.11 This is a flowchart showing the operation of the work factor analysis device 200 according to Embodiment 2. The following equation 7 shows the extraction condition.

[0116] [Formula 7]

[0117]

[0118] In step S103, the correction unit 250 performs the correction on the node at time t={0, 1, ..., T-1}. t ik , extract the best transition source node and calculate the likelihood. Fig.12 This is a flowchart of the calculation of likelihood by the correction unit 250 of the work factor analysis device 200 according to the second embodiment. Fig.13 The specific processing code is shown.

[0119] In step S104, the maximum likelihood operation history bestPath={bestPath t}. Fig.14 This is a detailed flowchart of the process of extracting the maximum likelihood operation history by the correction unit 250 of the operation factor analysis device 200 according to the second embodiment. First, in step S301, the likelihood T-1 ik Become the largest node t ik , keep its index information (i tmp , k tmp ), to bestPath T-1 Register tmp Next, in step S302, the process of the following formula 8 is performed from t=T-1 to t=1, thereby probabilistically obtaining the most likely operation history bestPath.

[0120] [Formula 8]

[0121] (i, k) = (i tmp , k tmp )

[0122] (itmp , k tmp )=prxIdx t ik

[0123] bestPath t-1 =i tmp

[0124] As described above, the work element analysis device 200 of the second embodiment corrects the work element inferred by the work element inference unit 220 based on the occurrence probability of the work element in the work and the probability value of each work element, so that a higher-precision correction can be performed.

[0125] In the second embodiment, the probability value p of each work element is output from the work element estimation unit 220. t i and the work element inferred by the work element inference unit 220 as the work element inference information, but the work element inference information may be only the probability value p of each work element. t i In this case, the probability value p of each work element is calculated by the correction unit using the probability of occurrence of the work element in the work. t i Correction is performed and the work element with the highest probability is set as the work element correction information.

[0126] Fig.15 2 is a hardware configuration diagram of the work element analysis device 200 of the present embodiment. In the present embodiment, signals are input from the camera 101, the microphone 102, and the production equipment 103 via the first interface 201, the second interface 202, and the third interface 203 of the work element analysis device 200. The operation device 204 is a circuit such as a CPU or a DSP, and performs processing of the skeleton information extraction unit 211, the image feature extraction unit 212, the sound feature extraction unit 213, the production equipment feature extraction unit 214, the work information integration unit 215, the learning model generation unit 221, the inference unit 223, the work element teaching unit 230, the work element output probability calculation unit 241, the work element transition probability calculation unit 243, the correction unit 250, and the analysis unit 260. The main storage device 205 is a volatile memory that temporarily stores programs and data when the program is executed by the operation device 204. The auxiliary storage device is a non-volatile memory for long-term storage, and stores the learning model, work element output probability information, and work element transition probability information.

[0127] Description of Reference Numerals

[0128] 101. Camera

[0129] 102. Microphone

[0130] 103. Production equipment

[0131] 200. Operation factor analysis device

[0132] 201. Interface 1

[0133] 202. Interface 2

[0134] 203. Interface 3

[0135] 204.Computing device

[0136] 205.Main storage device

[0137] 206. Auxiliary storage device

[0138] 210. Operation information acquisition unit

[0139] 211. Skeleton Information Extraction Department

[0140] 212. Image feature extraction unit

[0141] 213. Sound feature extraction unit

[0142] 214.Production equipment feature extraction department

[0143] 215. Operation Information General Department

[0144] 220. Operational element inference department

[0145] 221. Learning completed model generation part

[0146] 222. Learning completed model storage unit

[0147] 223. Inference

[0148] 230. Operational elements teaching department

[0149] 240. Generation Probability Acquisition Unit

[0150] 241. Operation element output probability calculation unit

[0151] 242. Operation element output probability information storage unit

[0152] 243. Operation element transition probability calculation unit

[0153] 244. Operation element transition probability information storage unit

[0154] 250. Correction Department

[0155] 260.Analysis Department

[0156] 300.Analysis result storage unit

Claims

1. A work factor analysis device, characterized in that: The work element analysis device comprises: an operation information acquisition unit that acquires operation information generated as an operator performs an operation; a work element inferring unit for inferring work elements constituting the work based on the work information acquired by the work information acquiring unit; an occurrence probability acquisition unit that acquires an occurrence probability of the work element in the work, the occurrence probability being calculated using a value obtained by adding an adjustment value to the total value of each of the work elements estimated by the work element estimation unit; and A correction unit corrects the work element estimated by the work element estimation unit based on the occurrence probability acquired by the occurrence probability acquisition unit.

2. The work element analysis device according to claim 1, characterized in that: The work information acquired by the work information acquisition unit is skeleton information and voice feature value.

3. The work element analysis device according to claim 1, characterized in that: The operation element inference unit inputs the operation information acquired by the operation information acquisition unit into the learning unit, and infers the operation element by using the learning unit, wherein the learning unit performs machine learning on the correlation between the operation element and the operation information in order to infer the operation elements constituting the process of the operation based on the operation information.

4. The work element analysis device according to claim 2, characterized in that: The operation element inference unit inputs the operation information acquired by the operation information acquisition unit into the learning unit, and infers the operation element by using the learning unit, wherein the learning unit performs machine learning on the correlation between the operation element and the operation information in order to infer the operation elements constituting the process of the operation based on the operation information.

5. The work factor analysis device according to claim 1, characterized in that: The occurrence probability acquired by the occurrence probability acquisition unit is a probability calculated based on the work element for past work information and statistics of the work element estimated by the work element estimation unit based on the past work information.

6. The work element analysis device according to claim 2, characterized in that: The occurrence probability acquired by the occurrence probability acquisition unit is a probability calculated based on the work element for past work information and statistics of the work element estimated by the work element estimation unit based on the past work information.

7. The work element analysis device according to claim 3, characterized in that: The occurrence probability acquired by the occurrence probability acquisition unit is a probability calculated based on the work element for past work information and statistics of the work element estimated by the work element estimation unit based on the past work information.

8. The work element analysis device according to claim 4, characterized in that: The occurrence probability acquired by the occurrence probability acquisition unit is a probability calculated based on the work element for past work information and statistics of the work element estimated by the work element estimation unit based on the past work information.

9. The work element analysis device according to any one of claims 1 to 8, characterized in that: The occurrence probability acquired by the occurrence probability acquisition unit is a probability calculated based on statistics of the work element after the work element has been transformed in the past.

10. The work element analysis device according to any one of claims 1 to 8, characterized in that: The work element estimation unit outputs a probability value of each of the work elements obtained based on the work information acquired by the work information acquisition unit, The correction unit corrects the work element estimated by the work element estimation unit based on the probability value of each of the work elements which is the estimation result of the work element estimation unit.

11. The work factor analysis device according to claim 9, characterized in that: The work element estimation unit outputs a probability value of each of the work elements obtained based on the work information acquired by the work information acquisition unit, The correction unit corrects the work element estimated by the work element estimation unit based on the probability value of each of the work elements which is the estimation result of the work element estimation unit.

12. A method for analyzing work elements, wherein a work element analyzing device analyzes work elements constituting a work based on work information generated by a worker's work, wherein: The operation factor analysis method comprises: An operation information obtaining step of obtaining operation information generated as the operator performs the operation; a work element inferring step of inferring work elements constituting the work based on the work information acquired in the work information acquiring step; an occurrence probability obtaining step of obtaining the occurrence probability of the work element in the work, the occurrence probability being calculated using a value obtained by adding an adjustment value to the total value of each of the work elements inferred in the work element inference step; and The correction step corrects the work element estimated in the work element estimation step based on the occurrence probability acquired in the occurrence probability acquisition step.

Images