Program, information processing method, and information processing device

Abstract

Provided is a program or the like capable of accurately determining whether or not an object to be subjected to a job performs a prescribed job. A computer distinguishes, for each action included in a job, an animation obtained by imaging an object to be subjected to the job. In addition, the computer determines whether or not each action is a standard action on the basis of the respective discriminated animation. Then, the computer analyzes whether or not the job is a standard job on the basis of the determination results of the actions.

Description

Technical Field

[0001] The present disclosure relates to a program, an information processing method, and an information processing device. Background Art

[0002] Patent document 1 discloses the following technology: for a process consisting of a plurality of operations whose order is predetermined, it is detected whether the movement of an object is in accordance with the order of each operation. In the technology disclosed in patent document 1, for each operation constituting a process, it is determined whether the object is performing the operation in accordance with the prescribed operation according to process information that specifies the operation name, operation order, standard operation time, etc.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-82137 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] In the technology disclosed in Patent Document 1, whether the operation is performed in accordance with the regulations is determined by whether each operation is performed within the standard operation time. However, even if the operation is performed within the standard operation time, it is not necessarily in accordance with the prescribed operation content (action content). Therefore, it is difficult to accurately determine whether the operation content is performed in accordance with the regulations only by determining whether the operation time is the standard operation time.

[0008] An object of the present disclosure is to provide a program or the like that can accurately determine whether a target for performing a work is performing a work in accordance with a prescribed procedure.

[0009] Solutions for solving problems

[0010] A program of one method disclosed herein causes a computer to perform the following processing: an animation obtained by filming an object performing an operation is differentiated according to each action included in the operation, based on the differentiated animations, whether each action is a standard action, and based on the determination result of each action, whether the operation is a standard operation.

[0011] Effects of the Invention

[0012] In one aspect of the present disclosure, it is possible to accurately determine whether a subject performing work is performing work in accordance with regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1A It is an explanatory diagram showing an example of an animation to be processed.

[0014] Figure 1BIt is an explanatory diagram showing an example of an animation to be processed.

[0015] Figure 2 It is a block diagram showing a configuration example of an information processing device.

[0016] Figure 3 It is an explanatory diagram showing a configuration example of a learning model.

[0017] Figure 4 It is an explanatory diagram showing a configuration example of a learning model.

[0018] Figure 5 This is a flowchart showing an example of a procedure for generating a motion analysis model.

[0019] Fig. 6A It is an explanatory diagram showing a screen example.

[0020] Figure 6B It is an explanatory diagram showing a screen example.

[0021] Figure 6C It is an explanatory diagram showing a screen example.

[0022] Figure 7 This is a flowchart showing an example of the analysis processing procedure of a job.

[0023] Figure 8 It is an explanatory diagram showing a screen example.

[0024] Fig. 9 This is a flowchart showing an example of the analysis processing procedure for the main factors of each work element.

[0025] Fig. 10A It is an explanatory diagram showing a screen example.

[0026] Fig. 10B It is an explanatory diagram showing a screen example.

[0027] Fig.11 This is a flowchart showing an example of a procedure for analyzing the operation time of each operation.

[0028] Fig.12 It is an explanatory diagram showing a screen example.

[0029] Fig.13 This is a flowchart showing an example of a Gantt chart creation process procedure.

[0030] Fig.14 This is an explanatory diagram showing an example of a Gantt chart screen.

[0031] Fig.15 It is an explanatory diagram showing display examples of two analysis result screens.

[0032] Fig.16It is an explanatory diagram showing a configuration example of a loop animation extraction model.

[0033] Fig.17 This is a flowchart showing an example of a comparison processing procedure of a loop animation.

[0034] Fig.18A It is an explanatory diagram showing a screen example.

[0035] Fig.18B It is an explanatory diagram showing a screen example.

[0036] Fig. 18C It is an explanatory diagram showing a screen example.

[0037] Fig.18D It is an explanatory diagram showing a screen example.

[0038] Fig.19A It is an explanatory diagram showing a screen example.

[0039] Fig.19B It is an explanatory diagram showing a screen example.

[0040] Fig.19C It is an explanatory diagram showing a screen example.

[0041] Fig. 20A It is an explanatory diagram showing a screen example.

[0042] Fig. 20B It is an explanatory diagram showing a screen example.

[0043] Fig. 20C It is an explanatory diagram showing a screen example.

[0044] Fig.20D It is an explanatory diagram showing a screen example.

[0045] Fig.21A It is an explanatory diagram showing a screen example.

[0046] Fig. 21B It is an explanatory diagram showing a screen example.

[0047] Fig. 22 This is a flowchart showing an example of a comparison processing procedure of loop animations according to the third embodiment.

[0048] Fig.23 It is an explanatory diagram showing a screen example.

[0049] Fig.24 This is a flowchart showing an example of a comparison processing procedure of loop animations according to the fourth embodiment.

[0050] Fig.25 It is an explanatory diagram showing an example of a reference cycle animation.

[0051] Fig.26 This is a flowchart showing an example of a comparison processing procedure of loop animations according to the fifth embodiment.

[0052] Fig.27A It is an explanatory diagram showing a screen example.

[0053] Fig.27B It is an explanatory diagram showing a screen example. DETAILED DESCRIPTION

[0054] Hereinafter, the program, the information processing method, and the information processing device of the present disclosure will be described in detail based on the drawings showing embodiments thereof.

[0055] (Implementation Method 1)

[0056] An information processing device will be described that analyzes whether a work performed by a worker (object) performing a work is a preset standard work based on a moving picture obtained by photographing the worker (object) performing the work. Figure 1A and Figure 1B 2 is an explanatory diagram showing an example of an animation to be processed. The animation to be processed in this embodiment is, for example, animation data containing a plurality of images (still images) such as 30 or 15 images per second. Figure 1A As shown in FIG. 1 , the image is obtained by shooting the worker performing the work from above the worker. Figure 1B As shown, the animation of the processing object is, for example, an image obtained by shooting from the beginning to the end of the work of assembling a product, and includes animations that are differentiated according to each of the multiple work elements included in the assembly work (work element animation, hereinafter referred to as loop animation). In addition, the loop animation includes animations that are differentiated according to each of the multiple actions included in each work element (differentiation animation, hereinafter referred to as action animation). Figure 1B In the example shown, the animation obtained by filming the workers performing the assembly operation of product variety A can be distinguished according to each cyclic animation obtained by filming the workers performing the work elements such as "assembly of box A" and "assembly of component A", and the cyclic animation of the work element "assembly of box A" can be distinguished according to each action animation obtained by filming the workers performing the actions such as "taking out the box", "walking", "installing the partition", and "closing the box".

[0057] Figure 2: is a block diagram showing an example of the configuration of an information processing device. The information processing device 10 is a device capable of realizing various information processing and information transmission and reception, and is composed of, for example, a server computer, a personal computer, a workstation, etc. The information processing device 10 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, etc., and the above-mentioned units are interconnected via a bus. The control unit 11 includes one or more processors such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), or an AI chip (semiconductor for AI). The control unit 11 performs the information processing and control processing that the information processing device 10 should perform by appropriately executing the program 12P stored in the storage unit 12.

[0058] The storage unit 12 includes RAM (Random Access Memory), flash memory, hard disk, SSD (Solid State Drive), etc. The storage unit 12 stores the program 12P (program product) executed by the control unit 11 and various data. In addition, the storage unit 12 temporarily stores data generated when the control unit 11 executes the program 12P. The program 12P and various data can be written into the storage unit 12 during the manufacturing stage of the information processing device 10, or can be downloaded from other devices by the control unit 11 via the communication unit 13 and stored in the storage unit 12. In addition, the storage unit 12 stores learning models such as the variety prediction model M1, the cycle prediction model M2, the boundary prediction model M3, the action analysis model M4, and the main factor classification model M5 after the training data has been learned through machine learning. It is assumed that the learning models M1 to M5 are used as program modules constituting artificial intelligence software. The learning models M1 to M5 are used to perform predetermined operations on input values ​​and output the operation results. In the storage unit 12, information such as the layers possessed by the learning models M1 to M5, the nodes constituting each layer, and the weights (coupling coefficients) between the nodes are stored as information defining the learning models M1 to M5. It should be noted that the cycle prediction model M2, the boundary prediction model M3, and the motion analysis model M4 are prepared in plurality for each type of product to be assembled by the operator. In addition, the storage unit 12 stores a plurality of animation files 12a of the processing object.

[0059] The communication unit 13 is a communication module for performing processing related to wired communication or wireless communication, and transmits and receives information with other devices via a network N. The network N may be the Internet or a public communication line, or a LAN (Local Area Network) constructed in a facility where the information processing device 10 is installed. The input unit 14 receives operation inputs made by the user, and sends a control signal corresponding to the operation content to the control unit 11. The display unit 15 is a liquid crystal display or an organic EL display, etc., and displays various information according to instructions from the control unit 11. A part of the input unit 14 and the display unit 15 may be a touch panel configured as an integral body.

[0060] The reading unit 16 reads information stored in a mobile storage medium 10a including a CD (Compact Disc), a DVD (Digital Versatile Disc), a USB (Universal Serial Bus) memory, an SD (Secure Digital) card, etc. The program 12P and various data stored in the storage unit 12 can be read from the mobile storage medium 10a by the control unit 11 via the reading unit 16 and stored in the storage unit 12.

[0061] The information processing device 10 may be a multi-computer including a plurality of computers, or a virtual machine virtually constructed by software in one device. In the case where the information processing device 10 is composed of a server computer, the information processing device 10 may be a local server, or a cloud server connected to a network such as the Internet. In addition, the program 12P may be executed on a single computer, or on a plurality of computers connected to each other via a network N. Moreover, the input unit 14 and the display unit 15 are not necessary, and the information processing device 10 may be configured to accept operations through a connected computer, or may be configured to output information to be displayed to an external display device.

[0062] Figure 3 and Figure 4 This is an explanatory diagram showing an example of the configuration of learning models M1 to M5. Learning models M1 to M5 are configured using, for example, RNN (Recurrent Neural Network) or LSTM (Long Short-Term Memory). It should be noted that learning models M1 to M5 may be configured using algorithms such as CNN (Convolution Neural Network) and Transformer in addition to RNN and LSTM, and may also be configured by combining multiple algorithms.

[0063] The variety prediction model M1 is a learning model that has been learned in the following manner: an animation obtained by photographing the whole body of a worker at work is used as input, and an operation is performed based on the input animation to determine the variety of the product that the worker is assembling, and the result of the operation is output. It should be noted that the variety prediction model M1 is not limited to the composition of the input animation obtained by photographing the whole body of the worker, for example, it can also be the composition of the input animation obtained by photographing the appearance of the worker performing the assembly work, such as the animation obtained by photographing the upper body or hands of the worker. The variety prediction model M1 has: an input layer that inputs the animation; an intermediate layer that extracts feature quantities from the input animation; and an output layer that outputs information representing the variety of the product that the worker in the animation is assembling based on the operation results of the intermediate layer. The input layer has input nodes that sequentially input images (frames) of a time series contained in the animation. The intermediate layer uses various functions and thresholds to calculate output values ​​based on the animation input via the input layer. The output layer has a plurality of output nodes respectively associated with pre-set varieties, and the probability (confidence) that each variety should be determined as a variety of the product to be assembled by the operator is output from each output node. The output value from each output node of the output layer is, for example, a value of 0 to 1, and the total of the probabilities output from each output node is 1.0 (100%). For example, when an animation is input to the variety prediction model M1, the information processing device 10 determines the output node that outputs the maximum output value among the output values ​​(confidence) from each output node, and determines the variety associated with the determined output node as the variety of the product to be assembled by the operator in the animation. It should be noted that, instead of outputting a plurality of output nodes for outputting the confidence for each variety, the output layer of the variety prediction model M1 may also be composed of a single output node that outputs information indicating the variety with the greatest confidence.

[0064] The variety prediction model M1 can be generated by machine learning using training data including an animation for training and information (correct labels) indicating the variety of products that the worker in the animation performs assembly work. The variety prediction model M1 is learned in the following manner: when the animation included in the training data is input, the output value from the output node corresponding to the variety indicated by the correct label included in the training data is made close to 1, and the output value from the other output nodes is made close to 0. In the learning process, the variety prediction model M1 performs operations in the intermediate layer and the output layer based on the input animation, and calculates the output value from each output node, and compares the calculated output value of each output node with the value corresponding to the correct label (1 for the output node corresponding to the correct label and 0 for the other output nodes) to optimize the parameters used for the operation processing in the intermediate layer and the output layer in a manner that makes the two approximate. The parameters are weights (coupling coefficients) between nodes in the intermediate layer and the output layer, coefficients of functions, thresholds, etc. The parameter optimization method is not particularly limited, and the back error propagation method, the gradient descent method, etc. can be used. Thus, when an animation is input, a product prediction model M1 is obtained that predicts the product type that a worker in the animation is assembling and outputs the prediction result.

[0065] The cycle prediction model M2 is a learning model that has been learned in the following manner: an animation obtained by shooting the whole body of a worker in operation is used as input, and an operation is performed to distinguish the animation into a loop animation based on the input animation, and the result of the operation is output. The cycle prediction model M2 is not limited to the composition of the input animation obtained by shooting the whole body of the worker. For example, it can also be the composition of the input animation obtained by shooting the appearance of the assembly work being performed by the worker, such as the animation obtained by shooting the upper body or hands of the worker. The cycle prediction model M2 has: an input layer for inputting the animation; an intermediate layer that extracts feature quantities from the input animation; and an output layer that outputs the start time and end time of each loop animation obtained by distinguishing the animation based on the operation result of the intermediate layer. The intermediate layer of the cycle prediction model M2 calculates the output value based on the animation input via the input layer, and the output layer has an output node that outputs the start time and end time of each loop animation obtained by distinguishing the animation. It should be noted that the start time and end time of the loop animation represent the elapsed time from the start of the animation playback. When an animation is input to the loop prediction model M2 , the information processing device 10 classifies the animation into each loop animation based on the output value (the start time and the end time of the loop animation) from the output node.

[0066] The cycle prediction model M2 is generated using a setting animation for setting the boundaries of each cycle animation (each operation element) in the animation. The setting animation uses an animation obtained by filming the appearance of a worker who can perform standard actions for each action. Specifically, when a setting animation is input into the cycle prediction model M2, the cycle prediction model M2 performs operations in the intermediate layer and the output layer based on the input animation to calculate the output value from the output node, and outputs the time that divides the setting animation into the boundaries of each cycle animation (the start time and the end time of the cycle animation). The user can correct the time of the boundary of each cycle output by the cycle prediction model M2, and the cycle prediction model M2 optimizes the parameters used for the operation processing in the intermediate layer and the output layer based on the corrected time of each boundary. In this way, a cycle prediction model M2 is obtained that predicts the start time and the end time of each cycle animation in the animation when the animation is input and outputs the prediction result.

[0067] The boundary prediction model M3 is a learning model that has been learned in the following manner: an animation that has been classified as a loop animation is used as an input, and a calculation is performed to classify each loop animation into an action animation based on the input animation, and a calculation is performed to predict a time period in which a boundary can be set other than the boundary of the classified action animation, and the result of the calculation is output. The boundary prediction model M3 has: an input layer for inputting an animation; an intermediate layer that extracts a feature value from the input animation; and an output layer that outputs the start time and end time of the action animation obtained by classifying each loop animation, and the time period in which the boundary of the action animation can be set based on the calculation result of the intermediate layer. The intermediate layer of the boundary prediction model M3 calculates an output value based on the animation input via the input layer, and the output layer has output nodes that output the start time and end time of each action animation obtained by classifying each loop animation in the animation, and the time period in which the boundary of the action animation can be set. When an animation is input to the boundary prediction model M3, the information processing device 10 classifies the loop animation in the animation into each action animation based on the output value (the start time and end time of the action animation) from the output node.

[0068] The boundary prediction model M3 is generated using a setting animation for setting the boundaries of each action animation (each action). The animation used for the generation of the boundary prediction model M3 is an animation used for the generation of the loop prediction model M2, and is an animation that is divided into each loop animation by the loop prediction model M2. When the setting animation is input into the boundary prediction model M3, the boundary prediction model M3 performs operations in the intermediate layer and the output layer based on the input animation to calculate the output value from the output node, and outputs the time (the start time and end time of the action animation) that divides the setting animation into the boundaries of each action animation, and the settable time period of the boundary of the action animation. The user can correct the time of the boundary of each action output by the boundary prediction model M3, and can also correct the settable time period of the boundary output by the boundary prediction model M3. The boundary prediction model M3 optimizes the parameters used for the calculation processing in the intermediate layer and the output layer based on the corrected time of each boundary and the settable time period. As a result, a boundary prediction model M3 is obtained that predicts the start time and end time of each action animation in the animation, and the settable time period of the boundary of each action animation when the animation is input, and outputs the prediction result.

[0069] The motion analysis model M4 is a learning model that has been learned in the following manner: an animation that has been classified as an action animation, that is, an animation with boundaries between action animations set, is used as input, and operations are performed based on the input animation being classified into each action animation contained in the cyclic animation in the animation, and operations are performed on whether the actions of the operator in the classified action animation are standard actions, and the results of the operations are output. The motion analysis model M4 has: an input layer for inputting the animation; an intermediate layer that extracts feature values ​​from the input animation; and an output layer that outputs the start time and end time of the action animation contained in each cyclic animation in the animation, and analysis results (information related to the actions performed by the operator) indicating whether the actions of the operator in each action animation are standard actions based on the calculation results of the intermediate layer. The intermediate layer of the motion analysis model M4 calculates output values ​​based on the animation input via the input layer, and the output layer has output nodes for the start time and end time of each action animation in the output animation, and the analysis results of each action animation. When an animation is input into the motion analysis model M4, the information processing device 10 distinguishes the cyclic animation in the animation as an action animation based on the output value from the output node (the start time and end time of the action animation, the analysis result), and determines whether the action of the operator in each action animation is a standard action.

[0070] The motion analysis model M4 is generated using an animation obtained by filming a worker performing a standard motion for each motion as a setting animation. The animation generated by the motion analysis model M4 is an animation generated by the boundary prediction model M3, and is an animation that is distinguished as each motion animation by the boundary prediction model M3. When a setting animation is input into the motion analysis model M4, the motion analysis model M4 performs operations in the intermediate layer and the output layer based on the input animation to calculate the output value from the output node, and outputs the time (the start time and the end time of the motion animation) that distinguishes the setting animation as the boundary of each motion animation, and the analysis result indicating whether the motion of each motion animation is a standard motion. The user can correct the time of the boundary of each motion output by the motion analysis model M4, and the motion analysis model M4 optimizes the parameters used for the calculation processing in the intermediate layer and the output layer based on the corrected time of each boundary. It should be noted that the analysis result here is information indicating that each motion is a standard motion. Therefore, when the output analysis result is information indicating that it is not a standard motion, the motion analysis model M4 corrects the analysis result and optimizes the parameters based on the corrected analysis result. Thus, a motion analysis model M4 is obtained which predicts the start time and end time of each action animation in the animation when the animation is input, as well as the analysis results of the actions of each action animation, and outputs the prediction results. By using such a motion analysis model M4, it is determined whether the actions of the operator captured in the animation of the analysis object are appropriate as a comparison result with the standard action. It should be noted that the motion analysis model M4 can also be constructed as follows: in addition to the analysis results indicating whether the actions of the operator in each action animation are standard actions, when the actions of the operator are not standard actions, information indicating the main factors that are considered not to be standard actions is also output. The main factors that are considered not to be standard actions include, for example, an action time that is longer or shorter than the standard action time by more than a predetermined time.

[0071] The main factor classification model M5 is a learning-completed model after learning in the following manner: taking the cyclic animation distinguished from the animation as input, performing an operation based on the input cyclic animation to judge the main factor that the work element performed by the operator in the cyclic animation is not a standard work, and outputting the result of the operation. The main factor classification model M5 has: an input layer that inputs the cyclic animation; an intermediate layer that extracts feature quantities from the input cyclic animation; and an output layer that outputs information indicating that the work element of the operator in the cyclic animation is not a main factor of the standard work (information related to the main factor) based on the operation result of the intermediate layer. The intermediate layer of the main factor classification model M5 calculates the output value based on the cyclic animation input via the input layer, and the output layer has a plurality of output nodes respectively associated with the pre-set main factors, and outputs the probability (confidence) that each main factor should be judged as a main factor that the work element performed by the operator is not a standard work from each output node. The output value from each output node of the output layer is, for example, a value of 0 to 1, and the total of the probabilities output from each output node is 1.0 (100%). The main factors that make it considered that the operator's work element is not a standard work include, for example: the lack of an action included in the work element, the change in the execution order of the actions included in the work element, the action of the action included in the work element is different from the standard action, the action time of the action included in the work element is longer or shorter than the standard action time by more than a predetermined time, etc. For example, when a loop animation is input into the main factor classification model M5, the information processing device 10 determines the output node that outputs the maximum output value among the output values ​​(confidence) from each output node, and determines the main factor associated with the determined output node as the main factor that makes it considered that the operator's work element in the loop animation is not a standard work. It should be noted that, instead of having multiple output nodes that output the confidence of each main factor, the output layer of the main factor classification model M5 can also be composed of a single output node that outputs information representing the main factor with the greatest confidence.

[0072] The main factor classification model M5 can be generated by machine learning using training data including a training loop animation and information indicating that the main factor of the worker's work element in the loop animation is not a standard work (correct label). The main factor classification model M5 is learned in the following manner: when the loop animation included in the training data is input, the output value from the output node corresponding to the main factor indicated by the correct label included in the training data is made close to 1, and the output value from other output nodes is made close to 0. In the learning process, the main factor classification model M5 performs operations in the intermediate layer and the output layer based on the input animation, calculates the output value from each output node, and compares the calculated output value of each output node with the value corresponding to the correct label (1 for the output node corresponding to the correct label and 0 for other output nodes) to optimize the parameters used for the operation processing in the intermediate layer and the output layer in a manner that the two are approximate. Here, too, the optimized parameters are the weights (coupling coefficients) between the nodes in the intermediate layer and the output layer, the coefficients of the function, the threshold value, etc., and the method for optimizing the parameters can use the back error propagation method, the gradient descent method, etc. Thus, when a loop animation is input, a major factor classification model M5 is obtained which predicts that the work element of the worker in the loop animation is not a major factor of the standard work and outputs the prediction result.

[0073] The learning of the learning models M1 to M5 can also be performed by other learning devices. The learning models M1 to M5 generated by learning by other learning devices are downloaded from the learning device to the information processing device 10 via the network N or the mobile storage medium 10a and stored in the storage unit 12. In the present embodiment, the cyclic prediction model M2, the boundary prediction model M3, and the motion analysis model M4 are generated by single-sample learning using a set of training data (setting animation) or a small amount of training data, but can also be generated by learning using multiple training data. In addition, the learning models M1 to M5 are not limited to the input of the animation obtained by shooting the operator or the cyclic animation distinguished from the animation. For example, it can also be configured that the information processing device 10 performs skeleton estimation of the operator based on each frame (each image) included in the animation to generate skeleton data, and inputs the skeleton data of the time series into the learning models M1 to M5. In this case, the control unit 11 of the information processing device 10 uses a technology such as Open Pose to extract the joint position of the person in the image, and performs skeleton estimation of the operator in each frame, and extracts the joint position of the operator. The control unit 11 may further acquire time-series skeleton data by extracting joint positions from each frame included in the animation, and use the acquired time-series skeleton data as input to the learning models M1 to M5 .

[0074] Next, the generation process of the motion analysis model M4 in the information processing device 10 according to the present embodiment will be described. Figure 5 is a flowchart showing an example of a generation process sequence of the motion analysis model M4. Figure 6A to Figure 6C The following processing is executed by the control unit 11 of the information processing device 10 according to the program 12P stored in the storage unit 12.

[0075] When generating a motion analysis model M4 for each variety, the user selects a setting animation for the motion analysis model M4. The setting animation for the motion analysis model M4 is an animation that has distinguished each action included in each cycle (each work element) using the cycle prediction model M2 and the boundary prediction model M3. The control unit 11 of the information processing device 10 accepts the selection of the setting animation for the motion analysis model M4 through the operation via the input unit 14 (S11). The control unit 11 inputs the selected animation into the motion analysis model M4, and based on the output information from the motion analysis model M4, distinguishes the animation of the processing object into an action animation, and extracts boundary candidates for each action (S12). Specifically, the control unit 11 inputs each frame contained in the animation file 12a into the motion analysis model M4 in sequence, and determines the time of the boundary candidate for each action based on the start time and end time of each action animation output from the motion analysis model M4.

[0076] When the control unit 11 extracts the boundary candidates of each action animation in the animation to be processed, Fig. 6A The boundary editing screen shown is displayed on the display unit 15 (S13). The boundary editing screen is a screen for changing the time of the boundary candidate of each action based on the start time and end time of each action animation output from the action analysis model M4. Fig. 6A The screen shown has: an animation area A1, which displays the animation of the processing object; and an editing area A2, which displays the time of the boundary candidate extracted from the animation. In the editing area A2, the start time of each action animation distinguished from the animation is displayed as the time of the boundary candidate between each action animation and the previous action animation, and a delete button B1 (× button) for instructing to delete each boundary candidate and an add button B2 (+ button) for instructing to create a new boundary between each boundary candidate are provided. It should be noted that it is also possible to configure so that a name can be given to the boundary of each action. In this case, the name given to each boundary can be displayed in the editing area A2 together with the time of each boundary candidate.

[0077] The control unit 11 determines whether a deletion instruction for any boundary candidate is received by operating the delete button B1 in the editing area A2 (S14), and if it is determined that a deletion instruction is received (S14: YES), the control unit 11 deletes the boundary candidate instructed to be deleted (S15). If a boundary candidate is deleted, the control unit 11 deletes the boundary candidate instructed to be deleted from the boundary candidates displayed in the editing area A2, and displays the boundary candidates after the deleted boundary candidate in sequence in advance. If it is determined that a deletion instruction is not received (S14: NO), the control unit 11 skips step S15 and determines whether an addition instruction for a new boundary is received by operating the add button B2 in the editing area A2 (S16).

[0078] When it is determined that a new boundary addition instruction has been received (S16: YES), the control unit 11 Figure 6B The border addition screen shown is displayed on the display unit 15 (S17). Fig. 6A In the screen shown, the add button B2 between the second and third boundary candidates is operated. In this case, Figure 6B As shown, a boundary addition screen for adding a new boundary is displayed at any playback time in the time period between the second and third boundary candidates. It should be noted that when generating the setting animation of the motion analysis model M4, when the settable time period of the boundary output from the boundary prediction model M3 is obtained, the control unit 11 can display the settable time period in the boundary addition screen. In this case, the user can specify the time of the new boundary in consideration of the settable time period.

[0079] The user inputs the time for setting the boundary into the boundary adding screen, and the control unit 11 receives the new boundary time through the boundary adding screen (S18). Figure 6B When the Add button is operated in the screen shown in FIG. 1 , the control unit 11 adds a boundary at the accepted time (S19). Figure 6C As shown, the control unit 11 adds a new boundary between the second and third boundary candidates, and sequentially delays the display of boundary candidates after the added boundary. If it is determined that no instruction to add a new boundary is received (S16: NO), the control unit 11 skips steps S17 to S19.

[0080] The control unit 11 determines Fig. 6A , C, whether the save button is operated (S20). If it is determined that the save button is not operated (S20: NO), the process returns to step S14 and repeats the processing of steps S14 to S19. Fig. 6AWhen the "Next" button is pressed in the screen shown in FIG. 2 and FIG. 3 , the control unit 11 changes the display content of the editing area A2 to the boundary candidate after the boundary candidate displayed in the bottom row. Fig. 6A When the cancel button is operated in the screen shown in FIG. 2 and FIG. 3C, the control unit 11 interrupts and ends the editing process of the boundary candidate.

[0081] When it is determined that the save button has been operated (S20: YES), the control unit 11 determines the boundary candidate after editing (deletion and addition) through the editing area A2 as a boundary, and stores the time of each boundary in the storage unit 12 (S21). Specifically, the control unit 11 associates the boundary number indicating the order of appearance of each boundary with the time of each boundary, and stores it in, for example, a DB set in the storage unit 12. It should be noted that when each boundary is given a name, the name of each boundary can also be stored. Then, the control unit 11 sets (optimizes) the parameters for the operation processing in the intermediate layer and output layer of the motion analysis model M4 based on the time of each boundary (S22). Here, the control unit 11 sets the parameters of the motion analysis model M4 in a manner that can distinguish the setting animation input to the motion analysis model M4 according to each time of each boundary stored in step S21. In addition, the control unit 11 sets the parameters of the motion analysis model M4 in the following manner: based on the analysis result output by the motion analysis model M4 when the setting animation is input, the analysis result represented as a standard action is output for each action animation. Thereby, it is possible to generate the motion analysis model M4 which, when an animation is input, divides the animation into motion animations and outputs an analysis result of whether the motion of each divided motion animation is a standard motion.

[0082] In the above process, the following structure is used: by deleting the boundaries between the action animations extracted using the action analysis model M4 and adding new boundaries, the boundaries between the action animations are edited. With such a structure, the parameters of the action analysis model M4 can be set with high precision based on the edited boundaries between the action animations. In addition, the editing method of each boundary is not limited to the above process, and for example, it can also be configured so that the editing area A2 can directly change the time of each boundary candidate.

[0083] exist Figure 5, the processing for setting the parameters of the motion analysis model M4 is shown, but the parameters of the loop prediction model M2 and the boundary prediction model M3 can also be set by the same processing. For example, when setting the parameters of the loop prediction model M2, the control unit 11 inputs the animation of the processing object into the loop prediction model M2, and based on the output information from the loop prediction model M2, the animation of the processing object is divided into loop animations, and the boundary candidates of each loop are extracted. Then, the control unit 11 accepts the user's editing of the boundary candidates of each loop animation, and can set the parameters of the loop prediction model M2 based on the time of the boundaries of each loop animation after editing. In addition, when setting the parameters of the boundary prediction model M3, the control unit 11 inputs the animation of the processing object into the boundary prediction model M3, and based on the output information from the boundary prediction model M3, the animation of the processing object is divided into action animations, and the boundary candidates of each action are extracted. Then, the control unit 11 accepts the user's editing of the boundary candidates of each action, and can set the parameters of the boundary prediction model M3 based on the time of the boundaries of each action after editing.

[0084] Next, a description will be given of a process of analyzing whether or not a worker is performing a standard operation based on a moving image obtained by photographing a worker performing the operation in the information processing device 10 of the present embodiment. Figure 7 is a flowchart showing an example of the analysis processing sequence of a job. Figure 8 The following processing is executed by the control unit 11 of the information processing device 10 according to the program 12P stored in the storage unit 12.

[0085] When analyzing whether the work of the operator is a standard work, the user selects an animation obtained by filming the operator of the analysis object. The control unit 11 of the information processing device 10 accepts the selection of the animation of the analysis object by operating through the input unit 14 (S31). The control unit 11 inputs the selected animation (animation file 12a) into the variety prediction model M1, and determines the variety of the product (operation object) that the operator in the animation of the analysis object performs the assembly work based on the output information from the variety prediction model M1 (S32). Specifically, the control unit 11 inputs each frame contained in the animation file 12a into the variety prediction model M1 in sequence, and when the maximum output value (confidence) from the variety prediction model M1 reaches a predetermined value or more (for example, 0.7 or more), the variety corresponding to the output node that outputs the maximum output value is determined as the variety of the product that the operator in the animation is assembling. It should be noted that the control unit 11 may not use the variety prediction model M1 to determine the variety of the operation object. For example, when the operation object includes different items of each variety, the control unit 11 can detect the items contained in the animation and determine the variety based on the detected items.

[0086] When the control unit 11 determines the product that the operator is assembling, it selects the cycle prediction model M2, boundary prediction model M3, and motion analysis model M4 corresponding to the determined product (S33). It should be noted that a plurality of cycle prediction models M2, boundary prediction models M3, and motion analysis models M4 are prepared for each product. The control unit 11 uses the selected cycle prediction model M2 to divide the animation of the analysis object into cycle animations and extract the boundaries of each cycle (S34). Here, the control unit 11 inputs each frame contained in the animation file 12a into the cycle prediction model M2 in sequence, and determines (predicts) the start time and end time of each cycle animation contained in the animation based on the output information from the cycle prediction model M2. It should be noted that it can also be configured that after the processing of step S34, the control unit 11 displays an editing screen for changing the boundaries of each cycle extracted from the animation (deleting and adding new ones, etc.) on the display unit 15 (refer to Figure 6A to Figure 6C ) and receives instructions to change the time of the boundaries of each loop through the editing screen and makes the changes.

[0087] Next, the control unit 11 (differentiation unit) uses the boundary prediction model M3 selected in step S33 to differentiate the animation after extracting the boundaries of each cycle into an action animation that captures the actions performed by the operator, and extracts the boundaries of each action (S35). Here, the control unit 11 sequentially inputs each frame included in the animation that has been differentiated as a loop animation into the boundary prediction model M3, and determines (predicts) the start time and end time of each action animation included in each loop animation based on the output information from the boundary prediction model M3. It should be noted that it can also be configured that, after the processing of step S35, the control unit 11 displays an editing screen for changing (deleting, adding, etc.) the boundaries (differentiation positions) of each action extracted from the animation on the display unit 15 (refer to Figure 6A to Figure 6C ) receives instructions to change the time of the boundaries of each action through the editing screen and makes the changes.

[0088] Next, the control unit 11 uses the motion analysis model M4 selected in step S33 to analyze whether the motion of the operator captured in each motion animation contained in the animation is a standard motion based on the animation after the boundaries of each motion are extracted (S36). Here, the control unit 11 sequentially inputs each frame contained in the animation into the motion analysis model M4, and determines (predicts) the start time and end time of each motion animation contained in the animation, and the analysis result indicating whether the motion of the operator in each motion animation is a standard motion based on the output information from the motion analysis model M4. The control unit 11 (determination unit) can determine whether the motion performed by the operator is a standard motion based on the analysis result.

[0089] The control unit 11 (analysis unit) analyzes, for each cyclic animation in the animation, whether the work element performed by the operator captured in the cyclic animation is a standard work (S37) based on the analysis result of the action animation included in the cyclic animation. Here, when the control unit 11 obtains the analysis result that any action animation included in the cyclic animation is not a standard action, it determines that the work element of the operator in the cyclic animation is not a standard work. For example, when any one of the action animations included in the cyclic animation that captured a work element is missing, the control unit 11 determines that the action is omitted, and the work element of the operator in the cyclic animation is not a standard work. In addition, when the order of appearance of any one of the action animations included in the cyclic animation is different, the control unit 11 determines that the order is swapped, and the work element of the operator in the cyclic animation is not a standard work. In addition, when the action of the operator in any one of the action animations included in the cyclic animation is different from the standard action, the control unit 11 determines that it is an abnormal activity, and the work element of the operator in the cyclic animation is not a standard work. In addition, when the time (action time performed by the operator) in any of the action animations included in the loop animation is longer than the preset standard action time by more than a predetermined time, the control unit 11 determines that the action time is long, and the operation element of the operator in the loop animation is not a standard operation. When it is shorter than the predetermined time, it is determined that the action time is short, and the operation element of the operator in the loop animation is not a standard operation. Such analysis and processing can be performed using the main factor classification model M5. In this case, the control unit 11 inputs the loop animation into the main factor classification model M5, and analyzes whether the operation element is a standard operation based on the output information from the main factor classification model M5. It should be noted that the action content that is considered not to be a standard operation is not limited to the above example. When the control unit 11 obtains the analysis result that all the action animations included in the loop animation are standard actions, the operation element of the operator in the loop animation is determined to be a standard operation.

[0090] The control unit 11 displays the analysis results (S38) of each loop animation on the display unit 15. For example, the control unit 11 generates Figure 8 The analysis result screen shown is displayed on the display unit 15. Figure 8The analysis result screen shown has: an animation area A3, which displays the animation of the analysis object; and a result area A4, which displays the analysis results of the actions included in the work elements performed by the operator in the animation. In the result area A4, for the actions performed by the operator in each action animation distinguished from the animation, the action number indicating the order of appearance, the name given to each action, and the analysis result are displayed. The analysis result includes: the action time required for the execution of each action, and a mark (0 or ×) indicating whether each action is a standard action. It should be noted that a play stop button B3 for indicating the playback and stop of the displayed animation is provided in the animation area A3. When a playback instruction is given via the play stop button B3, the control unit 11 plays the animation of the analysis object in the animation area A3, and displays the analysis results of each action animation analyzed based on the played animation in the result area A4. As a result, it is possible to grasp the analysis results while confirming the actions of the operator using the played animation.

[0091] Through the above processing, the information processing device 10 identifies the operation elements performed by the operator and the actions contained in each operation element based on the animation obtained by shooting the operator, analyzes whether it is a standard operation according to each action, and analyzes whether each operation element is a standard operation based on the analysis results of each action. In this way, the operation performed by the operator is divided into action units, and the appropriateness (specifically, whether it is a standard action) is determined according to each action. By using the result, the operation elements (a cycle of operation) of the operator can be determined with higher accuracy. In addition, the appropriateness of each action is determined using the action analysis model M4, so not only the action time of each action is considered, but also the action content (the activity of the operator) is considered to determine whether the standard action is performed. As a result, it is possible to analyze with high accuracy whether the operator is performing the standard action (according to the prescribed operation) set in advance for the action analysis model M4. In addition, based on the result obtained by determining whether each action is appropriate, inappropriate actions and operation elements containing inappropriate actions can be prompted, so that improvement guidance for the operator becomes easy.

[0092] The control unit 11 determines whether the above processing has been completed for all frames included in the animation to be analyzed (S39). If it is determined that it has not been completed (S39: NO), the control unit 11 returns to the processing of step S34 and repeats the processing of steps S34 to S38. In this way, it is possible to analyze whether each work element performed by the operator is appropriate based on each loop animation included in the animation to be analyzed, and the obtained analysis results can be presented to the user.

[0093] When the control unit 11 determines that the above processing has been completed for the animation of the analysis object (S39: YES), the control unit 11 associates the analysis result with the animation of the analysis object and stores it in the storage unit 12 (S40). For example, the control unit 11 stores the identification information (e.g., the cycle number indicating the order of appearance), the time of the boundary of each cycle, and the analysis result of the work element (cycle) in the DB prepared by the storage unit 12 for each cycle animation extracted from the animation of the analysis object, and stores the identification information (e.g., the action number indicating the order of appearance), the time of the boundary of each action, and the analysis result of the action in the DB prepared by the storage unit 12 for each action animation in each cycle animation. Through the above processing, based on the animation obtained by shooting the operator performing the operation, it is analyzed whether the operator is performing the standard action preset in the action analysis model M4, and the analysis result is displayed to the user.

[0094] In the above process, the type of product that the operator is working on is determined based on the animation of the analysis object, and the analysis process using the motion analysis model M4 prepared for each type of product is performed. Therefore, for example, even if assembly work of multiple different types of products is performed at the same work site, analysis processing can be performed for each type of product, and the user performing the analysis processing does not need to be aware of the type in each animation, and can perform analysis processing on each animation with high accuracy.

[0095] The analysis results of each animation obtained based on the above analysis process can help understand the tendency of the work content performed by the operator and be used to study the improvement strategy for the assembly work. For example, based on the analysis results of each loop animation included in each animation, the tendency of each major factor in the case where the work elements in each loop animation are not standard work can be analyzed and countermeasures can be taken. Fig. 9 is a flowchart showing an example of the analysis and processing sequence of the main factors of each work element. Fig. 10A and Fig. 10B The following processing is executed by the control unit 11 of the information processing device 10 according to the program 12P stored in the storage unit 12.

[0096] When analyzing the occurrence of major factors in non-standard operations for each work element performed by the operator, the user selects an animation obtained by photographing the operator to be analyzed. The animation is obtained by Figure 7The processing is performed to produce an animation after the analysis processing of each operation element. The animation of the processing object here is an animation obtained by filming an operator performing an assembly operation of a product of a certain type. It should be noted that the following processing can be performed to analyze the operation content of an operator based on multiple operations performed by the operator, or it can be performed to analyze the operation content of multiple operators based on operations performed by multiple operators.

[0097] The control unit 11 extracts a loop animation from the animation of the processing object (S51), and based on the extracted loop animation, determines that the work element performed by the operator in the loop animation is not the main factor of the standard work (S52). Here, the control unit 11 inputs each frame included in the loop animation into the main factor classification model M5 in sequence, and based on the output information from the main factor classification model M5, determines that the work of the operator in the loop animation is not the main factor of the standard work. It should be noted that the control unit 11 can also be based on Figure 7 The analysis results of each loop animation obtained by the analysis process shown are the main factors that determine that the work elements performed by the operators in each loop animation are not standard work. The control unit 11 determines any main factors such as missing actions, sequence changes, abnormal activities, long action time, short action time, etc. The control unit 11 associates the identified main factors with the loop animation of the processing object and stores them in the storage unit 12 (S53). For example, the control unit 11 associates the identified main factors with the loop number of each loop animation and stores them.

[0098] Next, the control unit 11 calculates the occurrence ratio of each main factor based on the identified main factor (S54), and displays the calculated occurrence ratio of each main factor on the display unit 15 (S55). Fig. 10A The main factor screen shown is displayed on the display unit 15. Fig. 10A The main factor screen shown shows an overview of the work elements included in the assembly work of product A, and displays the order of appearance (cycle number), name, and occurrence ratio of each main factor for each work element. It should be noted that the occurrence ratio of each main factor includes not only the ratio of cyclic animations that are judged as non-standard work, but also the ratio of cyclic animations that are judged as standard work. In the case where the main factors based on the analysis results of each action animation have been judged and stored in the storage unit 12 together with the animation, the control unit 11 does not perform the processing of steps S52 to S53, but reads the main factors based on the analysis results from the storage unit 12 and performs the processing of steps S54 to S55.

[0099] The control unit 11 determines whether there is an unprocessed animation that has not been processed in steps S51 to S55 (S56). If it is determined that there is an unprocessed animation (S56: YES), the control unit 11 returns to the process of step S51 and executes the processes of steps S51 to S55 for the unprocessed animation. If it is determined that there is no unprocessed animation (S56: NO), the control unit 11 moves to the process of step S57.

[0100] Fig. 10A The screen shown is constructed in such a way that the user receives instructions for playing the animation of each main factor of each work element. For example, the user can perform a predetermined operation (such as left clicking the mouse) on any main factor of any work element to instruct the user to play the animation of the operated main factor. Fig. 10A In the process, the control unit 11 receives an instruction to play the animation of the main factor of the omitted action occurring for the work element with the loop number 1 by operating the cursor as indicated by the arrow A5. The control unit 11 determines whether the main factor to be played is received (S57), and when it is determined that it is received (S57: YES), it determines the animation of the selected main factor occurring (the animation corresponding to the main factor) (S58). Here, the control unit 11 retrieves the animation of the main factor of the selected work element from the animations stored in the storage unit 12. Then, the control unit 11 reads the retrieved animation from the storage unit 12 and displays it on the display unit 15 (S59). For example, the control unit 11 displays the animation as shown in the following figure: Fig. 10B The analysis result screen shown is displayed on the display unit 15 . Fig. 10B The screen shown has Figure 8 The control unit 11 is configured to play the plurality of animations determined in step S58 in sequence, and the "go to previous animation" button is pressed to select the next animation. Fig. 10B When the screen shown is operated, the display content of the animation area A3 and the result area A4 is changed to the animation displayed previously, and the analysis result of the animation. In addition, when the "Go to next animation" button is operated, the control unit 11 changes the display content of the animation area A3 and the result area A4 to the animation to be displayed next, and the analysis result of the animation. It should be noted that the control unit 11 can be configured to play the multiple animations determined in step S58 in an arbitrary order. For example, the animations can be played in order from old to new or from new to old according to the recording date and time, and the animations of each operator can be played collectively according to each operator, or they can be played randomly. In addition, it can be Fig. 10BA plurality of animation areas A3 are provided in the illustrated screen, and a cyclic animation in which a main factor selected in each animation is occurring is synchronously displayed in each animation area A3.

[0101] When the control unit 11 determines that no main factor to be played is received (S57: NO), the processing of steps S58 to S59 is skipped and the above processing is terminated. Through the above processing, each cyclic animation included in the animation can be grouped according to each main factor that is considered not to be a standard operation, and the occurrence ratio of each main factor can be analyzed. In addition, the animation of each main factor occurring can be played according to each main factor, so that the improvement strategy can be studied efficiently according to each cyclic animation (operation element).

[0102] In this embodiment, for each animation, the analysis result of whether it is a standard operation can be obtained according to each cycle (operation element), and the analysis result of whether it is a standard action can be obtained according to each action included in each cycle. Therefore, in addition to the analysis results of each cycle animation, animation retrieval can also be performed based on the analysis results of each action animation. For example, for a predetermined cycle (for example, the first operation element), animations that are judged as predetermined actions (for example, the first action) that are not standard actions can be retrieved. Therefore, when verifying the work content of each operator, a retrieval process that takes into account the analysis results of each cycle and the analysis results of each action can be performed, and efficient verification processing can be achieved.

[0103] Next, a description will be given of a process of analyzing the variation in motion time of motion included in each work element based on the analysis result of each motion animation included in the animation. Fig.11 is a flowchart showing an example of the analysis and processing procedure of the action time of each action. Fig.12 The following processing is executed by the control unit 11 of the information processing device 10 according to the program 12P stored in the storage unit 12.

[0104] When analyzing the deviation of the action time of each action performed by the operator, the user selects an animation to be analyzed. Figure 7 Here, too, the animation of the processing object is an animation obtained by filming the workers who are assembling a product of one type. It can be multiple animations obtained by filming one worker, or it can be animations obtained by filming multiple workers.

[0105] The control unit 11 measures the action time of each action based on each action animation included in the animation of the processing object (S71). For example, the control unit 11 obtains the action number, start time, and end time (time of each boundary) of each action animation from each animation. Then, the control unit 11 calculates the time from the start time to the end time for each action animation. The control unit 11 generates a scatter diagram (chart) by associating the action time calculated for each action with each action and plotting it, and displays it on the display unit 15 (S72). For example, the control unit 11 generates a scatter diagram (chart) as shown in FIG. Fig.12 The scatter plot shown in the upper right corner of the screen is displayed. Fig.12 The scatter diagram shown in the upper right corner of the figure shows the deviation of the action time for each action included in the work element "assembly of box A" included in the assembly work of variety A. Specifically, each action is associated with each position on the horizontal axis, the vertical axis represents the action time of each action, and the action time of each action is associated with each position on the horizontal axis and drawn. It should be noted that the control unit 11 measures the standard action time of each action based on each action animation in the animation obtained by shooting the operator performing the standard operation used as the setting animation of the action analysis model M4, and draws the measured standard action time of each action with a white diamond on the scatter diagram. In this way, the standard action time of each action can be prompted, and the comparison result of the action time of each operator relative to the standard action time can be easily grasped.

[0106] It should be noted that, when the control unit 11 measures the action time of each action for a loop animation, it adds up the action time of each action to calculate the action time (loop time) in the work element. Fig.12 As shown in the upper left of the screen, the control unit 11 also generates and displays a scatter diagram of the calculated cycle time. In this way, in addition to the deviation of the operation time of each operation, the deviation of the cycle time required for the work element (cycle) can also be prompted.

[0107] Next, the control unit 11 calculates the minimum value, the first quartile, the median value, the third quartile, and the maximum value of the action time for each action based on the action time measured in step S71 (S73). Then, the control unit 11 generates a box plot (chart) showing the deviation of the action time of each action based on the values ​​calculated for each action, and displays it on the display unit 15 (S74). For example, the control unit 11 generates the following: Fig.12 The box plot shown in the lower right corner of the screen is displayed. Fig.12The box plot shown in the lower right corner of is similar to the scatter plot, associating each action with each position on the horizontal axis, and the vertical axis represents the action time of each action. The deviation of the action time is represented by boxes and lines representing the minimum value, first quartile, median value, third quartile, and maximum value of the action time of each action, respectively. Here, the control unit 11 draws the standard action time of each action with white diamonds on the box plot. In addition, regarding the cycle time, the control unit 11 calculates the minimum value, first quartile, median value, third quartile, and maximum value, and based on the calculated values, Fig.12 As shown in the lower left corner of the screen, a box plot showing the deviation of the cycle time is generated and displayed on the display unit 15.

[0108] The control unit 11 determines whether there is an unprocessed animation that has not been processed in steps S71 to S74 (S75). If it is determined that there is an unprocessed animation (S75: YES), the control unit 11 returns to the process of step S71 and executes the processes of steps S71 to S74 for the unprocessed animation. If it is determined that there is no unprocessed animation (S75: NO), the control unit 11 moves to the process of step S76.

[0109] Fig.12 The scatter diagram in the screen shown is configured to receive instructions for playing the animation corresponding to each mark through each drawn point (mark). For example, the user can instruct to play the animation corresponding to the operated mark by performing a predetermined operation on any mark (such as left clicking the mouse). Fig.12 In the above, the control unit 11 receives an instruction to play the animation corresponding to the selected mark by operating the cursor as shown by the arrow A6. The control unit 11 determines whether the mark to be played is received (S76), and when it is determined that it is received (S76: YES), it determines the animation corresponding to the selected mark (S77). In the scatter diagram, the mark drawn in association with each action is associated with the identification information of the animation (for example, the animation ID assigned to the animation), so the control unit 11 can determine the animation corresponding to the mark by determining the identification information of the animation corresponding to the selected mark. Thus, the control unit 11 can determine the animation to be played selected via the mark in the scatter diagram from the animations stored in the storage unit 12.

[0110] Then, the control unit 11 reads the determined animation from the storage unit 12 and displays it on the display unit 15 (S78). In this case, the control unit 11 displays the determined animation as follows: Figure 8The analysis result screen shown. It should be noted that the control unit 11 may be configured to display the action animation corresponding to the selected mark in the determined animation. In this case, the action animation of the selected action can be confirmed. When the control unit 11 determines that the mark to be played is not received (S76: NO), it skips the processing of steps S77 to S78 and ends the above processing. Through the above processing, for each cyclic animation included in the animation, the deviation of the action time of each action included in the cyclic animation can be prompted together with the deviation of the cycle time using a scatter plot and a box plot. In addition, the animation selected by each mark of the scatter plot can be played. Thereby, the deviation of the cycle time of each work element and the deviation of the action time of each action can be considered, and the improvement strategy can be studied according to each work element or according to each action.

[0111] Next, a description will be given of a process of creating a Gantt chart showing the progress status of each work element performed by a worker in the animation using the result of segmenting the animation into loop animations. Fig.13 This is a flowchart showing an example of the process sequence for creating a Gantt chart. Fig.14 The following processing is executed by the control unit 11 of the information processing device 10 according to the program 12P stored in the storage unit 12.

[0112] When creating a Gantt chart showing the progress of each work element performed by the operator, the user selects an animation to be processed. The animation to be processed here is an animation that is classified as a cyclic animation using the cyclic prediction model M2, and can also be, for example, Figure 7 The processing is performed on the animation after the analysis processing of each work element. The control unit 11 receives the selection of the animation to be made into the Gantt chart through the operation of the input unit 14 (S81). The control unit 11 obtains the start time and end time of each loop animation distinguished from the selected animation (S82). For the animation distinguished as a loop animation, the start time and end time of each loop animation (the time of the boundary of each loop) are associated with the animation and stored in the storage unit 12, and the control unit 11 reads the start time and end time of each loop animation from the storage unit 12.

[0113] The control unit 11 creates a strip chart (horizontal bar) representing the execution time period from the start timing of each work element to the end timing of the work based on the acquired start time and end time of each loop animation, and creates a Gantt chart (chart) by arranging the created strip charts of each work element (S83). For example, the control unit 11 creates the following Fig.14 Gantt chart shown. Fig.14The Gantt chart shown displays the names of the work elements included in the assembly work of the product A in the vertical direction, the horizontal axis represents the date and time, and a strip chart representing the execution time period of each work element is displayed in association with each work element. The control unit 11 displays the prepared Gantt chart on the display unit 15 (S84). With such a Gantt chart, the progress of the work performed by each operator can be easily grasped. Fig.14 The Gantt chart shown can be configured to display, for example, a strip chart showing the start time and end time of each action included in the selected work element when each work element is selected. In this case, a Gantt chart can be provided that can indicate the progress of each action included in each work element in addition to the progress of each work element. It should be noted that when a Gantt chart is produced based on an animation obtained by filming a worker performing a standard operation, a process table (standard operation combination ticket) that specifies the sequence of each work element included in a series of operations and the standard operation time can be produced.

[0114] Next, a process of displaying the analysis results of a plurality of animations in parallel will be described. Hereinafter, the configuration of a screen displaying the analysis results of two animations in parallel will be described as an example, but the configuration of a screen displaying the analysis results of three or more animations in parallel may also be used. Fig.15 1 is an explanatory diagram showing an example of displaying two analysis result screens. The control unit 11 of the information processing device 10 receives an instruction to execute a process of comparing the analysis results of two animations, and when receiving the selection of two animations that have been analyzed, generates analysis result screens for each animation based on the selected animation and the analysis results of the animation, and displays them side by side on the display unit 15. Fig.15 In the example shown, the analysis result screens of each animation are displayed on the left and right sides of the screen. Figure 8 The same structure is used, but the result area A4 is displayed below the animation area A3. Fig.15 The result area A4 in the display is configured to display information related to the motion animation currently being displayed in the animation area A3 and the motion animations before and after it. As the animation in the animation area A3 is played, the analysis results of the motion animation being displayed are sequentially displayed.

[0115] exist Fig.15 The screen shown includes: a "Start Alignment" button for moving the playback position of the two animations displayed in the animation area A3 to the beginning position, a simultaneous stop button for stopping the two animations together, and an action comparison mode button for synchronizing and displaying the action animations in the two animations. Fig.15The screen shown is provided with an animation change button for instructing to change the animation displayed in each animation area A3. When the animation change button is operated, the control unit 11 of the information processing device 10 displays the animation names of the selectable animations in a list, and when receiving the selection of any animation, switches the display contents of the animation area A3 and the result area A4 based on the selected animation and the analysis result of the animation. In this way, the analysis result of any animation can be compared and displayed.

[0116] When the action comparison mode button is operated, the control unit 11 can synchronize (link) each action contained in each animation and display it in the animation area A3. Thus, the action content of the operator in the two animations can be compared for each action. It should be noted that the control unit 11 can also synchronize each loop animation contained in each animation and display it in the animation area A3 in sequence according to each action from the beginning. Thus, the control unit 11 can prompt the analysis results of any two animations in parallel, and can display the action animation or loop animation according to each action or according to each operation element. Thus, for example, when the animations shot when different operators perform the same type of assembly operation are displayed synchronously, the operation status of the two operators can be compared. In addition, when the animations shot when an operator performs the same type of assembly operation are displayed synchronously, the operation status of an operator, for example, on different dates, can be compared.

[0117] In this embodiment, based on the animation obtained by filming the operator performing the operation, it is determined whether each action performed by the operator is a standard action, and based on the determination result of each action, it is determined whether each operation element is a standard operation. In addition, since the determination of whether each action is a standard action is performed using the action analysis model M4, the action content of each action (the activity of the operator) is also considered to determine whether the standard action is performed with high accuracy. By using the results of such high-precision determination, appropriate improvement measures can be studied, and improvement guidance can be provided to the operator.

[0118] In this embodiment, Figure 7 The analysis process shown can be executed in real time based on the animation obtained by filming the operator in the process of operation, in addition to the configuration of executing the analysis process after the operation is completed. In this case, for the assembly operation being performed by the operator, it is determined whether each action is a standard operation, and based on the determination result, it is possible to determine whether each operation element is a standard operation, and to provide improvement guidance during the execution of the operation.

[0119] In the present embodiment, at least one of the prediction processing of varieties using the variety prediction model M1, the differentiation processing of cyclic animations using the cyclic prediction model M2, the differentiation processing of action animations using the boundary prediction model M3, and the analysis processing using the action analysis model M4 is not limited to being performed locally by the information processing device 10. For example, a server that performs the prediction processing of varieties using the variety prediction model M1 may be provided. In this case, the information processing device 10 is configured to send the animation of the processing object to the server, and the variety predicted by the server using the variety prediction model M1 is sent to the information processing device 10. In this case as well, the information processing device 10 can perform, for example, based on the variety predicted by the server. Figure 7 The processing after step S33.

[0120] Alternatively, a server may be provided to perform a process of distinguishing looping animations using the loop prediction model M2. In this case, the information processing device 10 is configured to send the animation of the processing object to the server, and the animation distinguished as a looping animation by the server is sent to the information processing device 10. In this case as well, the information processing device 10 can perform, for example, based on the animation distinguished as a looping animation by the server. Figure 7 In addition, a server that performs the distinguishing process of the action animation using the boundary prediction model M3 may be provided. In this case, the information processing device 10 is configured to send the animation of the processing object to the server, and the animation distinguished as the action animation by the server is sent to the information processing device 10. In this case as well, the information processing device 10 can perform, for example, based on the animation distinguished as the action animation by the server. Figure 7 In addition, a server that performs analysis processing using the motion analysis model M4 may be provided. In this case, the information processing device 10 is configured to send the animation of the processing object to the server, and the result of the analysis processing of the animation performed by the server is sent to the information processing device 10. In this case as well, the information processing device 10 can perform, for example, based on the result of the analysis processing performed by the server. Figure 7 Even in the case of the above-described configuration, the same processing as in the present embodiment can be performed and the same effects can be obtained.

[0121] In this embodiment, whether the actions and operations performed by the operator are standard actions and the composition of standard operations is analyzed based on the animation obtained by filming the operator performing the operation, but the analysis object is not limited to the operator. For example, it is also possible to analyze whether the actions and operations of the robot are standard actions and the composition of standard operations based on the animation obtained by filming the robot configured to perform a predetermined operation.

[0122] (Implementation Method 2)

[0123] In this embodiment, the following information processing device is described: the information processing device extracts a looping animation of each work element from an animation obtained by filming a worker repeatedly performing a work element (the same work element), compares the extracted looping animations and provides a prompt. It should be noted that the looping animation included in an animation can be obtained by filming a single worker or by filming multiple different workers. The information processing device 10 of this embodiment has the same Figure 2 The configuration of the information processing device 10 of the first embodiment is the same as that of the information processing device 10 shown in the embodiment 1, and therefore, the description of the configuration is omitted. It should be noted that the storage unit 12 of the information processing device 10 of the present embodiment stores a loop animation extraction model M6 instead of the loop prediction model M2.

[0124] Fig.16 This is an explanatory diagram showing an example of the configuration of the loop animation extraction model M6. The loop animation extraction model M6 is configured using algorithms such as RNN, LSTM, CNN, and Transformer, and can also be configured by combining multiple algorithms. The loop animation extraction model M6 is a learning model that has been learned in the following manner: an animation obtained by filming a worker who repeatedly performs a work element, and the start time and end time of a loop animation specified for the animation are taken as input, and a loop animation similar to the specified loop animation is extracted from the input animation, and the calculation result is output. The loop animation extraction model M6 can also be configured to input an animation that captures the movements of the worker, such as an animation obtained by filming the whole body, upper body, or hands of the worker.

[0125] The loop animation extraction model M6 has: an input layer that inputs an animation and the start time and end time of a loop animation in the animation; an intermediate layer that extracts the feature quantity of the operator's action contained in the loop animation with the start time and end time input, and the feature quantity of the operator's action contained in the animation, and performs calculations to extract a loop animation having the same feature quantity as the loop animation (a loop animation similar to the loop animation) from the input animation; and an output layer that outputs the start time and end time of each loop animation extracted from the animation based on the calculation results of the intermediate layer. The start time and end time of each loop animation output from the loop animation extraction model M6 represent the elapsed time from the start of the animation playback.

[0126] The loop animation extraction model M6 can be generated by machine learning using the following training data: the training data includes an animation for training and the start time and end time of a loop animation in the animation, and the start time and end time of multiple loop animations that are designated as correct for the animation. The loop animation extraction model M6 is learned in the following manner: when the start time and end time of the animation and a loop animation included in the training data are input, the start time and end time of each loop animation as the correct one is output. In the learning process, the loop animation extraction model M6 performs operations in the intermediate layer and the output layer based on the input animation and the start time and end time of a loop animation, extracts the features of the animation and the action of the operator included in the loop animation, extracts the loop animation having the same features as the extracted loop animation from the animation of the processing object, compares the start time and end time of each extracted loop animation with the start time and end time of each correct loop animation, and optimizes the parameters used for the operation processing in the intermediate layer and the output layer in a manner that makes the two approximate. The optimized parameters are the weights (coupling coefficients) between the nodes in the intermediate layer and the output layer, the coefficients of the function, the thresholds, etc., and the parameter optimization method can utilize the back error propagation method, the gradient descent method, etc. Thus, a cyclic animation extraction model M6 is obtained, which distinguishes each cyclic animation in the animation and outputs the start time and end time of each cyclic animation when the start time and end time of the animation and a cyclic animation are input. The learning of the cyclic animation extraction model M6 can also be performed using other learning devices. In this case, the learned cyclic animation extraction model M6 generated by other learning devices is downloaded from the learning device to the information processing device 10 via the network N or via the mobile storage medium 10a, and stored in the storage unit 12.

[0127] Next, the following processing is described: in the information processing device 10 of this embodiment, loop animations obtained by photographing workers performing work elements are extracted from the animations obtained by photographing workers, and the extracted plurality of loop animations are compared and presented. Fig.17 is a flowchart showing an example of the comparison processing sequence of the loop animation. Figures 18A to 21B It is an explanatory diagram showing a screen example.

[0128] When confirming the work performed by the operator using the loop animation, the user selects the animation to be processed. The control unit 11 of the information processing device 10 selects the animation to be processed. Fig.18A The animation selection screen shown is displayed on the display unit 15, and the selection of any animation is accepted from the screen by operation via the input unit 14 (S91). Here, one animation can be selected, or multiple animations can be selected. Fig.18AIn the example, animation 1 and animation 2 are selected. The animation of the processing object takes the time for the operator (object) to perform a series of work elements as a loop animation, which includes multiple loop animations. It should be noted that Fig.18A The screen is configured such that, by selecting a camera, thumbnails of animations shot with the selected camera are displayed.

[0129] Next, the user specifies the start time and end time of a loop animation for the animation selected as the processing object. The control unit 11 displays, for example, Fig.18B The screen shown in the figure is displayed, and the designation of the start time and the end time of a loop animation is accepted from the screen (S92). Fig.18B The screen has: a display bar for the selected animation, and an input bar R1 for inputting the start time and end time of a loop animation in the displayed animation. An indicator showing the playback time of the displayed animation is displayed in the input bar R1, and the user specifies the start time and end time of a loop animation by moving the start mark C1 and the end mark C2 set in the input bar R1 to the desired playback time. At this time, the user can use the enlargement and reduction buttons C3 to enlarge or reduce the interval of the playback time shown by the indicator.

[0130] A "Find Similar Jobs" button is provided in the input field R1. The "Find Similar Jobs" button is used to instruct to extract a loop animation similar to the loop animation from the animation to be processed based on a loop animation with a specified start time and end time. The control unit 11 determines whether the "Find Similar Jobs" button is operated (S93). If it is determined that it is not operated (S93: NO), it returns to step S92 to continue to accept the input of the start time and end time of a loop animation. If it is determined that the "Find Similar Jobs" button is operated (S93: YES), the control unit 11 extracts a loop animation similar to the loop animation specified in step S92 from the animation selected in step S91 (S94). Here, the control unit 11 inputs the selected animation (animation file 12a) and the start time and end time of the specified loop animation into the loop animation extraction model M6, and obtains the start time and end time of each loop animation extracted from the animation to be processed as output information from the loop animation extraction model M6.

[0131] The control unit 11 displays the start time and end information of each loop animation extracted from the animation (S95). Fig. 18CAs shown, the control unit 11 displays each looping animation with a rectangle C4 indicating the start time and the end time according to the playback time of the animation indicated by the indicator in the input field R1, and displays the start time and the end time of each looping animation in the animation information field R2. In this embodiment, the control unit 11 gives the names of video 1, video 2, etc. to each looping animation extracted from the animation. Fig. 18C Video 1, video 2, etc. displayed in the animation information column R2 show each loop animation.

[0132] The control unit 11 extracts one of the plurality of loop animations extracted from the automatic animation (S96), and uses the boundary prediction model M3 to classify the extracted loop animation as an action animation (S97). The processing of step S97 is similar to that of step S98. Figure 7 Similar to step S35 in the above, the control unit 11 sequentially inputs each frame included in the loop animation into the boundary prediction model M3 corresponding to the type of work being done by the worker in the animation, and determines (predicts) the start time and end time of each action animation included in the loop animation based on the output information from the boundary prediction model M3. After the processing of step S97, the control unit 11 can perform editing processing for changing (deleting or adding a new one, etc.) the boundary (dividing position) of each action animation (refer to Figure 6A to Figure 6C ). It should be noted that, in the present embodiment, the following setting is performed: when the boundary prediction model M3 is used to distinguish the loop animation into action animations, the action animations are distinguished into a predetermined number (e.g., 10). The number of distinctions of the action animations may be predetermined and set in the boundary prediction model M3, or may be arbitrarily set by the user, in which case the set number of distinctions may be input as input data of the boundary prediction model M3.

[0133] The control unit 11 determines whether the processing of steps S96 to S97 has been completed for all loop animations extracted from the animation in step S94 (S98). If it is determined that it has not been completed (S98: NO), it returns to step S96 and repeats the processing of steps S96 to S97 for the unprocessed loop animations. Thus, all loop animations are divided into multiple action animations. If it is determined that the processing of all loop animations has been completed (S98: YES), the control unit 11 displays the analysis information of each loop animation (S99). For example, the control unit 11 counts the required time of each loop animation (the loop time from the start time to the end time), generates a scatter plot (chart) of the loop time of each loop animation, and a box plot showing the minimum value, first quartile, median value, third quartile, and maximum value of the loop time, and displays the analysis information as shown in FIG. Fig.18D The analysis information column R3 is shown as shown. Fig.18DIn the scatter diagram, the vertical axis (vertical direction) represents the cycle time. Each point (mark) drawn in the scatter diagram is configured to display the cycle time of the loop animation corresponding to the operated mark when a predetermined operation (such as selection with a cursor) is performed, and is configured to start playing the corresponding loop animation when a predetermined operation (such as left clicking of a mouse) is performed. Fig.18D In the example of FIG. 5 , the control unit 11 displays the loop time of the loop animation corresponding to the mark selected by the cursor indicated by the arrow C5.

[0134] Fig. 18C The animation information column R2 in the screen is configured to rearrange the display order of the loop animation extracted from the animation to be processed, and is provided with a rearrangement button C6 for instructing execution of the rearrangement. Fig.19A As shown, the rearrangement button C6 is provided with a pull-down menu that allows selection of any one of the start time order, the order of jobs from fast to slow, or the order of jobs from slow to fast as the rearrangement rule. Fig.19B In the example, the rearrangement rule of the operation from slow to fast is selected, and the name, start time, end time, and cycle time of each loop animation are displayed in the order from long to short cycle time.

[0135] exist Fig.19A and Fig.19B The animation information column R2 in the screen is provided with a check box for selecting each loop animation, and is configured so that when two or more loop animations are selected, Fig.19B The "Concentrated Comparison" button is displayed as shown. After the user selects the looping animation that he wants to compare and confirm with the check box, he operates the "Concentrated Comparison" button. The control unit 11 accepts the selection of the looping animation of the analysis object by checking the check box in the animation information column R2 (S100), and when more than two looping animations are selected, it determines whether the "Concentrated Comparison" button is operated (S101). When it is determined that the "Concentrated Comparison" button is not operated (S101: NO), the control unit 11 returns to step S100 and continues to accept the selection of the looping animation.

[0136] When it is determined that the "Concentrated Comparison" button has been operated (S101: YES), the control unit 11 determines the baseline loop animation (baseline animation) that should be used as a model from the loop animation selected in step S100 (S102). For example, the control unit 11 determines the loop animation whose loop time is the central value as the baseline loop animation. It should be noted that the control unit 11 can also determine the loop animation whose loop time is closest to the average value of the loop times of each loop animation as the baseline loop animation, and can also determine the loop animation with the shortest loop time as the baseline loop animation. In addition, the control unit 11 can also determine the loop animation selected by the user via the input unit 14 as the baseline loop animation. In addition, the control unit 11 can also calculate the similarity of each selected loop animation with other loop animations, for example, determine the loop animation with the highest total similarity with other loop animations as the baseline loop animation. The similarity can be calculated using a correlation coefficient or a cosine similarity. In addition, the control unit 11 can be configured to use a learning model constructed by machine learning to infer the similarity of two loop animations. For example, a learning model composed of CNN and learned in a manner that outputs the similarity of two loop animations when two loop animations are input can be used. In this case, the control unit 11 can input the two loop animations into the learned learning model, and infer the similarity of the two loop animations based on the output information from the learning model. In addition, the control unit 11 is not limited to determining the composition of the baseline loop animation from the selected loop animation, and can also determine the baseline loop animation from the loop animation extracted in step S94. In this case, the control unit 11 can determine the central value, average value, or shortest loop animation of the loop time from the loop animation extracted in step S94 as the baseline loop animation, and can also determine the loop animation with the largest total similarity with other loop animations as the baseline loop animation.

[0137] The control unit 11 generates a comparison screen for comparing the reference loop animation and the loop animation of the analysis object (S103). Fig.19C The comparison screen shown in FIG. 1 is displayed on the display unit 15 (S104). Fig.19CIn the screen, the looping animation of video 115 is displayed as a baseline looping animation (template animation), and the looping animation of video 2 is displayed as an analysis object. Specifically, the baseline looping animation (video 115) is displayed on the left side of the animation display bar R4, and the looping animation of the analysis object (video 2) is displayed on the right side. It should be noted that the displayed baseline looping animation and the looping animation of the analysis object are configured so that they can be changed from the looping animation selected in step S100 to any looping animation by operation via the input unit 14. Specifically, a switching button C7 for the baseline looping animation and a switching button C8 for the analysis object are provided in the animation display bar R4, and a drop-down menu for selecting any one of the selectable looping animations is provided in the switching buttons C7 and C8. In addition, the looping animation of the analysis object is provided with a menu that can be used to select, for example, the looping animation of the analysis object. Fig.19B The order displayed in the animation information bar R2 is switched by the switch button C9.

[0138] exist Fig.19C The analysis information column R3 in the screen displays a graph D1 for the reference cycle animation and the cycle animation of the analysis object displayed in the animation display column R4. The graph D1 shows the action animations (in Fig.19C 1, interval 2...) in the figure. Graph D1 associates each action (each interval) with each position on the horizontal axis, and the vertical axis represents the difference between the required time of each action animation of the cyclic animation of the analysis object and the required time of each action animation of the benchmark cyclic animation (interval 1, interval 2...). In Graph D1, the required time of each action animation of the benchmark cyclic animation is drawn with a white circle, and the required time of each action animation of the cyclic animation of the analysis object is drawn with a black circle. Through Graph D1, the benchmark cyclic animation and the cyclic animation of the analysis object can be compared in action animation units (interval units), and it is possible to easily grasp whether the required time of the cyclic animation of the analysis object is long or short (that is, whether each action is fast or slow) based on the benchmark cyclic animation. In addition, the analysis information column R3 displays a scatter plot obtained by plotting the cycle time of each cyclic animation selected in step S100, and a box plot showing the minimum value, first quartile, median value, third quartile, and maximum value of the cycle time. Fig.19C The scatter plot plots the cycle time of the baseline looping animation with white circles, the cycle time of the looping animation of the analysis object with large black circles, and the cycle time of other looping animations with small black circles.

[0139] In addition, the analysis information column R3 is shown below the graph D1 and the scatter diagram of the cycle time. Fig. 20AAs shown, a scatter plot and a box plot of the time required for each action animation (each interval) in the cyclic animation selected in step S100 are displayed. The scatter plot here represents each interval on the horizontal axis and the time required for each interval on the vertical axis. The required time of the baseline cyclic animation is plotted with a white circle, the required time of the cyclic animation of the analysis object is plotted with a large black circle, and the required time of other cyclic animations is plotted with a small black circle. In this way, the deviation of the required time for each interval of the selected cyclic animation including the baseline cyclic animation can be prompted. In addition, the analysis information column R3 displays a cumulative chart D2 obtained by cumulatively displaying the required time of each action animation (each interval) for each cyclic animation selected in step S100. The cumulative chart D2 shows a bar chart obtained by accumulating the required time of each action animation (interval 1, interval 2...) for each cyclic animation in sequence from the bottom. It should be noted that the cumulative chart D2 displays the bar chart of the baseline cyclic animation, the bar chart of the cyclic animation of the analysis object, and the bar chart of other cyclic animations in sequence from the left. Fig. 20A In the example, only the required time for interval 6 is marked with a hatched line, but the required time for each interval is displayed in the same color (display method). This makes it easy to understand the length of time required for each action animation for each looping animation. Each interval in the cumulative chart D2 is configured to display the required time for the corresponding looping animation interval when a predetermined operation (such as selection using the cursor) is performed, and is configured to start playback of the corresponding looping animation interval (action animation) when a predetermined operation (such as left-clicking the mouse) is performed. Fig. 20A In the example of FIG. 1 , section 6 of video 10 is selected, and the required time of the selected section 6 is displayed.

[0140] like Fig.19C and Fig. 20A As shown, the comparison screen also has a section comparison column R5, in which the horizontal axis represents the playback time of the loop animation, and shows the start time and end time of each action animation (each section) in the base loop animation and the loop animation to be analyzed. Fig.19C and Fig. 20A In the example, the start time of the baseline loop animation and the loop animation of the analysis object are made consistent, the start time and end time of each action animation are shown, and a mark C10 indicating the playback position of the two loop animations in the animation display bar R4 is added. The comparison screen of this embodiment is configured so that the baseline loop animation and the loop animation of the analysis object are displayed (played) synchronously (linked) according to each action animation (each interval) obtained by distinguishing each loop animation. For example, in Fig.19C and Fig. 20AIn the example, the time required for interval 1 of video 115 is longer than the time required for interval 1 of video 2. Therefore, when the two looping animations are played, video 2 waits until the playback of interval 1 of video 115 is finished after the playback of interval 1 is finished. When the playback of interval 1 of video 115 is finished, the playback of interval 2 of the two looping animations starts. Fig. 20B The section comparison column R5 shows the state of the time when the playback of the section 1 of the two loop animations ends, and the mark C10 indicates the end time of the section 1 of each loop animation. Then, the playback of the section 2 of each loop animation starts.

[0141] In addition, each interval displayed in the interval comparison column R5 is configured so that when a predetermined operation (such as a left click of a mouse) is performed, the start time of the operated (selected) action animation (interval) is made consistent, and the start time and end time of each action animation are displayed. Fig. 20C In the example of , interval 4 is selected, the start time of interval 4 is made consistent, and the start time and end time of each interval of the two loop animations are displayed. Thus, the length of the required time of the selected interval can be easily grasped. At this time, the action animation of the selected action (interval) in the two loop animations can be looped (repeatedly played) in the animation display bar R4. In addition, when an arbitrary interval is selected via the interval comparison bar R5, the action animation in the two loop animations is synchronously displayed in the animation display bar R4 for the selected interval. Thus, the actions of the workers in the two loop animations can be compared for each interval. In addition, the comparison screen is configured so that when the two loop animations are displayed in the animation display bar R4, when the switching instruction of the analysis object is given by the switching buttons C8 and C9, the base loop animation and the loop animation of the analysis object instructed to be switched are played from the same interval as the interval (action) being displayed at that moment. In the case of the switching instruction of the base loop animation by the switching button C7, the base loop animation instructed to be changed and the loop animation of the analysis object being displayed are played from the same interval as the interval being displayed at that moment.

[0142] In addition, each interval in the interval comparison column R5 is configured so that, when a predetermined operation (such as a double-click of a mouse) is performed, the action animation of the operated (selected) interval is further differentiated into detailed action animations. It should be noted that the action animation obtained by a single differentiation process on the cyclic animation of the processing object is used as the action animation of each action of the large category, the action animation of the action of the large category is used as the action animation of each action of the medium category, and the action animation of the action of the medium category is used as the action animation of each action of the small category. Fig. 20CIn the example, interval 4 is selected. In this case, the control unit 11 takes the action animation of the selected interval 4 (the action animation of the large category) as the processing object, performs the same processing as step S97, and uses the boundary prediction model M3 to distinguish the action animation of interval 4 into action animations of actions of the medium category (interval 4-1, interval 4-2...). Here, the control unit 11 sequentially inputs each frame included in the action animation of interval 4 into the boundary prediction model M3, and determines (predicts) the start time and end time of the action animation of each action of the medium category included in the action animation of interval 4 based on the output information from the boundary prediction model M3. Here too, the control unit 11 can set or input the number of distinctions (for example, 5) of the action animations of the large category actions into the action animations of the medium category actions, and can also edit the boundaries of the action animations of each action of the medium category after distinction. Fig.20D The interval comparison column R5 shows the display state of the action animation (interval 4-1, interval 4-2, etc.) of the action classified in the above. Fig.20D When any interval (e.g., interval 4-1) in the interval comparison column R5 is selected, the control unit 11 takes the action animation (the action animation of the middle classification) of the selected interval 4-1 as the processing object, performs the same processing as step S97, and uses the boundary prediction model M3 to divide the action animation of the interval 4-1 into action animations of small classifications (not shown). The user can divide the loop animation into action animations in stages by performing such processing multiple times, and can perform analysis processing using action animations with the desired precision. In addition, only the interval that the user wants to confirm can be divided into more detailed action animations, so unnecessary division processing can be suppressed.

[0143] The control unit 11 displays Fig.19C and Fig. 20AAfter the comparison result is shown, for example, when the switching of the reference loop animation is indicated by the switching button C7, the reference loop animation is changed to the selected other loop animation, and the content of the comparison screen is updated. In addition, when the switching of the loop animation of the analysis object is indicated by the switching buttons C8 and C9, the control unit 11 changes the analysis object to the selected other loop animation, and updates the content of the comparison screen. In addition, when the control unit 11 receives the selection of each mark drawn in the graph D1 displayed in the analysis information column R3, the scatter diagram of the cycle time of each loop animation, or the scatter diagram of the required time of each action animation (each interval), the required time or cycle time corresponding to the selected mark is displayed. Moreover, when the control unit 11 receives the selection of each interval shown in the cumulative graph D2 displayed in the analysis information column R3, the required time of the selected interval is displayed, and when the playback of the interval is indicated, the action animation of the interval is displayed in the animation display column R4.

[0144] The control unit 11 determines whether to terminate the above-mentioned processing (S105). For example, the control unit 11 determines to terminate the processing when receiving an instruction to terminate the above-mentioned processing from the user via the input unit 14. When the control unit 11 determines not to terminate the above-mentioned processing (S105: NO), each time an operation on the comparison screen is received, the control unit 11 stands by while performing a process corresponding to the received operation. When the control unit 11 determines to terminate the above-mentioned processing (S105: YES), the control unit 11 stores the comparison analysis result displayed on the comparison screen in the storage unit 12 (S106), and terminates a series of processing. For example, the control unit 11 stores the animation selected in step S91, the start time and end time of the loop animation extracted from each animation in step S94, and the start time and end time of the action animation obtained by distinguishing each loop animation in step S97 in a folder set in the storage unit 12. In this way, by reading out the folder storing a series of comparison analysis results, the comparison analysis results can be confirmed at any time, and further comparison analysis processing can be performed by further operating the displayed comparison screen. It should be noted that not only a series of comparative analysis results can be stored in each folder, but also each cyclic animation extracted from the animation of the processing object can be stored in each folder.

[0145] Through the above processing, in this embodiment, a plurality of loop animations are automatically extracted (divided) from the animation obtained by filming the operator based on the designated loop animation, and each loop animation is automatically distinguished according to each action performed by the operator. In addition, among the plurality of loop animations selected as analysis objects from the loop animations extracted from the automatic animation, a reference loop animation to be used as a model is determined, and the comparison results between the reference loop animation and the other loop animations are prompted. By synchronously playing the reference loop animation and the other loop animations in units of action animations when displaying them, comparison in units of action animations becomes easy.

[0146] like Fig. 20A As shown, a favorite button C11 for instructing the display of a favorite screen is provided in the comparison screen. The favorite screen can read the comparison analysis results generated by the above-mentioned processing and stored in the folder of the storage unit 12 by one click. When the favorite button C11 is operated, the control unit 11 displays the following Fig.21A The collection screen R6 shown. The collection screen R6 is provided with: a comparison tab for displaying a selection screen for accepting selection of the comparison analysis result stored in the storage unit 12; a video tab for displaying a selection screen for accepting selection of an animation obtained by photographing the operator with a camera; and an all tab for displaying a selection screen for accepting selection of either the comparison analysis result or the animation. Fig.21A In the example, all tags are selected, and the comparison analysis results and animation selection screen are displayed. Fig.21A In the collection screen of the screen, for example, when "Comparative Analysis - Video 115 × Animation 1" is selected, the control unit 11 reads the corresponding comparative analysis result from the storage unit 12 and stores it in the storage unit 12. Fig.19C and Fig. 20A The comparison screen shown is displayed on the display unit 15. By using the favorites screen in this way, a desired comparison analysis result or animation can be read and displayed with one click.

[0147] Fig.19C , Fig. 20A and Fig.21A The animation display bar R4 in the screen is a structure that displays only a loop animation of an analysis object and can be switched using the switch buttons C8 and C9. Fig. 21B As shown in FIG. 1 , it is also possible to display a plurality of loop animations (animation groups) to be analyzed in the animation display field R4. Fig.17The loop animations other than the base loop animation selected in step S100 are displayed as analysis objects. In this case, a graph D1 showing the comparison result of the required time of each section can be displayed in the analysis information section R3 for all loop animations displayed in the animation display section R4, and the start time and end time of each section of all loop animations can be displayed in the section comparison section R5.

[0148] (Implementation method 3)

[0149] In the second embodiment, the required time of each action animation (each interval) in the loop animations to be compared (the base loop animation and the loop animation of the analysis object) is compared and prompted. In the present embodiment, the following information processing device is described: the information processing device prompts loop animations that require longer time for each interval than the base loop animation and loop animations that have lower similarity in each interval with the base loop animation from the loop animations selected as the analysis object. The information processing device 10 of the present embodiment has Figure 2 The configuration of the information processing device 10 of the first embodiment shown is the same as that of the first embodiment, and therefore the description of the configuration is omitted.

[0150] Fig. 22 is a flowchart showing an example of a comparison processing procedure of a loop animation according to the third embodiment. Fig.23 It is an explanatory diagram showing a screen example. Fig. 22 The treatment shown is in Fig.17 The process shown in the figure is obtained by adding steps S111 to S114 between steps S102 and S103. Fig.17 The same steps are omitted. Fig. 22 in, omit Fig.17 Illustration of steps S91 to S101 in .

[0151] In the information processing device 10 of this embodiment, the control unit 11 executes Fig.17The same processing as steps S91 to S102 in is performed. Then, the control unit 11 extracts one (S111) from the loop animation selected as the analysis object in step S100, and calculates the difference in required time between the extracted loop animation and each interval (each action animation) in the baseline loop animation (the difference in the excess part with the baseline loop animation as the benchmark) (S112). In addition, the control unit 11 calculates the similarity between the extracted loop animation and each interval (each action animation) in the baseline loop animation (S113). Here too, the similarity can use correlation coefficient, cosine similarity, etc., and can also be composed of a learning model that estimates the similarity between two action animations. The control unit 11 determines whether the processing of steps S112 to S113 has been completed for all the loop animations selected as the analysis object (S114). If it is determined that it has not been completed (S114: NO), it returns to step S111. If it is determined that it has been completed (S114: YES), it moves to step S103. In step S103 here, the control unit 11 generates the following Fig.23 The comparison screen is shown. Fig.23 The screen has Fig. 20A In the same structure as the screen of FIG. 1 , in the cumulative chart D2, for the loop animation with the largest difference in the required time of each section calculated in step S112 (difference from the reference loop animation), a message indicating that the required time of each section is the largest is displayed. Fig.23 In the example, it is shown that the required time for intervals 3 and 9 in the loop animation of video 2 is the longest. It should be noted that the message attached to the cumulative graph D2 is not limited to the configuration in which the difference from the baseline loop animation is the largest, but can also be displayed when the difference is the largest and is greater than a predetermined value. In addition, in this embodiment, instead of Fig.23 The comparison screen may also be a structure in which a message showing the smallest similarity of each interval calculated in step S113 (the similarity with the baseline loop animation) is added to the cumulative chart D2. In this case, loop animations in which the actions of the operator in the action animation are not similar (deviate) from the baseline loop animation can be prompted. For loop animations with intervals that require a long time and loop animations with intervals that have a small similarity with the baseline loop animation, there is a high possibility that there are some problems with the operator's work. By prompting the user with such loop animations and intervals, the user can verify whether the operator's work content is appropriate.

[0152] It should be noted that the control unit 11 may calculate a score corresponding to the difference in the time required for each interval (for example, the smaller the difference, the higher the score), and use the sum of the scores of each interval as the score of the loop animation to evaluate the loop animation. In addition, the control unit 11 may calculate a score corresponding to the similarity of each interval (each action animation) between the loop animation of the analysis object and the reference loop animation (for example, the greater the similarity, the higher the score), and use the sum of the scores of each interval as the score of the loop animation to evaluate the loop animation.

[0153] (Implementation 4)

[0154] In the second and third embodiments, the reference loop animation is determined from the loop animation selected as the analysis object. In the present embodiment, an information processing device is described as follows: the information processing device generates a reference loop animation based on the loop animation extracted from the animation of the processing object. The information processing device 10 of the present embodiment has Figure 2 The configuration of the information processing device 10 of the first embodiment shown is the same as that of the first embodiment, and therefore the description of the configuration is omitted.

[0155] Fig.24 is a flowchart showing an example of a comparison processing procedure of a loop animation according to a fourth embodiment. Fig.25 It is an explanatory diagram showing an example of a reference cycle animation. Fig.24 The treatment shown is in Fig.17 The process shown is obtained by adding steps S121 to S128 between steps S99 and S100. Fig.17 The same steps are omitted. Fig.24 in, omit Fig.17 Illustration of steps S91 to S98 and S103 to S106 in .

[0156] In the information processing device 10 of this embodiment, the control unit 11 executes Fig.17 The same processing as steps S91 to S99 in FIG. Fig. 18C or Fig.18D The screen shown here is provided with a button (not shown) for receiving an instruction to generate a base loop animation based on a loop animation extracted from the animation to be processed. By operating this button, an instruction to generate a base loop animation is received. The control unit 11 determines whether an instruction to generate a base loop animation is received (S121). If it is determined that no instruction to generate a base loop animation is received (S121: NO), the process moves to step S100. In this case, the control unit 11 executes the same Fig.17 The same processing as the embodiment 2 shown.

[0157] When it is determined that the instruction to generate the reference loop animation has been received (S121: YES), the control unit 11 extracts the action animations of the same action from each loop animation classified as the action animation in step S97 (S122). The control unit 11 calculates the score corresponding to the required time for the action animation extracted from each loop animation (S123). For example, the control unit 11 assigns a higher score to each action animation as the required time is shorter. Then, the control unit 11 calculates the similarity of all combinations in the action animations extracted from each loop animation (S124), and calculates the score corresponding to the calculated similarity (S125). For example, the control unit 11 calculates the similarity of one action animation with other action animations in sequence, assigns a higher score as the similarity is higher, and uses the sum of the scores corresponding to each similarity as the score of the action animation. The control unit 11 calculates the corresponding score of the similarity with other action animations by performing the same process on all action animations.

[0158] The control unit 11 determines the action animation used for the baseline loop animation based on the score calculated in step S123 and the score calculated in step S125 for each action (each interval) (S126). For example, the control unit 11 determines the action animation with the highest total of the score calculated in step S123 and the score calculated in step S125 as the action animation used for the baseline loop animation. The control unit 11 determines whether the processing of steps S123 to S126 has been completed for the action animations of all actions (S127). If it is determined that it has not been completed (S127: NO), it returns to step S122 and repeats the processing of steps S122 to S126 for the unprocessed actions. It should be noted that the action animation used for the baseline loop animation can be any action image extracted from the animation of the processing object, and can be an action animation of different operators.

[0159] When it is determined that the processing of the motion animations for all the actions has been completed (S127: YES), the control unit 11 splices the motion animations determined for each action in step S126 to generate a reference loop animation (S128). Fig.25 Animation of the benchmark cycle shown. Fig.25In the example, a reference loop animation is generated based on interval 1 of the loop animation of video 20, interval 2 of the loop animation of video 30, ..., interval 9 of video 40, and interval 10 of video 60. Then, the control unit 11 moves to step S100. It should be noted that in step S102 in this case, the control unit 11 determines the reference loop animation generated in step S128 as the reference loop animation that should be used as a model. Through the above processing, a reference loop animation can be generated using an action animation that has a short required time in any action (interval) and a high similarity with the action animation in other loop animations. It is believed that the higher the similarity between the action animation performed by the operator and the other action animations, the more standard the action is performed. Thus, the most suitable reference loop animation based on the action animation that has a short required time and performs a standard action can be generated. It should be noted that in the above processing, the action animation used for the reference loop animation is not limited to a structure determined based on the total of the score corresponding to the required time and the score corresponding to the similarity of the action animation in other loop animations. For example, the action animation that takes the shortest time or the action animation that has the greatest similarity to other action animations may be determined as the action animation used for the base loop animation. In addition, the action animation extracted from each loop animation may be subjected to a determination process of whether it is a standard action, such as the main factor classification model M5 used in Implementation Example 1, and the action animation determined to be a standard action may be processed in steps S123 to S126. In this case, a base loop animation based on an action animation obtained by shooting a more appropriate and more standard action can be generated.

[0160] (Implementation 5)

[0161] In the second to fourth embodiments, a video of one worker is processed. In the present embodiment, an information processing device that processes video of multiple workers is described. The information processing device 10 of the present embodiment has Figure 2 The configuration of the information processing device 10 of the first embodiment shown is the same as that of the first embodiment, and therefore the description of the configuration is omitted.

[0162] Fig.26 is a flowchart showing an example of a comparison processing procedure of a loop animation according to a fifth embodiment. Fig.27A and Fig.27B It is an explanatory diagram showing a screen example. Fig.26 The treatment shown is in Fig.17 The process shown is obtained by adding steps S131 to S136 between steps S91 and S92. Fig.17 The same steps are omitted. Fig.26 in, omit Fig.17Illustration of steps S94 to S106 in .

[0163] In the information processing device 10 of this embodiment, the control unit 11 Fig.18A The selection of any animation is accepted in the animation selection screen shown (S91). The control unit 11 performs object detection processing on the selected animation, and detects the subject (subject person) in the animation (S131). The object detection processing can be performed using a learning model composed of object detection algorithms such as CNN, SSD (Single Shot Multibox Detector), YOLO (You Only Look Once), or algorithms that implement semantic segmentation such as SegNet, FCN (Fully Convolutional Network), U-Net, etc. The control unit 11 determines whether the detected subject person is multiple (S132). If it is not determined to be multiple (S132: NO), that is, if the detected subject person is only one, it moves to step S92. In this case, the control unit 11 executes the same as Fig.17 The same processing as the embodiment 2 shown.

[0164] If it is determined that there are multiple detected objects (S132: YES), Fig.27A As shown in FIG. 1 , the control unit 11 displays a message prompting the user to select an analysis subject from the subjects in the animation (S133). Fig.27A The animation displayed in the screen is configured so that a subject can be selected through a predetermined operation (such as a left click of a mouse), and the control unit 11 accepts the selection of a subject through the predetermined operation on the animation (S134). It should be noted that the control unit 11 uses a boundary box or the like to clearly indicate the detected subject in the displayed animation, and can accept the selection of any subject by accepting the selection of any boundary box. The control unit 11 determines the area of ​​the selected subject in the animation (S135). Here, the control unit 11 can determine an area of ​​a predetermined size with the point at which the predetermined operation was performed as the area of ​​the analysis subject, or can determine an area of ​​the area of ​​each subject detected in step S131 that includes the point at which the predetermined operation was performed as the area of ​​the analysis subject. Fig.27B In the example of , a pull-down menu is displayed by performing a predetermined operation on the subject person on the right side of the animation, and when “Select as subject person” is selected in the pull-down menu, the control unit 11 determines the subject person as the analysis target.

[0165] When a plurality of subjects are included in the video, each subject is captured in the same area in the video. Fig.27AIn the example, the subject photographed on the left side of the animation is always photographed on the left side, and the subject photographed on the right side is always photographed on the right side. Therefore, in the present embodiment, the control unit 11 extracts the area including the analysis subject from each frame of the animation selected in step S91, and generates a partial animation (S136) including only the analysis subject. Then, the control unit 11 takes the partial animation including one analysis subject as the processing object, and performs the processing after step S92. Therefore, by the same processing as the information processing device 10 of embodiments 2 to 4, the comparison result of the loop animation extracted from the partial animation can be prompted. It should be noted that in the present embodiment, it is also possible to perform the processing after step S92 based on the area of ​​the analysis subject in the animation of the processing object instead of generating the partial animation including the analysis subject from the animation of the processing object.

[0166] In the above process, even when a plurality of workers are included in the animation, an arbitrarily selected worker can be used as the analysis object, and by the same process as in the above embodiments 2 to 4, a comparison of loop animations extracted from the animations obtained by photographing the worker selected as the analysis object can be performed. In the above embodiments 2 to 5, the animation to be processed is not limited to the animation obtained by photographing the worker performing the operation, but may be the animation obtained by photographing a robot configured to perform a predetermined operation.

[0167] The matters described in the above-mentioned embodiments can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in all combinations, regardless of the reference format. Moreover, although the form of recording a claim that references two or more other claims (multiple claim form) is used in the claims, it is not limited to this. It is also possible to record in the form of recording multiple claims that reference at least one multiple claim (multiple-reference-multiple claim).

[0168] It should be understood that the embodiments disclosed this time are illustrative in all aspects and not restrictive. The scope of the present invention is shown by the scope of the claims, not by the above meaning, and is intended to include all changes within the meaning and scope equivalent to the scope of the claims.

[0169] Description of reference numerals:

[0170] 10Information processing device

[0171] 11. Control Unit

[0172] 12 Storage unit

[0173] 13Ministry of Communications

[0174] 14 Input section

[0175] 15 Display unit

[0176] M1 Variety Forecast Model

[0177] M2 Cycle Prediction Model

[0178] M3 Boundary Prediction Model

[0179] M4 motion analysis model

[0180] M5 main factor classification model

Claims

1. A program, characterized in that Causes the computer to perform the following processing: The animation obtained by filming the object performing the operation is divided into different actions according to each action included in the operation; Based on the respectively distinguished animations, determining whether each action is a standard action; as well as Based on the determination results of each action, it is analyzed whether the operation is a standard operation.

2. The program according to claim 1, characterized in that The computer is caused to execute the following processing: inputting the animation into a learning model, and outputting information related to the action performed by the object in the distinguishing animation contained in the animation, wherein the learning model is learned to output information related to the action performed by the object when an animation obtained by photographing the object is input; as well as Based on the output information related to the action, it is determined whether the action in the distinguishing animation is a standard action.

3. The program according to claim 1 or 2, characterized in that The computer is caused to execute the following processing: Whether each action is a standard action is determined based on whether the action time of each action in each segmented animation is the standard action time.

4. The program according to claim 1, characterized in that The computer is caused to execute the following processing: Separating the animation into each operation element including a plurality of actions; and The separately distinguished work element animations are distinguished according to each of the above-mentioned actions.

5. The program according to claim 2, characterized in that The computer is caused to execute the following processing: Based on the animation, determining the type of the work object of the work performed by the object in the animation; Selecting the learning model corresponding to the determined variety; as well as The animation is input into the selected learning model, and information related to the action performed by the object in the distinguishing animation contained in the animation is output.

6. The program according to claim 1, characterized in that The computer is caused to execute the following processing: Inputting the animation into a learning model and outputting information related to the action performed by the object in the animation, wherein the learning model is learned to output information related to the action included in the operation when an animation obtained by filming an object performing an operation is input; as well as Based on the output information related to the action, the animation is differentiated for each of the actions.

7. The program according to claim 1, characterized in that The computer is caused to execute the following processing: outputting a distinguishing position for distinguishing the animation according to each of the actions; and Accept changes to each partition location.

8. The program according to claim 1, characterized in that The computer is caused to execute the following processing: For any playback position of the animation, setting of a division position divided for each of the actions is accepted.

9. The program according to claim 4, characterized in that The computer is caused to execute the following processing: Inputting a plurality of the work element animations into a learning model respectively, outputting information related to a main factor that the work element performed by the object in the work element animation is not a standard work, wherein the learning model learns to output information related to the main factor when the work element animation is input; as well as Based on the output information related to the main factor, the plurality of work element animations are grouped for each of the main factors.

10. The program according to claim 9, characterized in that The computer is caused to execute the following processing: An animation including the work element animation is output for each of the main factors.

11. The program according to claim 1, characterized in that The computer is caused to execute the following processing: Based on the divided animations obtained by dividing each of the plurality of animations into different animations for each of the actions, the divided animations of the actions in the plurality of animations are synchronously output.

12. The program according to claim 1, characterized in that The computer is caused to execute the following processing: measuring the action time of each action in each animation based on each divided animation obtained by dividing each of the plurality of animations into each of the actions; as well as A graph showing the measured deviations in the motion time of each motion in each animation is generated.

13. The program according to claim 12, characterized in that The graph is a graph that associates the action time of each action in each animation with each action. The program causes the computer to execute the following processing: When a selection of a point plotted in the graph is accepted, a distinguishing animation of a corresponding action in the animation corresponding to the selected point is output.

14. The program according to claim 4, characterized in that The computer is caused to execute the following processing: Based on the work element animation obtained by segmenting the animation, a graph showing work start timing and work end timing for each of the work elements is generated.

15. The program according to claim 1 or 2, characterized in that The computer is caused to execute the following processing: Differentiate the plurality of animations according to each action included in the job; and Based on the respectively differentiated differentiated animations, differentiated animations of the respective actions in the plurality of the animations are synchronously output.

16. The program according to claim 15, characterized in that The computer is caused to execute the following processing: Selecting a benchmark animation from among the plurality of said animations for comparison with the other animations; and The selected reference animation and the distinguishing animations of the respective actions in the other animations except the reference animation are synchronously output.

17. The program according to claim 16, characterized in that The computer is caused to execute the following processing: When a change instruction of the other animation is received during output of the reference animation and any one of the other animations, the reference animation and the other animation instructed to be changed are synchronously output from the action-specific animation being output.

18. The program according to claim 16, characterized in that The computer is caused to execute the following processing: From the differentiated animations of each of the plurality of other animations, differentiated animations having a low degree of similarity between each differentiated animation in the reference animation and each differentiated animation in the other animations are extracted.

19. The program according to claim 16, characterized in that The computer is caused to execute the following processing: From the differentiated animations of each of the plurality of other animations, a differentiated animation in which a motion related to the differentiated animation requires a long time is extracted.

20. The program according to claim 16, characterized in that The computer is caused to execute the following processing: extracting, from each of the plurality of the animations, a differentiation animation to be included in the base animation for each action; and The distinguishing animations extracted for each action are spliced ​​together to generate the reference animation.

21. The program according to claim 1 or 2, characterized in that The animation uses the time for the object to perform a series of operations as a loop animation, and includes multiple loop animations. The program causes the computer to execute the following processing: Accepts a specification of a looping animation in the animation; Extracting a looping animation similar to a designated looping animation from the animation; as well as The extracted loop animations are divided into different groups according to the actions.

22. The program according to claim 21, characterized in that The plurality of animations obtained by photographing one or more of the objects are stored in a storage unit, The program causes the computer to execute the following processing: extracting a loop animation similar to the designated loop animation from the plurality of animations stored in the storage unit; and The extracted loop animations are divided into different groups according to the actions.

23. The program according to claim 1 or 2, characterized in that The computer is caused to execute the following processing: In the case where the animation includes a plurality of objects, accepting a selection of any one of the objects; and The animation is differentiated according to each action performed by the selected object.

24. An information processing method, characterized in that: The following processing is performed by the computer: The animation obtained by filming the object performing the operation is divided into different actions according to each action included in the operation; Based on the respectively distinguished animations, determining whether each action is a standard action; as well as Based on the determination results of each action, it is analyzed whether the operation is a standard operation.

25. An information processing device, characterized in that: have: a distinguishing unit for distinguishing the animation obtained by photographing the object performing the operation according to each action included in the operation; a determination unit that determines whether each action is a standard action based on the respectively distinguished distinguished animations; as well as The analyzing unit analyzes whether the work is a standard work based on the determination result of each action.

Images