Program, information processing method, and information processing device

Abstract

Provided is a program or the like capable of accurately determining whether or not a subject performing a work is performing a prescribed work. A computer distinguishes each motion included in a work based on an animation obtained by photographing a subject performing the work. In addition, the computer determines whether or not each motion is a standard motion based on the distinguished animations respectively. Then, the computer analyzes whether or not the work is a standard work based on the determination results for each motion.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a program, an information processing method, and an information processing apparatus. BACKGROUND

[0002] In Patent Literature 1, a technology is disclosed in which, for a process constituted by a plurality of jobs whose order is determined in advance, it is detected whether or not the behavior of an object is in accordance with the order of the jobs. In the technology disclosed in Patent Literature 1, for each job constituting the process, it is determined whether or not the object is performing the prescribed job in accordance with the process information in which the job name, the job order, the standard job time, and the like are prescribed.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2021-82137 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] In the technology disclosed in Patent Literature 1, it is determined whether or not the prescribed job is being performed in accordance with the order of the jobs, based on whether or not each job is being performed within the standard job time. However, even a job being performed within the standard job time is not necessarily a job being performed in accordance with the prescribed job content (action content). Therefore, it is difficult to accurately determine whether or not the prescribed job content is being performed, only by the determination of whether or not the job time is the standard job time.

[0008] An object of the present disclosure is to provide a program and the like capable of accurately determining whether or not an object performing a job is performing a prescribed job.

[0009] SOLUTION TO PROBLEM

[0010] The program of one embodiment of the present disclosure causes a computer to perform processing of distinguishing an animation obtained by photographing an object performing a job, for each action included in the job, based on the distinguished animation, determining whether or not each action is a standard action, and analyzing whether or not the job is a standard job, based on the determination result for each action.

[0011] EFFECT OF THE INVENTION

[0012] In one embodiment of the present disclosure, it is possible to accurately determine whether or not an object performing a job is performing a prescribed job. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1A is an explanatory diagram illustrating an example of an animation of a processing object.

[0014] Figure 1Bis an explanatory diagram showing an example of an animation of a processing target.

[0015] Figure 2 is a block diagram showing an example of a configuration of an information processing apparatus.

[0016] Figure 3 is an explanatory diagram showing an example of a configuration of a learning model.

[0017] Figure 4 is an explanatory diagram showing an example of a configuration of a learning model.

[0018] Figure 5 is a flowchart showing an example of a generation processing sequence of a motion analysis model.

[0019] Figure 6A is an explanatory diagram showing an example of a screen.

[0020] Figure 6B is an explanatory diagram showing an example of a screen.

[0021] Figure 6C is an explanatory diagram showing an example of a screen.

[0022] Figure 7 is a flowchart showing an example of an analysis processing sequence of a work.

[0023] Figure 8 is an explanatory diagram showing an example of a screen.

[0024] Figure 9 is a flowchart showing an example of an analysis processing sequence of a main factor of each work element.

[0025] Figure 10A is an explanatory diagram showing an example of a screen.

[0026] Figure 10B is an explanatory diagram showing an example of a screen.

[0027] Figure 11 is a flowchart showing an example of an analysis processing sequence of a motion time of each motion.

[0028] Figure 12 is an explanatory diagram showing an example of a screen.

[0029] Figure 13 is a flowchart showing an example of a production processing sequence of a Gantt chart.

[0030] Figure 14 is an explanatory diagram showing an example of a screen of a Gantt chart.

[0031] Figure 15 is an explanatory diagram showing an example of display of two analysis result screens.

[0032] Figure 16Fig. 1 is an explanatory diagram showing a configuration example of a cycle animation extraction model.

[0033] Figure 17 Fig. 2 is a flowchart showing an example of a comparison processing sequence of a cycle animation.

[0034] Figure 18A Fig. 3 is an explanatory diagram showing a screen example.

[0035] Figure 18B Fig. 4 is an explanatory diagram showing a screen example.

[0036] Figure 18C Fig. 5 is an explanatory diagram showing a screen example.

[0037] Figure 18D Fig. 6 is an explanatory diagram showing a screen example.

[0038] Figure 19A Fig. 7 is an explanatory diagram showing a screen example.

[0039] Figure 19B Fig. 8 is an explanatory diagram showing a screen example.

[0040] Figure 19C Fig. 9 is an explanatory diagram showing a screen example.

[0041] Figure 20A Fig. 10 is an explanatory diagram showing a screen example.

[0042] Figure 20B Fig. 11 is an explanatory diagram showing a screen example.

[0043] Figure 20C Fig. 12 is an explanatory diagram showing a screen example.

[0044] Figure 20D Fig. 13 is an explanatory diagram showing a screen example.

[0045] Figure 21A Fig. 14 is an explanatory diagram showing a screen example.

[0046] Figure 21B Fig. 15 is an explanatory diagram showing a screen example.

[0047] Figure 22 Fig. 16 is a flowchart showing an example of a comparison processing sequence of a cycle animation of Embodiment 3.

[0048] Figure 23 Fig. 17 is an explanatory diagram showing a screen example.

[0049] Figure 24 Fig. 18 is a flowchart showing an example of a comparison processing sequence of a cycle animation of Embodiment 4.

[0050] Figure 25 Fig. 19 is an explanatory diagram showing an example of a reference cycle animation.

[0051] Figure 26 is a flowchart showing an example of a comparative processing sequence of the cycle animation of Embodiment 5.

[0052] Figure 27A is an explanatory diagram showing an example of a screen.

[0053] Figure 27B is an explanatory diagram showing an example of a screen. DETAILED DESCRIPTION

[0054] Hereinafter, the program, the information processing method, and the information processing apparatus of the present disclosure will be described in detail based on the drawings showing embodiments thereof.

[0055] (Embodiment 1)

[0056] An information processing apparatus analyzes whether or not a work performed by a worker (subject) is a standard work set in advance, based on an animation obtained by photographing the worker performing the work. Figure 1A and Figure 1B is an explanatory diagram showing an example of an animation of a processing target. The animation of the processing target of the present embodiment is, for example, animation data including a plurality of images (still images) such as 30 or 15 images for 1 second, for example, as shown in Figure 1A is a photographed image obtained by photographing a worker performing a work from above the worker. In addition, as shown in Figure 1B is an explanatory diagram showing an example of an animation of a processing target. The animation of the processing target of the present embodiment is, for example, animation data including a plurality of images (still images) such as 30 or 15 images for 1 second, for example, as shown in Figure 1B is an explanatory diagram showing an example of an animation of a processing target. The animation of the processing target of the present embodiment is, for example, animation data including a plurality of images (still images) such as 30 or 15 images for 1 second, for example, as shown in

[0057] Figure 2is a block diagram showing a configuration example of an information processing apparatus. The information processing apparatus 10 is an apparatus capable of realizing various information processing and transmission and reception of information, and is constituted by, for example, a server computer, a personal computer, a workstation, or the like. The information processing apparatus 10 includes a control section 11, a storage section 12, a communication section 13, an input section 14, a display section 15, a reading section 16, and the like, which are connected to each other via a bus. The control section 11 includes one or a plurality of processors such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), or an AI chip (semiconductor for AI). The control section 11 performs information processing and control processing that the information processing apparatus 10 should perform, by appropriately executing a program 12P stored in the storage section 12.

[0058] The storage section 12 includes a RAM (Random Access Memory), a flash memory, a hard disk, an SSD (Solid State Drive), or the like. The storage section 12 stores the program 12P (program product) and various data that the control section 11 executes. In addition, the storage section 12 temporarily stores data and the like generated when the control section 11 executes the program 12P. The program 12P and various data can be written in the storage section 12 at the manufacturing stage of the information processing apparatus 10, or can be downloaded in the storage section 12 from other apparatus by the control section 11 via the communication section 13. In addition, the storage section 12 stores learned models such as the variety prediction model M1, the cycle prediction model M2, the boundary prediction model M3, the action analysis model M4, and the main factor classification model M5, which have been learned by machine learning using training data. It is assumed that the learned models M1 to M5 are used as program modules that constitute an artificial intelligence software. The learned models M1 to M5 are used to perform a predetermined operation on an input value and output an operation result, and in the storage section 12, information of layers possessed by the learned models M1 to M5, information of nodes that constitute each layer, weights (coupling coefficients) between nodes, and the like are stored as information that defines the learned models M1 to M5. Note that the cycle prediction model M2, the boundary prediction model M3, and the action analysis model M4 are prepared for each variety of products on which an assembly work is performed by a worker. Furthermore, the storage section 12 stores a plurality of animation files 12a that are processing targets.

[0059] The communication section 13 is a communication module for performing processing related to wired or wireless communication, and transmits and receives information to and from other devices via a network N. The network N can be the Internet or a public communication line, or can be a LAN (Local Area Network) constructed within a facility in which the information processing device 10 is installed. The input section 14 accepts an operation input by a user, and sends a control signal corresponding to the operation content to the control section 11. The display section 15 is a liquid crystal display or an organic EL display, and displays various information according to an instruction from the control section 11. A part of the input section 14 and the display section 15 can be a touch panel configured as one body.

[0060] The reading section 16 reads information stored in a mobile storage medium 10a including a CD (Compact Disc), a DVD (Digital Versatile Disc), a USB (Universal Serial Bus) memory, an SD (Secure Digital) card, and the like. The program 12P and various data stored in the storage section 12 can be read from the mobile storage medium 10a by the control section 11 via the reading section 16, and stored in the storage section 12.

[0061] The information processing device 10 can be a multi-computer configured by a plurality of computers, or can be a virtual machine virtually constructed by software in one device. In a case where the information processing device 10 is configured by a server computer, the information processing device 10 can be a local server, or can be a cloud server communicatively connected via a network such as the Internet. In addition, the program 12P can be executed on a single computer, or can be executed on a plurality of computers connected to each other via the network N. Furthermore, the input section 14 and the display section 15 are not necessarily required, and the information processing device 10 can be configured to accept an operation by a connected computer, or can be configured to output information to be displayed to an external display device.

[0062] Figure 3 and Figure 4 is an explanatory diagram showing a configuration example of the learning models M1 to M5. The learning models M1 to M5 are configured using, for example, an RNN (Recurrent Neural Network) or an LSTM (Long Short-Term Memory). Note that the learning models M1 to M5 can be configured using an algorithm such as a CNN (Convolution Neural Network), a Transformer, or the like, in addition to the RNN or the LSTM, or can be configured by combining a plurality of algorithms.

[0063] The variety prediction model M1 is a learning completion model that has been learned in such a manner that, as input, an animation obtained by photographing the entire body of a worker in a work, based on the input animation, performs an operation of discriminating the variety of a product on which the photographed worker performs assembly work, and outputs the result of the operation. Note that the variety prediction model M1 is not limited to a configuration in which an animation obtained by photographing the entire body of a worker is input, and may, for example, be a configuration in which an animation obtained by photographing the appearance of a worker performing assembly work is input, like an animation obtained by photographing the upper body or hands of a worker. The variety prediction model M1 has an input layer that inputs an animation, an intermediate layer that extracts a feature amount from the input animation, and an output layer that outputs information indicating the variety of a product on which a worker in an animation performs assembly work, based on the operation result of the intermediate layer. The input layer has input nodes that sequentially input images (frames) included in a time series in an animation. The intermediate layer calculates an output value based on an animation input via the input layer using various functions and threshold values and the like. The output layer has a plurality of output nodes respectively associated with varieties set in advance, and outputs, from each output node, a probability (confidence) that each variety should be discriminated as the variety of a product on which a worker performs assembly work. The output value from each output node of the output layer is, for example, a value of 0 to 1, and the total of the probabilities output from each output node becomes 1.0 (100%). The information processing apparatus 10, for example, in a case where an animation is input in the variety prediction model M1, determines an output node that outputs the largest output value among the output values (confidences) from each output node, and determines the variety associated with the determined output node as the variety of a product on which a worker in the animation performs assembly work. Note that, instead of a plurality of output nodes that output confidences for each variety, the output layer of the variety prediction model M1 can be a configuration having one output node that outputs information indicating the variety with the largest confidence.

[0064] The product variety prediction model M1 can be generated by machine learning using training data including the training animation and information (correct label) indicating the variety of the product on which the worker in the animation performs the assembly work. The product variety prediction model M1 is learned so that, when the animation included in the training data is input, the output value from the output node corresponding to the variety indicated by the correct label included in the training data approaches 1, and the output value from the other output nodes approaches 0. In the learning process, the product variety prediction model M1 performs the operation in the intermediate layer and the output layer based on the input animation, and calculates the output value from each output node, and compares the calculated output value of each output node with the value corresponding to the correct label (1 for the output node corresponding to the correct label, and 0 for the other output nodes) to optimize the parameters for the operation process in the intermediate layer and the output layer in such a manner that they approach each other. The parameters are the weights (coupling coefficients) between the nodes in the intermediate layer and the output layer, the coefficients of the functions, the threshold values, and the like. The optimization method of the parameters is not particularly limited, and the backpropagation method, the gradient descent method, or the like can be used. Thus, the product variety prediction model M1 that predicts the variety of the product on which the worker in the animation performs the assembly work when the animation is input and outputs the prediction result is obtained.

[0065] The cycle prediction model M2 is a learning completed model that has been learned in such a manner that, when an animation obtained by photographing the entire body of the worker in the work is input, an operation of dividing the animation into cycle animations based on the input animation is performed, and the result of the operation is output. The cycle prediction model M2 is not limited to the configuration in which the animation obtained by photographing the entire body of the worker is input, and may, for example, be configured so that an animation obtained by photographing the appearance of the worker performing the assembly work is input, like the animation obtained by photographing the upper body or the hands of the worker. The cycle prediction model M2 has an input layer that inputs the animation, an intermediate layer that extracts a feature quantity from the input animation, and an output layer that outputs the start time and the end time of each cycle animation obtained by dividing the animation based on the operation result of the intermediate layer. The intermediate layer of the cycle prediction model M2 calculates the output value based on the animation input via the input layer, and the output layer has output nodes that output the start time and the end time of each cycle animation obtained by dividing the animation. Note that the start time and the end time of the cycle animation indicate the elapsed time from the start of the playback of the animation. When the animation is input in the cycle prediction model M2, the information processing apparatus 10 divides the animation by each cycle animation based on the output value (the start time and the end time of the cycle animation) from the output node.

[0066] The cycle prediction model M2 is generated using a setting animation for setting boundaries of each cycle animation (each work element) in an animation. The setting animation uses an animation obtained by shooting a work appearance of a worker who can perform a standard motion for each motion. Specifically, in a case where the setting animation is input to the cycle prediction model M2, the cycle prediction model M2 performs an operation in the intermediate layer and the output layer based on the input animation to calculate an output value from an output node, and outputs a time (a start time and an end time of a cycle animation) that divides the setting animation into boundaries of each cycle animation. A user can correct the time of the boundary of each cycle output by the cycle prediction model M2, and the cycle prediction model M2 optimizes parameters for the operation process in the intermediate layer and the output layer based on the corrected time of each boundary. Thus, the cycle prediction model M2 that predicts a start time and an end time of each cycle animation in an input animation and outputs a prediction result is obtained.

[0067] The boundary prediction model M3 is a learning completed model that is learned in such a manner that an animation that has been divided into cycle animations is input, an operation that divides each cycle animation into a motion animation based on the input animation and an operation that predicts a time period in which a boundary can be set other than a boundary of the divided motion animation are performed, and a result of the operation is output. The boundary prediction model M3 has an input layer that inputs an animation, an intermediate layer that extracts a feature quantity from the input animation, and an output layer that outputs a start time and an end time of a motion animation obtained by dividing each cycle animation and a time period in which a boundary of the motion animation can be set based on an operation result of the intermediate layer. The intermediate layer of the boundary prediction model M3 calculates an output value based on the animation input via the input layer, and the output layer has output nodes that respectively output a start time and an end time of each motion animation obtained by dividing each cycle animation in the animation and a settable time period of a boundary of the motion animation. The information processing apparatus 10 divides cycle animations in the animation by each motion animation based on the output value (a start time and an end time of a motion animation) from the output node in a case where the animation is input to the boundary prediction model M3.

[0068] The boundary prediction model M3 is generated using the setting animation for setting the boundaries of each action animation (each action). The animation used for the generation of the boundary prediction model M3 is the animation used for the generation of the cycle prediction model M2, and is an animation that is divided into each cycle animation by the cycle prediction model M2. In a case where the setting animation is input to the boundary prediction model M3, the boundary prediction model M3 performs the operation in the intermediate layer and the output layer based on the input animation to calculate the output value from the output node, and outputs the time of the boundary that divides the setting animation into each action animation (the start time and the end time of the action animation), and the settable time period of the boundary of the action animation. The user can correct the time of the boundary of each action output by the boundary prediction model M3, and in addition, can correct the settable time period of the boundary output by the boundary prediction model M3, and the boundary prediction model M3 optimizes the parameters used for the operation processing in the intermediate layer and the output layer based on the corrected time of each boundary and the settable time period. Thus, the boundary prediction model M3 that predicts the start time and the end time of each action animation in the input animation, and the settable time period of the boundary of each action animation, and outputs the prediction result is obtained.

[0069] The action analysis model M4 is a learning completed model that is learned in such a manner that an animation that has been divided into action animations, that is, an animation for which the boundaries between the action animations have been set, is input, an operation that divides each action animation included in a cycle animation in the input animation based on the input animation, and an operation that analyzes whether the action of the worker in the divided action animation is a standard action are performed, and a result of the operation is output. The action analysis model M4 has an input layer that inputs an animation, an intermediate layer that extracts a feature amount from the input animation, and an output layer that outputs the start time and the end time of the action animation included in each cycle animation in the animation, and an analysis result (information related to the action performed by the worker) that indicates whether the action of the worker in each action animation is a standard action, based on the operation result of the intermediate layer. The intermediate layer of the action analysis model M4 calculates an output value based on the animation input via the input layer, and the output layer has output nodes that respectively output the start time and the end time of each action animation in the animation, and the analysis result of each action animation. In a case where the animation is input to the action analysis model M4, the information processing apparatus 10 divides the cycle animation in the animation into the action animation based on the output value (the start time and the end time of the action animation, the analysis result) from the output node, and determines whether the action of the worker in each action animation is a standard action.

[0070] The action analysis model M4 is generated using animation obtained by photographing a worker performing a standard action for each action as setting animation. The animation used for generation of the action analysis model M4 is the animation used for generation of the boundary prediction model M3, and is animation classified into each action animation by the boundary prediction model M3. In a case where the setting animation is input to the action analysis model M4, the action analysis model M4 performs calculation in the intermediate layer and the output layer based on the input animation to calculate an output value from an output node, and outputs a time of a boundary classifying the setting animation into each action animation (a start time and an end time of the action animation), and an analysis result indicating whether or not the action of each action animation is a standard action. The user can correct the time of the boundary of each action output by the action analysis model M4, and the action analysis model M4 optimizes parameters used for calculation processing in the intermediate layer and the output layer based on the corrected time of each boundary. Note that the analysis result here is information indicating that each action is a standard action, and thus, in a case where the output analysis result is information indicating that it is not a standard action, the action analysis model M4 corrects the analysis result, and optimizes parameters based on the corrected analysis result. Thus, the action analysis model M4 that predicts a start time and an end time of each action animation in input animation, and an analysis result of the action of each action animation, and outputs a prediction result is obtained. By using such an action analysis model M4, it is determined whether or not the action of the worker photographed in the analysis target animation is appropriate as a result of comparison with a standard action. Note that the action analysis model M4 can also be configured to output information indicating a main factor considered to be a non-standard action in addition to the analysis result indicating whether or not the action of the worker in each action animation is a standard action in a case where the action of the worker is not a standard action. The main factor considered to be a non-standard action can be, for example, an action time longer or shorter than a standard action time by a predetermined time or more.

[0071] The main factor classification model M5 is a learning completion model that has learned in such a manner that, as input, a cycle animation that is distinguished from the animation, performs an operation of judging that a work element performed by a worker in the cycle animation is not a main factor of the standard work based on the input cycle animation, and outputs a result of the operation. The main factor classification model M5 has an input layer that inputs the cycle animation, an intermediate layer that extracts a feature amount from the input cycle animation, and an output layer that outputs information (information related to a main factor) indicating that a work element of a worker in the cycle animation is not a main factor of the standard work based on an operation result of the intermediate layer. The intermediate layer of the main factor classification model M5 calculates an output value based on the cycle animation input via the input layer, and the output layer has a plurality of output nodes respectively associated with the main factors set in advance, and outputs, from each output node, a probability (a degree of certainty) that each main factor should be judged to be a main factor of a work element performed by a worker that is not the standard work. The output value from each output node of the output layer is, for example, a value of 0 to 1, and a total of the probabilities output from each output node becomes 1.0 (100%). As the main factor that a work element of a worker is not the standard work, for example, there can be mentioned a lack of an action included in the work element, a change in an execution order of an action included in the work element, a difference in a movement of an action included in the work element from a standard action, a longer or shorter movement time of an action included in the work element than a standard movement time by a predetermined time or more, and the like. The information processing apparatus 10 determines an output node that outputs the largest output value among the output values (degrees of certainty) from each output node, for example, in a case where the cycle animation is input to the main factor classification model M5, and determines a main factor associated with the determined output node as a main factor that a work element of a worker in the cycle animation is not the standard work. Note that, instead of having a plurality of output nodes that output a degree of certainty for each main factor, the output layer of the main factor classification model M5 can also be configured to have one output node that outputs information indicating a main factor having the largest degree of certainty.

[0072] The main factor classification model M5 can be generated by machine learning using training data including a cycle animation for training and information (correct label) indicating that a work element of a worker in the cycle animation is not a main factor of standard work. The main factor classification model M5 learns in such a manner that, when the cycle animation included in the training data is input, the output value from the output node corresponding to the main factor indicated by the correct label included in the training data is made close to 1, and the output value from the other output nodes is made close to 0. In the learning process, the main factor classification model M5 performs operations in the intermediate layer and the output layer based on the input animation, calculates the output value from each output node, and compares the calculated output value of each output node with the value corresponding to the correct label (1 for the output node corresponding to the correct label and 0 for the other output nodes) to optimize the parameters for the operation process in the intermediate layer and the output layer in such a manner that they are made close to each other. Here, as well, the parameters to be optimized are the weights (coupling coefficients) between the nodes in the intermediate layer and the output layer, the coefficients of the functions, the threshold values, and the like, and the method of optimizing the parameters can use the backpropagation method, the gradient descent method, or the like. Thus, the main factor classification model M5 that predicts a main factor in which a work element of a worker in a cycle animation is not a standard work and outputs the prediction result when the cycle animation is input is obtained.

[0073] The learning of the learning models M1 to M5 can also be performed by other learning devices. The learning-completed learning models M1 to M5 generated by learning performed by other learning devices are downloaded to the information processing device 10 from the learning device via the network N or via the mobile storage medium 10a and stored in the storage section 12, for example. In the present embodiment, the cycle prediction model M2, the boundary prediction model M3, and the motion analysis model M4 are generated by one-shot learning in which a set of training data (setup animation) or a small amount of training data is used for learning, but can also be generated by learning using a plurality of training data. In addition, the learning models M1 to M5 are not limited to the configuration in which an animation obtained by photographing a worker or a cycle animation extracted from the animation is input. For example, the information processing device 10 can be configured to perform skeleton estimation of a worker based on each frame (each image) included in the animation to generate skeleton data, and input the time-series skeleton data to the learning models M1 to M5. In this case, the control section 11 of the information processing device 10 performs skeleton estimation of a worker in each frame and extracts the joint positions of the worker, for example, by using a technique such as Open Pose that extracts the joint positions of a person in an image. The control section 11 can also acquire time-series skeleton data by performing a process of extracting joint positions on each frame included in the animation, and input the obtained time-series skeleton data as input to the learning models M1 to M5.

[0074] The following describes the generation processing of the action analysis model M4 in the information processing apparatus 10 of the present embodiment. Figure 5 is a flowchart showing an example of the generation processing sequence of the action analysis model M4, Figure 6A-6C is an explanatory diagram showing a screen example. The following processing is performed by the control section 11 of the information processing apparatus 10 in accordance with the program 12P stored in the storage section 12.

[0075] In the case of generating the action analysis model M4 for each variety, the user selects the setting animation of the action analysis model M4. The setting animation of the action analysis model M4 is an animation in which each action included in each cycle (each work element) has been distinguished using the cycle prediction model M2 and the boundary prediction model M3. The control section 11 of the information processing apparatus 10 accepts the selection of the setting animation of the action analysis model M4 through operation via the input section 14 (S11). The control section 11 inputs the selected animation to the action analysis model M4, distinguishes each action animation from the processing target animation based on the output information from the action analysis model M4, and extracts boundary candidates of each action (S12). Specifically, the control section 11 sequentially inputs each frame included in the animation file 12a to the action analysis model M4, and determines the time of the boundary candidate of each action based on the start time and the end time of each action animation output from the action analysis model M4.

[0076] The control section 11, in the case where the boundary candidates of each action animation in the processing target animation have been extracted, displays the boundary editing screen as shown in Figure 6A in the display section 15 (S13). The boundary editing screen is a screen for changing the time of the boundary candidate of each action based on the start time and the end time of each action animation output from the action analysis model M4. Figure 6A The screen as shown in

[0077] The control unit 11 determines whether a deletion instruction for any boundary candidate has been received due to the operation of the delete button B1 in the editing area A2 (S14). If a deletion instruction has been received (S14: YES), the boundary candidate to be deleted is deleted (S15). If the boundary candidate has been deleted, the control unit 11 removes the boundary candidate to be deleted from the boundary candidates displayed in the editing area A2 and sequentially displays the boundary candidates after the deleted boundary candidate. If no deletion instruction has been received (S14: NO), the control unit 11 skips step S15 and determines whether a new boundary addition instruction has been received due to the operation of the add button B2 in the editing area A2 (S16).

[0078] If the control unit 11 determines that it has received an additional instruction for a new boundary (S16: YES), it will... Figure 6B The boundary-added screen shown is displayed on the display unit 15 (S17). Figure 6A In the shown screen, the additional button B2 between the second and third boundary candidates is activated. In this case, as shown... Figure 6B As shown, a boundary addition screen for adding new boundaries is displayed at any playback time within the time interval between the second and third boundary candidates. It should be noted that when generating the setting animation for the motion analysis model M4, if the settable time interval of the boundary output from the boundary prediction model M3 is obtained, the control unit 11 can display the settable time interval in the boundary addition screen. In this case, the user can consider the settable time interval to specify the time for the new boundary.

[0079] The user inputs the desired boundary time into the boundary addition screen, and the control unit 11 receives the new boundary time via the boundary addition screen (S18). Figure 6B If the add button is activated in the shown screen, the control unit 11 adds a boundary at the received time (S19). When a new boundary is added, as... Figure 6C As shown, the control unit 11 adds a new boundary between the second and third boundary candidates, and postpones the display of the boundary candidates after the added boundary. If it is determined that no instruction to add a new boundary has been received (S16: NO), the control unit 11 skips steps S17 to S19.

[0080] Control unit 11 judges that Figure 6A In the screen shown in C, check if the save button has been activated (S20). If the save button has not been activated (S20: NO), return to step S14 and repeat steps S14 to S19. It should be noted that in... Figure 6AIn a case where the "next" button in the screen illustrated in FIG. 10C is operated, the control section 11 changes the display content of the edit area A2 to the boundary candidate after the boundary candidate in the lowermost display. In addition, in a case where the cancel button in the screen illustrated in FIG. 10C is operated, the control section 11 interrupts and ends the edit processing of the boundary candidate. Figure 6A In a case where the cancel button in the screen illustrated in FIG. 10C is operated, the control section 11 interrupts and ends the edit processing of the boundary candidate.

[0081] In a case where it is judged that the save button is operated (S20: YES), the control section 11 determines the boundary candidate after the edit (deletion and addition) via the edit area A2 as a boundary, and stores the time of each boundary into the storage section 12 (S21). Specifically, the control section 11 associates the boundary number indicating the order of appearance of each boundary with the time of each boundary, and stores into, for example, a DB provided in the storage section 12. Note that, in a case where each boundary is given a name, the name of each boundary can also be stored. Then, the control section 11 sets (optimizes) the parameters of the operation processing in the intermediate layer and the output layer of the motion analysis model M4 based on the time of each boundary (S22). Here, the control section 11 sets the parameters of the motion analysis model M4 in such a manner that the set-up animation input to the motion analysis model M4 can be distinguished for each time of each boundary stored in step S21. In addition, the control section 11 sets the parameters of the motion analysis model M4 in such a manner that, based on the analysis result output by the motion analysis model M4 in a case where the set-up animation is input, each motion animation outputs the analysis result indicating a standard motion. Thereby, it is possible to generate the motion analysis model M4 that distinguishes each motion animation in a case where the animation is input, and outputs the analysis result of whether the motion of each distinguished motion animation is a standard motion.

[0082] In the above processing, it is configured to edit the boundary between each motion animation by performing the deletion of the boundary between the motion animations extracted using the motion analysis model M4, and the addition of a new boundary. With such a configuration, it is possible to set the parameters of the motion analysis model M4 with high precision based on the edited boundary between each motion animation. In addition, the method of editing each boundary is not limited to the above processing, and for example, it can be configured such that the edit area A2 can directly change the time of each boundary candidate.

[0083] In a case where the save button is operated (S20: YES), the control section 11 determines the boundary candidate after the edit (deletion and addition) via the edit area A2 as a boundary, and stores the time of each boundary into the storage section 12 (S21). Specifically, the control section 11 associates the boundary number indicating the order of appearance of each boundary with the time of each boundary, and stores into, for example, a DB provided in the storage section 12. Note that, in a case where each boundary is given a name, the name of each boundary can also be stored. Then, the control section 11 sets (optimizes) the parameters of the operation processing in the intermediate layer and the output layer of the motion analysis model M4 based on the time of each boundary (S22). Here, the control section 11 sets the parameters of the motion analysis model M4 in such a manner that the set-up animation input to the motion analysis model M4 can be distinguished for each time of each boundary stored in step S21. In addition, the control section 11 sets the parameters of the motion analysis model M4 in such a manner that, based on the analysis result output by the motion analysis model M4 in a case where the set-up animation is input, each motion animation outputs the analysis result indicating a standard motion. Thereby, it is possible to generate the motion analysis model M4 that distinguishes each motion animation in a case where the animation is input, and outputs the analysis result of whether the motion of each distinguished motion animation is a standard motion. Figure 5The processing at the time of setting the parameters of the motion analysis model M4 is shown in FIG. 12, but the parameters can be set by the same processing for the cycle prediction model M2 and the boundary prediction model M3. For example, in the case of setting the parameters of the cycle prediction model M2, the control section 11 inputs the animation of the processing target to the cycle prediction model M2, and based on the output information from the cycle prediction model M2, divides the animation of the processing target into cycle animations, and extracts boundary candidates of each cycle. Then, the control section 11 accepts editing of the boundary candidates of each cycle animation by the user, and based on the times of the boundaries of each cycle animation after editing, can set the parameters of the cycle prediction model M2. Also, in the case of setting the parameters of the boundary prediction model M3, the control section 11 inputs the animation of the processing target to the boundary prediction model M3, and based on the output information from the boundary prediction model M3, divides the animation of the processing target into motion animations, and extracts boundary candidates of each motion. Then, the control section 11 accepts editing of the boundary candidates of each motion by the user, and based on the times of the boundaries of each motion after editing, can set the parameters of the boundary prediction model M3.

[0084] Next, a process of analyzing whether or not the worker is performing a standard work based on an animation obtained by photographing the worker performing the work in the information processing apparatus 10 of the present embodiment will be described. Figure 7 is a flowchart showing an example of an analysis processing sequence of a work, Figure 8 is an explanatory diagram showing an example of a screen. The following processing is performed by the control section 11 of the information processing apparatus 10 according to the program 12P stored in the storage section 12.

[0085] In the case of analyzing whether or not the work of the worker is a standard work, the user selects an animation obtained by photographing the worker to be analyzed. The control section 11 of the information processing apparatus 10 accepts selection of the animation to be analyzed (S31) by operating through the input section 14. The control section 11 inputs the selected animation (animation file 12a) to the type prediction model Ml, and based on the output information from the type prediction model Ml, determines the type of the product (work target) on which the worker photographed in the animation of the work target is performing assembly work (S32). Specifically, the control section 11 sequentially inputs each frame included in the animation file 12a to the type prediction model Ml, and in the case where the largest output value (confidence) from the type prediction model Ml reaches a predetermined value or more (for example, 0.7 or more), determines the type corresponding to the output node from which the largest output value is output as the type of the product on which the worker in the animation is performing assembly. Note that the control section 11 can also determine the type of the work target without using the type prediction model Ml. For example, in the case where the work target includes an article different for each type, the control section 11 can detect the article included in the animation, and determine the type based on the detected article.

[0086] The control section 11 selects the cycle prediction model M2, the boundary prediction model M3, and the motion analysis model M4 corresponding to the determined product being assembled by the worker (S33). Note that a plurality of cycle prediction models M2, boundary prediction models M3, and motion analysis models M4 are prepared for each product. The control section 11 uses the selected cycle prediction model M2 to divide the animation to be analyzed into cycle animations, and extracts the boundaries of each cycle (S34). Here, the control section 11 sequentially inputs each frame included in the animation file 12a into the cycle prediction model M2, and determines (predicts) the start time and end time of each cycle animation included in the animation based on the output information from the cycle prediction model M2. Note that the control section 11 can be configured to display an edit screen for changing the boundaries of each cycle extracted from the animation (deletion, new addition, etc.) in the display section 15 after the process of step S34 (refer to Figure 6A-6C ), and accept a change instruction for the time of the boundary of each cycle via the edit screen and change it.

[0087] Next, the control section 11 (the division section) uses the boundary prediction model M3 selected in step S33 to divide the animation after the boundaries of each cycle are extracted into motion animations that capture the motion performed by the worker, and extracts the boundaries of each motion (S35). Here, the control section 11 sequentially inputs each frame included in the animation that has been divided into cycle animations into the boundary prediction model M3, and determines (predicts) the start time and end time of each motion animation included in each cycle animation based on the output information from the boundary prediction model M3. Note that the control section 11 can be configured to display an edit screen for changing the boundaries (division positions) of each motion extracted from the animation (deletion, new addition, etc.) in the display section 15 after the process of step S35 (refer to Figure 6A-6C ), and accept a change instruction for the time of the boundary of each motion via the edit screen and change it.

[0088] Next, the control section 11 uses the motion analysis model M4 selected in step S33 to analyze whether the motion of the worker captured in each motion animation included in the animation after the boundaries of each motion are extracted is a standard motion (S36). Here, the control section 11 sequentially inputs each frame included in the animation into the motion analysis model M4, and determines (predicts) the start time and end time of each motion animation included in the animation, and an analysis result indicating whether the motion of the worker in each motion animation is a standard motion based on the output information from the motion analysis model M4. The control section 11 (the determination section) can determine whether the motion performed by the worker is a standard motion based on the analysis result.

[0089] The control section 11 (analysis section) analyzes whether the work elements performed by the work personnel photographed in each cycle animation in the animation are standard work based on the analysis result of the motion animation included in the cycle animation (S37). Here, the control section 11 determines that the work elements of the work personnel in the cycle animation are not standard work in a case where the analysis result of any of the motion animations included in the cycle animation is not standard motion. For example, in a case where any of the motion animations included in the cycle animation in which one work element is photographed is missing, the control section 11 determines that the work elements of the work personnel in the cycle animation are not standard work due to the omission of the motion. In addition, in a case where the appearance order of any of the motion animations included in the cycle animation is different, the control section 11 determines that the work elements of the work personnel in the cycle animation are not standard work due to the exchange of the order. In addition, in a case where the motion of the work personnel in any of the motion animations included in the cycle animation is different from the standard motion, the control section 11 determines that the work elements of the work personnel in the cycle animation are not standard work due to the abnormal activity. In addition, in a case where the time (the time of the motion performed by the work personnel) in any of the motion animations included in the cycle animation is longer than the standard motion time set in advance by a predetermined time or more, the control section 11 determines that the work elements of the work personnel in the cycle animation are not standard work due to the long motion time, and in a case where the time is shorter than the standard motion time by a predetermined time or more, the control section 11 determines that the work elements of the work personnel in the cycle animation are not standard work due to the short motion time. Such analysis processing can be performed using the main factor classification model M5. In this case, the control section 11 inputs the cycle animation into the main factor classification model M5, and analyzes whether the work elements are standard work based on the output information from the main factor classification model M5. Note that the motion content considered to be not standard work is not limited to the above examples. The control section 11 determines that the work elements of the work personnel in the cycle animation are standard work in a case where the analysis result of all of the motion animations included in the cycle animation is standard motion.

[0090] The control section 11 displays the analysis result of each cycle animation in the display section 15 (S38). For example, the control section 11 generates the analysis result screen shown in FIG. 9, and displays it in the display section 15. Figure 8 Figure 8 ​The illustrated analysis result screen has an animation area A3 that displays the animation of the analysis target, and a result area A4 that displays analysis results of the actions included in the work elements performed by the worker in the animation. In the result area A4, for the actions performed by the worker in each action animation distinguished from the animation, an action number indicating the order of appearance, a name given to each action, and an analysis result are displayed. The analysis result includes an action time required for the execution of each action, and a mark (O or X) indicating whether each action is a standard action. Note that a play stop button B3 for instructing the play and stop of the animation in display is provided in the animation area A3, and in a case where play instruction is made via the play stop button B3, the control section 11 causes the animation of the analysis target to be played in the animation area A3, and displays the analysis results of each action animation analyzed based on the played animation in the result area A4. Thus, it is possible to grasp the analysis results while confirming the actions of the worker with the played animation.

[0091] Through the above processing, the information processing apparatus 10 identifies the work elements performed by the worker and each action included in each work element based on the animation obtained by photographing the worker, analyzes whether each action is a standard action on a per-action basis, and analyzes whether each work element is a standard work based on the analysis results of each action. In this way, the work performed by the worker is divided into action units, and it is determined whether each action is appropriate (specifically, whether it is a standard action), and by using the results thereof, it is possible to determine whether the work elements (one cycle of work) of the worker are appropriate with higher accuracy. In addition, whether each action is appropriate is determined using the action analysis model M4, and thus, it is determined whether a standard action is performed not only considering the action time of each action but also the action content (the activity of the worker). Thus, it is possible to analyze with high accuracy whether the worker is performing a standard action (work according to the regulations) set in advance to the action analysis model M4. In addition, it is possible to prompt the inappropriate action and the work element including the inappropriate action based on the results obtained by determining whether each action is appropriate, and thus, it is easy to give improvement guidance to the worker.

[0092] The control section 11 determines whether the above processing is completed for all frames included in the animation of the analysis target (S39), and in a case where it is determined that the processing is not completed (S39: NO), returns to the processing of step S34, and repeats the processing of steps S34 to S38. Thus, it is possible to analyze whether each work element performed by the worker is appropriate based on each cycle animation included in the animation of the analysis target, and it is possible to prompt the user with the obtained analysis results.

[0093] The control section 11, in a case where it is determined that the above processing is ended for the animation of the analysis target (S39: YES), associates the analysis result with the animation of the analysis target and stores it in the storage section 12 (S40). For example, the control section 11, for each cycle animation extracted from the animation of the analysis target, stores identification information (for example, cycle number indicating the order of appearance), the time of the boundary of each cycle, and the analysis result for the work element (cycle) in the DB prepared by the storage section 12, and, for each motion animation in each cycle animation, stores identification information (for example, motion number indicating the order of appearance), the time of the boundary of each motion, and the analysis result for the motion in the DB prepared by the storage section 12. Through the above processing, based on the animation obtained by photographing the worker performing the work, it is analyzed whether the worker is performing the standard motion that is set in advance in the motion analysis model M4, and the analysis result is displayed to the user.

[0094] In the above processing, based on the animation of the analysis target, the kind of work that the worker is working on is determined, and the analysis processing using the motion analysis model M4 prepared for each kind is executed. Therefore, for example, even in a case where assembly work of different kinds of kinds is performed in the same work site, analysis processing can be performed for each kind, and the user who performs the analysis processing can perform analysis processing for each animation with high accuracy without being aware of the kind in each animation.

[0095] The analysis result for each animation obtained based on the above analysis processing can grasp the tendency of the work content performed by the worker, and is used for research on improvement strategies for assembly work. For example, from the analysis result for each cycle animation included in each animation, it is possible to analyze the tendency of each main factor in a case where the work element in each cycle animation is not the standard work and take countermeasures. Figure 9 is a flowchart showing an example of the analysis processing sequence for each work element, Figure 10A and Figure 10B is an explanatory diagram showing a screen example. The following processing is performed by the control section 11 of the information processing apparatus 10 according to the program 12P stored in the storage section 12.

[0096] As for each work element performed by the worker, in a case where the occurrence condition of the main factor in a case where it is not the standard work is analyzed, the user selects an animation obtained by photographing the worker performing the work, which is obtained by Figure 7The processing involves analyzing and processing the animation of each work element. The animation being processed here is obtained by filming workers assembling a particular product. It should be noted that the following processing can be performed to analyze the work content of a single worker across multiple tasks, or to analyze the work content of multiple workers across multiple tasks.

[0097] The control unit 11 extracts a looping animation from the animation of the object being processed (S51). Based on the extracted looping animation, it determines that the work elements performed by the worker in the looping animation are not the main factors of the standard operation (S52). Here, the control unit 11 sequentially inputs each frame contained in the looping animation into the main factor classification model M5, and based on the output information from the main factor classification model M5, determines that the worker's work in the looping animation is not the main factor of the standard operation. It should be noted that the control unit 11 can also determine the main factors of the standard operation based on the output information from the main factor classification model M5. Figure 7 The analysis results obtained from the analysis and processing shown for each loop animation determine that the work elements performed by the operators in each loop animation are not the main factors of standard operation. The control unit 11 identifies any main factors such as omission of actions, change of sequence, abnormal activities, long action time, and short action time. The control unit 11 associates the identified main factors with the loop animation of the processing object and stores them in the storage unit 12 (S53). For example, the control unit 11 associates the identified main factors with the loop number of each loop animation and stores them.

[0098] Next, the control unit 11 calculates the occurrence ratio of each major factor based on the identified major factors (S54), and displays the calculated occurrence ratio of each major factor on the display unit 15 (S55). For example, the control unit 11 generates a... Figure 10A The main elements are shown on the screen and displayed on the display unit 15. Figure 10A The main factors screen displays an overview of the work elements included in the assembly operation of product A, showing the order of appearance (cyclic number), name, and occurrence ratio of each main factor for each work element. It should be noted that the occurrence ratio of each main factor includes not only the proportion of cyclic animations judged as not being standard operations, but also the proportion of cyclic animations judged as being standard operations. Since the main factors based on the analysis results of each motion animation have been identified and stored in the storage unit 12 along with the animations, the control unit 11 does not perform steps S52 to S53, but instead reads the main factors based on the analysis results from the storage unit 12 and performs steps S54 to S55.

[0099] The control unit 11 determines whether there are any unprocessed animations that have not undergone the processing steps S51 to S55 (S56). If it determines that there are unprocessed animations (S56: YES), it returns to the processing step S51 and performs the processing steps S51 to S55 on the unprocessed animations. If it determines that there are no unprocessed animations (S56: NO), the control unit 11 moves to the processing step S57.

[0100] Figure 10A The displayed screen is structured to allow users to receive playback instructions for the animations of each key element of a task. For example, a user can execute a pre-defined action (e.g., left-clicking the mouse) on any key element of any task to instruct the playback of the animation of that key element. Figure 10A In this process, the control unit 11 receives a playback instruction for the animation of the main factor that was missed in the action by operating the cursor as shown by arrow A5, for the work element with cycle number 1. The control unit 11 determines whether the main factor to be played has been received (S57). If it is received (S57: YES), it determines the animation of the selected main factor (the animation corresponding to the main factor) (S58). Here, the control unit 11 retrieves the animation of the main factor of the selected work element from the animations stored in the storage unit 12. Then, the control unit 11 reads the retrieved animation from the storage unit 12 and displays it on the display unit 15 (S59). For example, the control unit 11 will display the animation of the main factor that was missed in the action by operating the cursor as shown by arrow A5. Figure 10B The analysis results are displayed on the display unit 15. Figure 10B The image shown has the same characteristics as Figure 8 The screen shown has the same configuration and includes "Go to Previous Animation" and "Go to Next Animation" buttons. The control unit 11 is configured to sequentially play the multiple animations determined in step S58, and the "Go to Previous Animation" button is used via... Figure 10B When the displayed screen is operated, the content displayed in animation area A3 and result area A4 is changed to the previously displayed animation and the analysis results of that animation. Additionally, when the "Go to Next Animation" button is operated, the control unit 11 changes the content displayed in animation area A3 and result area A4 to the animation to be displayed next and the analysis results of that animation. It should be noted that the control unit 11 can be configured to play multiple animations determined in step S58 in any order. For example, it can play the animations in order from oldest to newest or from newest to oldest according to the recording date and time, it can play the animations of each operator together, or it can play them randomly. Furthermore, it can... Figure 10BA plurality of animation areas A3 are provided in the illustrated screen, and a loop animation in which a main factor selected in each animation is occurring is synchronously displayed in each animation area A3.

[0101] The control section 11 skips the processes of steps S58-S59 and ends the above-described processing in a case where it is determined that the main factor to be played is not accepted (S57: NO). With the above-described processing, each loop animation included in an animation can be grouped by each main factor considered not to be a standard operation, and the occurrence ratio of each main factor can be analyzed. In addition, an animation in which each main factor is occurring can be played by each main factor, and thus, research on an improvement strategy can be efficiently performed by each loop animation (operation element).

[0102] In the present embodiment, for each animation, an analysis result of whether or not it is a standard operation can be obtained by each loop (operation element), and an analysis result of whether or not it is a standard action can be obtained by each action included in each loop. Thus, in addition to the analysis result of each loop animation, searching of an animation based on the analysis result of each action animation can be performed. For example, for a predetermined loop (for example, a first operation element), an animation in which a predetermined action (for example, a first action) is determined not to be a standard action can be searched. Thus, in verifying the operation content of each operator, searching processing considering the analysis result of each loop and the analysis result of each action can be performed, and efficient verification processing can be realized.

[0103] Next, a process of analyzing a deviation of an action time of an action included in each operation element based on an analysis result of each action animation included in an animation will be described. Figure 11 is a flowchart showing an example of a processing sequence of analyzing an action time of each action, Figure 12 is an explanatory diagram showing an example of a screen. The following processing is performed by the control section 11 of the information processing apparatus 10 according to the program 12P stored in the storage section 12.

[0104] In a case where a deviation of an action time of each action performed by an operator is analyzed, a user selects an animation to be analyzed, which is an animation obtained by photographing an operator performing an assembly operation of one kind of product. The animation to be analyzed can be a plurality of animations obtained by photographing one operator, or an animation obtained by photographing a plurality of operators. Figure 7 The processing of the animation after the analysis processing of each action is performed. Here also, the animation to be processed is an animation obtained by photographing an operator performing an assembly operation of one kind of product, and can be a plurality of animations obtained by photographing one operator, or an animation obtained by photographing a plurality of operators.

[0105] The control section 11 measures the motion time of each motion included in the animation of the processing target based on each motion animation included in the animation (S71). For example, the control section 11 acquires the motion number and the start time and the end time (the time of each boundary) of each motion animation from each animation. Then, the control section 11 calculates the time from the start time to the end time for each motion animation. The control section 11 generates a scatter plot (chart) obtained by associating and plotting the motion time calculated for each motion with each motion, and displays it in the display section 15 (S72). For example, the control section 11 generates and displays the scatter plot shown in the upper right of the screen of FIG. 13. Figure 12 Figure 12 The scatter plot shown in the upper right of FIG. 13 shows the deviation of the motion time of each motion included in the work element "Assembly of Box A" included in the assembly work of Variety A. Specifically, each motion is associated with each position of the horizontal axis, and the vertical axis indicates the motion time of each motion, and the motion time of each motion is plotted in association with each position of the horizontal axis. Note that the control section 11 measures the standard motion time of each motion based on each motion animation in the animation obtained by photographing the worker who performs the standard work using the work model M4 as the setting animation, and plots the measured standard motion time of each motion on the scatter plot with a white diamond. Thereby, the standard motion time of each motion can be prompted, and the comparison result of the motion time of each worker with respect to the standard motion time can be easily grasped.

[0106] Note that the control section 11, in a case where the motion time of each motion has been measured for one cycle animation, aggregates the motion time of each motion, thereby calculating the motion time (cycle time) in the work element. Then, as shown in the upper left of the screen of FIG. 14, the control section 11 also generates and displays a scatter plot obtained by plotting the calculated cycle time. Thereby, in addition to the deviation of the motion time of each motion, the deviation of the cycle time required for the work element (cycle) can be prompted. Figure 12

[0107] Next, the control section 11 calculates, for each motion, the minimum value, the first quartile, the median, the third quartile, and the maximum value of the motion time based on the motion time measured in step S71 (S73). Then, the control section 11 generates a box plot (chart) indicating the deviation of the motion time of each motion based on each value calculated for each motion, and displays it in the display section 15 (S74). For example, the control section 11 generates and displays the box plot shown in the lower right of the screen of FIG. 15. Figure 12 Figure 12 ​​​the box plot and the scatter plot, each action is associated with each position of the horizontal axis, and the vertical axis indicates the action time of each action, and the deviation of the action time is represented by a box and a line that respectively indicate the minimum value, the first quartile, the median value, the third quartile, and the maximum value of the action time of each action. Here, as well, the control unit 11 draws the standard action time of each action on the box plot with a white diamond. Also, as well as the cycle time, the control unit 11 calculates the minimum value, the first quartile, the median value, the third quartile, and the maximum value, and based on each of the calculated values, the control unit 11 displays a box plot that represents the deviation of the cycle time on the display unit 15, as shown in the lower left of the screen of FIG. 9. Figure 12

[0108] The control unit 11 determines whether there is unprocessed animation for which the processes of steps S71 to S74 have not been executed (S75), and in the case where it is determined that there is unprocessed animation (S75: YES), returns to the process of step S71 to execute the processes of steps S71 to S74 for the unprocessed animation. In the case where it is determined that there is no unprocessed animation (S75: NO), the control unit 11 moves to the process of step S76.

[0109] Figure 12 The scatter plot in the screen shown in FIG. 10 is configured to accept a play instruction for the animation corresponding to each marker via each drawn point (marker). For example, the user can make a play instruction for the animation corresponding to the marker that is operated by performing a predetermined operation (for example, left-click of a mouse) on an arbitrary marker. In the case where the user performs a predetermined operation on a marker, the control unit 11 determines the marker on which the operation is performed, and determines the animation corresponding to the marker. Figure 12 In the case where the user performs a predetermined operation on a marker, the control unit 11 determines the marker on which the operation is performed, and determines the animation corresponding to the marker. In the scatter plot, the marker drawn in association with each action is associated with the identification information of the animation (for example, animation ID assigned to the animation), and therefore, the control unit 11 can determine the animation corresponding to the marker by determining the identification information of the animation corresponding to the selected marker. Thus, the control unit 11 can determine the animation to be played that is selected via the marker in the scatter plot from the animations stored in the storage unit 12.

[0110] Then, the control unit 11 reads out the determined animation from the storage unit 12 and displays it on the display unit 15 (S78). In this case, the control unit 11 displays a box plot that represents the deviation of the action time of the determined animation based on the determined animation, as shown in the lower left of the screen of FIG. 11. Figure 8 ​The illustrated analysis result screen. Note that the control section 11 can be configured to display the motion animation of the determined animation corresponding to the selected marker. In this case, the motion animation of the selected motion can be confirmed. The control section 11 skips the processing of steps S77-S78 and ends the above processing in a case where it is judged that no marker to be played is accepted (S76: NO). Through the above processing, for each cycle animation included in the animation, the deviation of the motion time of each motion included in the cycle animation can be prompted using the scatter plot and the box plot together with the deviation of the cycle time. In addition, the animation selected via each marker of the scatter plot can be played. Thus, the investigation of the improvement strategy can be performed for each work element or for each motion, taking into account the deviation of the cycle time of each work element and the deviation of the motion time of each motion.

[0111] Next, a process of making a Gantt chart indicating the progress status of each work element performed by the worker in the animation using the result of dividing the animation into cycle animations will be described. Figure 13 is a flowchart showing an example of the processing sequence of making a Gantt chart, Figure 14 is an explanatory diagram showing an example of a Gantt chart screen. The following processing is performed by the control section 11 of the information processing apparatus 10 according to the program 12P stored in the storage section 12.

[0112] In a case where a Gantt chart indicating the progress status of each work element performed by the worker is made, the user selects the animation to be processed. The animation to be processed here is the animation divided into cycle animations using the cycle prediction model M2, and can also be, for example, the animation after the analysis processing of each work element is performed by the processing of Figure 7 The control section 11 accepts the selection of the animation to be made into a Gantt chart via the operation of the input section 14 (S81). The control section 11 acquires the start time and the end time of each cycle animation divided from the selected animation (S82). For the animation divided into cycle animations, the start time and the end time of each cycle animation (the time of the boundary of each cycle) are associated with the animation and stored in the storage section 12, and the control section 11 reads out the start time and the end time of each cycle animation from the storage section 12.

[0113] The control section 11 makes a strip chart (horizontal bar) indicating the execution time period from the work start timing of each work element to the work end timing based on the acquired start time and end time of each cycle animation, and makes a Gantt chart (chart) obtained by arranging the made strip charts of each work element (S83). For example, the control section 11 makes a Gantt chart as shown in Figure 14 . Figure 14The Gantt chart shown displays the names of the work elements included in the assembly operation of product A in the vertical direction, the horizontal axis represents the date and time, and the strip chart showing the execution time period of each work element is displayed in relation to each work element. The control unit 11 displays the created Gantt chart in the display unit 15 (S84). Using such a Gantt chart, the progress of the work performed by each operator can be easily grasped. Figure 14 The Gantt chart shown can be configured, for example, to display a strip chart showing the start and end times of each action within the selected work element when each work element is selected. In this case, a Gantt chart can be provided that indicates not only the progress of each work element but also the progress of each action within each work element. It should be noted that when a Gantt chart is created based on animation obtained from filming workers performing standard operations, a work schedule (standard operation combination ticket) can be created that specifies the sequence of work elements within a series of operations and the standard operation time.

[0114] Next, we will explain how to display the analysis results of multiple animations side-by-side. The following explanation uses the example of displaying the analysis results of two animations side-by-side, but it can also be done by displaying the analysis results of three or more animations side-by-side. Figure 15 This is an explanatory diagram showing an example of displaying two analysis result screens. The control unit 11 of the information processing device 10 receives an execution instruction to compare the analysis results of two animations. Upon receiving a selection of two animations that have already undergone analysis processing, it generates analysis result screens for each animation based on the selected animation and the analysis results for that animation, and displays them side-by-side on the display unit 15. Figure 15 In the example shown, the analysis results for each animation are displayed on the left and right sides of the screen. Each analysis result screen has a [specific feature / function]. Figure 8 The same structure, but the result area A4 is displayed below the animation area A3. Additionally, it is... Figure 15 The results area A4 displays information related to the motion animation currently being displayed in the animation area A3 and the motion animations before and after it. As the animation in the animation area A3 plays, the analysis results of the displayed motion animation are displayed sequentially.

[0115] exist Figure 15 The screen shown includes: a "Align at Start" button to move the playback position of the two animations displayed in animation area A3 to the beginning position; a "Stop Both Animations Simultaneously" button to stop both animations at the same time; and a "Motion Comparison Mode" button to synchronize and display the individual actions in the two animations. Additionally, in Figure 15The illustrated screen is provided with an animation change button for instructing a change of the animation displayed in each animation area A3. In a case where the animation change button is operated, the control section 11 of the information processing apparatus 10 displays the animation names of the selectable animations in a list, and in a case where the selection of an arbitrary animation is accepted, switches the display contents of the animation area A3 and the result area A4 based on the selected animation and the analysis result of the animation. Thereby, the analysis result of an arbitrary animation can be compared and displayed.

[0116] In a case where the action comparison mode button is operated, the control section 11 can synchronize (link) each action animation included in each animation in each action and display in the animation area A3. Thereby, the action contents of the workers in two animations can be compared for each action. Note that the control section 11 can also synchronize each cycle animation included in each animation from the beginning and display in the animation area A3 in order for each action. Thereby, the control section 11 can juxtapose the analysis results of an arbitrary two animations, and in addition, can display the action animation or the cycle animation for each action or for each work element. Thereby, for example, in a case where animations captured in a case where different workers perform the same kind of assembly work are displayed in synchronization, the work states of the two workers can be compared. In addition, in a case where animations captured in a case where one worker performs the same kind of assembly work are displayed in synchronization, the work states of the one worker performed on different dates, for example, can be compared.

[0117] In the present embodiment, based on the animation obtained by capturing the worker performing the work, whether each action performed by the worker is a standard action is determined, and based on the determination result of each action, whether each work element is a standard work is determined. In addition, since the determination of whether each action is a standard action is performed using the action analysis model M4, whether a standard action is performed is determined with high precision also taking into account the action contents (the activities of the worker) of each action. By using the result of such high-precision determination, appropriate improvement measures can be studied, and improvement guidance to the worker can be performed.

[0118] In the present embodiment, Figure 7 The illustrated analysis processing can also be performed in real time based on the animation obtained by capturing the worker during the work, in addition to the configuration performed after the work ends based on the animation obtained by capturing the worker. In this case, for the assembly work being performed by the worker, whether each action is a standard work is determined, and based on the determination result, the processing of determining whether each work element is a standard work can be performed, and improvement guidance can also be performed during the performance of the work.

[0119] In the present embodiment, at least one of the prediction process of the genre using the genre prediction model Ml, the genre of the cycle animation using the cycle prediction model M2, the genre of the motion animation using the boundary prediction model M3, and the analysis process using the motion analysis model M4 is not limited to being constituted by the information processing apparatus 10 locally. For example, a server that performs the prediction process of the genre using the genre prediction model Ml can be provided. In this case, the information processing apparatus 10 is constituted so as to transmit the animation of the processing target to the server, and the genre predicted by the server using the genre prediction model Ml is transmitted to the information processing apparatus 10. In this case as well, the information processing apparatus 10 can perform, for example, the process after step S33 based on the genre predicted by the server. Figure 7

[0120] In addition, a server that performs the genre of the cycle animation using the cycle prediction model M2 can be provided. In this case, the information processing apparatus 10 is constituted so as to transmit the animation of the processing target to the server, and the animation that is distinguished as the cycle animation by the server is transmitted to the information processing apparatus 10. In this case as well, the information processing apparatus 10 can perform, for example, the process after step S35 based on the animation that is distinguished as the cycle animation by the server. Figure 7 In addition, a server that performs the genre of the motion animation using the boundary prediction model M3 can be provided. In this case, the information processing apparatus 10 is constituted so as to transmit the animation of the processing target to the server, and the animation that is distinguished as the motion animation by the server is transmitted to the information processing apparatus 10. In this case as well, the information processing apparatus 10 can perform, for example, the process after step S36 based on the animation that is distinguished as the motion animation by the server. Figure 7 In addition, a server that performs the analysis process using the motion analysis model M4 can be provided. In this case, the information processing apparatus 10 is constituted so as to transmit the animation of the processing target to the server, and the result of the analysis process on the animation by the server is transmitted to the information processing apparatus 10. In this case as well, the information processing apparatus 10 can perform, for example, the process after step S37 based on the result of the analysis process by the server. Figure 7 In the case of being constituted as described above, the same process as in the present embodiment can be performed, and the same effect can be obtained.

[0121] In the present embodiment, the constitution is based on the animation obtained by photographing the worker who performs the work to analyze whether the motion and the work of the worker are a standard motion and a standard work, but the analysis target is not limited to the worker. For example, a constitution can be such that the motion and the work of a robot constituted so as to perform a predetermined work are analyzed based on the animation obtained by photographing the robot. ​

[0122] (Embodiment 2)

[0123] In the present embodiment, an information processing apparatus that extracts a cycle animation in which each work element is photographed from an animation obtained by photographing how a work person repeatedly performs one work element (the same work element), compares the extracted cycle animations, and gives a prompt is described. Note that the cycle animation included in one animation can be obtained by photographing one work person or can be obtained by photographing different work persons. The information processing apparatus 10 of the present embodiment has the same configuration as the information processing apparatus 10 of Embodiment 1 shown in FIG. 1, and thus a description of the configuration is omitted. Note that the storage section 12 of the information processing apparatus 10 of the present embodiment stores a cycle animation extraction model M6 instead of the cycle prediction model M2. Figure 2

[0124] Figure 16 is an explanatory diagram showing a configuration example of the cycle animation extraction model M6. The cycle animation extraction model M6 is configured using an algorithm such as RNN, LSTM, CNN, or Transformer, and can also be configured by combining a plurality of algorithms. The cycle animation extraction model M6 is a learning completed model that has been learned in such a way that an animation obtained by photographing a work person repeatedly performing one work element and a start time and an end time of one cycle animation specified for the animation are input, an operation of extracting a cycle animation similar to the specified cycle animation from the input animation is performed, and the operation result is output. The cycle animation extraction model M6 can also be configured as long as an animation in which a work person is photographed performing an action is input, like an animation obtained by photographing a work person's whole body, upper body, or hands.

[0125] The cycle animation extraction model M6 has an input layer that inputs an animation and a start time and an end time of one cycle animation in the animation, an intermediate layer that extracts a feature amount of a work person's action included in a cycle animation in which the start time and the end time are input and a feature amount of a work person's action included in the animation, and performs an operation of extracting a cycle animation having the same feature amount as the cycle animation (a cycle animation similar to the cycle animation) from the input animation, and an output layer that outputs a start time and an end time of each cycle animation extracted from the animation based on the operation result of the intermediate layer. The start time and the end time of each cycle animation output from the cycle animation extraction model M6 indicate elapsed time from the start of the animation.

[0126] ​The cycle animation extraction model M6 can be generated by machine learning using training data containing a training animation and start and end times of one cycle animation in the animation, and start and end times of a plurality of cycle animations designated as correct for the animation. The cycle animation extraction model M6 learns in such a manner that, when the animation and the start and end times of one cycle animation contained in the training data are input, the start and end times of each cycle animation that is correct are output. In the learning process, the cycle animation extraction model M6 performs operations in the intermediate layer and the output layer based on the input animation and the start and end times of one cycle animation, extracts features of the motion of the worker contained in the animation and the cycle animation, extracts a cycle animation having the same features as the extracted cycle animation from the animation to be processed, compares the start and end times of each cycle animation extracted with the start and end times of each cycle animation that is correct in such a manner that they are close to each other, and optimizes the parameters for the operation process in the intermediate layer and the output layer in such a manner. The optimized parameters are weights (coupling coefficients) between nodes in the intermediate layer and the output layer, coefficients of functions, threshold values, and the like, and the optimization method of the parameters can use a backpropagation method, a gradient descent method, and the like. As a result, the cycle animation extraction model M6 that distinguishes each cycle animation in the animation and outputs the start and end times of each cycle animation when the animation and the start and end times of one cycle animation are input is obtained. The learning of the cycle animation extraction model M6 can also be performed with another learning device, in which case the learning-completed cycle animation extraction model M6 generated by the other learning device is downloaded from the learning device to the information processing device 10 via the network N or via the mobile storage medium 10a, and stored in the storage section 12.

[0127] Next, a process of extracting cycle animations in which the appearance of the work element of the worker is photographed from an animation in which the worker is photographed in the information processing device 10 of the present embodiment, and comparing and prompting a plurality of extracted cycle animations will be described. Figure 17 is a flowchart showing an example of the comparison process sequence of cycle animations, Figure 18A-21B is an explanatory diagram showing an example of a screen.

[0128] In the case of confirming the appearance of the work performed by the worker using the cycle animation, the user selects an animation to be processed. The control section 11 of the information processing device 10 displays an animation selection screen such as that shown in Figure 18A is a flowchart showing an example of the comparison process sequence of cycle animations, Figure 18AIn the example, Animation 1 and Animation 2 are selected. The animation of the object is a loop animation that records the time it takes for the worker (object) to perform a series of work elements, and it contains multiple loop animations. It should be noted that... Figure 18A The screen is composed of thumbnails of the animations shot with the selected camera, which are displayed by selecting the camera.

[0129] Next, the user specifies the start and end times of a looping animation for the selected animation. Control unit 11 displays, for example, as follows: Figure 18B The screen shown is used to receive the start and end times of a looping animation (S92). Figure 18B The screen includes: a display bar for the selected animation and an input bar R1 for entering the start and end times of a looping animation within the displayed animation. An indicator showing the playback time of the displayed animation is displayed in input bar R1. The user specifies the start and end times of a looping animation by moving the start marker C1 and end marker C2 set in input bar R1 to the desired playback time. The user can then use the zoom in / out button C3 to zoom in or out on the interval of the playback time indicated by the indicator.

[0130] A "Find Similar Jobs" button is provided in the input field R1. This button is used to instruct the extraction of a looping animation similar to the looping animation of the processing object, based on a specified start and end time. The control unit 11 determines whether the "Find Similar Jobs" button has been operated (S93). If it is determined that it has not been operated (S93: NO), it returns to step S92 and continues to accept the input of the start and end times of a looping animation. If the control unit 11 determines that the "Find Similar Jobs" button has been operated (S93: YES), it extracts a looping animation similar to the looping animation specified in step S92 from the animation selected in step S91 (S94). Here, the control unit 11 inputs the selected animation (animation file 12a) and the specified start and end times of a looping animation into the looping animation extraction model M6, and obtains the start and end times of each looping animation extracted from the animation of the processing object as output information from the looping animation extraction model M6.

[0131] The control unit 11 displays the start time and end information of each loop animation extracted from the animation (S95). For example, such as Figure 18CAs shown, the control section 11 displays each loop animation with a rectangle C4 indicating the start time and the end time with respect to the play time of the animation indicated by the pointer of the input field Rl, and displays the start time and the end time of each loop animation in the animation information field R2. In the present embodiment, the control section 11 assigns the name of the video 1, the video 2,... to each loop animation extracted from the animation, and displays the video 1, the video 2,... in the animation information field R2, which shows each loop animation. Figure 18C

[0132] The control section 11 extracts one of the plurality of loop animations extracted from the animation (S96), and divides the extracted loop animation into motion animations using the boundary prediction model M3 (S97). The process of step S97 is the same as that of step S35 in Figure 7 , the control section 11 sequentially inputs each frame included in the loop animation into the boundary prediction model M3 corresponding to the variety in which the worker in the animation is working, and determines (predicts) the start time and the end time of each motion animation included in the loop animation based on the output information from the boundary prediction model M3. After the process of step S97, the control section 11 can perform editing processing for changing the boundary (division position) of each motion animation (deletion and new addition, etc.) (refer to Figure 6A-6C ). Note that in the present embodiment, the setting is made in such a manner that when the loop animation is divided into motion animations using the boundary prediction model M3, the loop animation is divided into a predetermined number (for example, 10) of motion animations. The number of divisions of the motion animations can be decided in advance and set in the boundary prediction model M3, or can be arbitrarily set by the user, in which case, it can be a configuration in which the set number of divisions is input as the input data of the boundary prediction model M3.

[0133] The control section 11 determines whether the process of steps S96 to S97 is completed for all the loop animations extracted from the animation in step S94 (S98), and in the case where it is determined that the process is not completed (S98: NO), returns to step S96 to repeat the process of steps S96 to S97 for the loop animations which have not been processed. Thus, all the loop animations are divided into a plurality of motion animations. In the case where it is determined that the process is completed for all the loop animations (S98: YES), the control section 11 displays analysis information of each loop animation (S99). For example, the control section 11 counts the required time (the loop time from the start time to the end time) of each loop animation, generates a scatter plot (chart) obtained by plotting the loop time of each loop animation, and a box plot showing the minimum value, the first quartile, the median value, the third quartile, and the maximum value of the loop time, and displays them in the analysis information field R3 as shown in Figure 18D . Figure 18D ​In the scatter plot, the vertical axis (vertical axis) represents the loop time. The points (markers) plotted in the scatter plot are configured such that, upon performing a predetermined action (e.g., selection using the cursor), the loop time corresponding to the operated mark is displayed, and that, upon performing a predetermined action (e.g., left-clicking the mouse), the corresponding loop animation begins playback. Figure 18D In the example, control unit 11 displays the loop time of the looping animation corresponding to the mark selected by the cursor indicated by arrow C5.

[0134] Figure 18C The animation information bar R2 in the screen is configured to rearrange the display order of looping animations extracted from the animation of the processed object, and includes a rearrangement button C6 to indicate whether to perform the rearrangement. Figure 19A As shown, the rearrangement button C6 has a drop-down menu that allows users to select any one of the following as the rearrangement rule: start time order, job order from fastest to slowest, or job order from slowest to fastest. Figure 19B In the example, the rearrangement rule is selected to sort the tasks from slowest to fastest, and the names, start times, end times, and loop times of each loop animation are displayed in order of longest to shortest loop time.

[0135] exist Figure 19A and Figure 19B The animation information bar R2 in the screen has checkboxes for selecting each looping animation, and is configured such that when two or more looping animations are selected, ... Figure 19B The "Centralized Comparison" button is displayed as shown. After the user selects the looping animations they want to compare and confirm using the checkboxes, they press the "Centralized Comparison" button. The control unit 11 accepts the selection of the looping animations of the analysis object by checking the checkboxes in the animation information bar R2 (S100). If two or more looping animations are selected, it determines whether the "Centralized Comparison" button has been pressed (S101). If it is determined that the "Centralized Comparison" button has not been pressed (S101: NO), the control unit 11 returns to step S100 and continues to accept the selection of looping animations.

[0136] In a case where it is determined that the "central comparison" button is operated (S101: YES), the control section 11 determines a reference cycle animation (reference animation) that should serve as a template from the cycle animations selected in step S100 (S102). For example, the control section 11 determines a cycle animation having a cycle time of a central value as the reference cycle animation. Note that the control section 11 can also determine a cycle animation having a cycle time closest to an average value of cycle times of the cycle animations as the reference cycle animation, or a cycle animation having the shortest cycle time as the reference cycle animation. In addition, the control section 11 can determine a cycle animation selected by the user via the input section 14 as the reference cycle animation. In addition, the control section 11 can calculate a degree of similarity to other cycle animations for each of the selected cycle animations, and determine a cycle animation having the highest total of degrees of similarity to other cycle animations as the reference cycle animation. The degree of similarity can be, for example, a correlation coefficient, a cosine similarity. In addition, the control section 11 can be configured to estimate a degree of similarity of two cycle animations using a learning model constructed through machine learning. For example, a learning model constituted by a CNN and learned so as to output a degree of similarity of two cycle animations when the two cycle animations are input can be used. In this case, the control section 11 can input two cycle animations to the learning model for which learning is completed, and estimate a degree of similarity of the two cycle animations based on output information from the learning model. Note that the control section 11 is not limited to the configuration of determining the reference cycle animation from the selected cycle animations, and can determine the reference cycle animation from the cycle animations extracted in step S94. In this case, the control section 11 can determine a cycle animation having a central value, an average value, or the shortest cycle time of cycle times from the cycle animations extracted in step S94 as the reference cycle animation, or a cycle animation having the highest total of degrees of similarity to other cycle animations as the reference cycle animation.

[0137] The control section 11 generates a comparison screen for comparing the reference cycle animation and the cycle animation to be analyzed (S103). For example, the control section 11 generates a comparison screen as shown in FIG. 10, and displays the generated comparison screen in the display section 15 (S104). In the comparison screen shown in FIG. 10, a cycle animation 1010 to be analyzed and a reference cycle animation 1020 are displayed in a side-by-side manner. In addition, a cycle time of the cycle animation 1010 to be analyzed and a cycle time of the reference cycle animation 1020 are displayed in a superimposed manner. In addition, a degree of similarity of the cycle animation 1010 to be analyzed to the reference cycle animation 1020 is displayed in a superimposed manner. Figure 19C Figure 19C ​In the screen of FIG. 15, the cycle animation of the video 115 is displayed as a reference cycle animation (template animation), and the cycle animation of the video 2 is displayed as an analysis target. Specifically, the reference cycle animation (video 115) is displayed on the left side of the animation display field R4, and the cycle animation of the analysis target (video 2) is displayed on the right side. Note that the reference cycle animation and the cycle animation of the analysis target in the display are configured to be able to be changed to an arbitrary cycle animation from the cycle animations selected in step S100 by operation through the input section 14. Specifically, the switching buttons C7 and C8 of the reference cycle animation and the switching buttons C8 of the analysis target are provided in the animation display field R4, and a drop-down menu capable of selecting any one of the selectable cycle animations is provided in the switching buttons C7 and C8. In addition, the cycle animation of the analysis target is provided with, for example, a switching button C9 capable of being switched in the order of the animation information field R2. Figure 19B

[0138] In the screen of FIG. 15, the cycle animation of the video 115 is displayed as a reference cycle animation (template animation), and the cycle animation of the video 2 is displayed as an analysis target. Specifically, the reference cycle animation (video 115) is displayed on the left side of the animation display field R4, and the cycle animation of the analysis target (video 2) is displayed on the right side. Note that the reference cycle animation and the cycle animation of the analysis target in the display are configured to be able to be changed to an arbitrary cycle animation from the cycle animations selected in step S100 by operation through the input section 14. Specifically, the switching buttons C7 and C8 of the reference cycle animation and the switching buttons C8 of the analysis target are provided in the animation display field R4, and a drop-down menu capable of selecting any one of the selectable cycle animations is provided in the switching buttons C7 and C8. In addition, the cycle animation of the analysis target is provided with, for example, a switching button C9 capable of being switched in the order of the animation information field R2. Figure 19C Figure 19C The analysis information field R3 in the screen of FIG. 15 displays a graph D1 for the reference cycle animation and the cycle animation of the analysis target displayed in the animation display field R4, and the graph D1 shows the comparison result of the required time of each motion animation (described as interval 1, interval 2,... in the analysis information field R3). The graph D1 associates each motion (each interval) with each position of the horizontal axis, and the vertical axis indicates the difference in the required time of each motion animation of the cycle animation of the analysis target with respect to the required time of each motion animation (interval 1, interval 2,...) of the reference cycle animation. In the graph D1, the required time of each motion animation of the reference cycle animation is plotted with a white circle, and the required time of each motion animation of the cycle animation of the analysis target is plotted with a black circle. With the graph D1, it is possible to compare the reference cycle animation and the cycle animation of the analysis target in the unit of motion animation (interval unit), and it is possible to easily grasp whether the required time of the cycle animation of the analysis target is long or short (i.e., whether each motion is fast or slow) with the reference cycle animation as a reference. In addition, the analysis information field R3 displays a scatter plot obtained by plotting the cycle time of each cycle animation selected in step S100, and a box plot showing the minimum value, the first quartile, the median value, the third quartile, and the maximum value of the cycle time. Figure 19C The scatter plot of the cycle time of the reference cycle animation is plotted with a white circle, the cycle time of the cycle animation of the analysis target is plotted with a large black circle, and the cycle time of the other cycle animations is plotted with a small black circle.

[0139] In addition, the analysis information field R3 displays the cycle time of each cycle animation selected in step S100 in the lower side of the graph D1 and the scatter plot of the cycle time as in the screen of FIG. 16. Figure 20A ​​The diagram shows a scatter plot and box plot of the required time for each motion animation (interval) in the loop animation selected in step S100. In this scatter plot, the horizontal axis represents each interval, and the vertical axis represents the required time for each interval. White circles represent the required time for the baseline loop animation, large black circles represent the required time for the loop animation of the analysis object, and small black circles represent the required time for other loop animations. This allows for indication of the deviation in the required time for each interval of the selected loop animation, which includes the baseline loop animation. Furthermore, the analysis information bar R3 displays a cumulative chart D2, which is obtained by accumulating the required time for each motion animation (interval 1, interval 2…) for each loop animation selected in step S100. Cumulative chart D2 shows a bar chart that accumulates the required time for each motion animation (interval 1, interval 2…) sequentially from bottom to top for each loop animation. It should be noted that cumulative chart D2 displays the bar chart for the baseline loop animation, the bar chart for the loop animation of the analysis object, and the bar charts for other loop animations sequentially from left to right. Figure 20A In the example, only the required time for interval 6 is shown with a shaded line, but the required time for each interval is displayed in the same color (display method). This allows for easy assessment of the required time for each motion animation within each loop. The intervals in the cumulative chart D2 are configured such that, upon performing a predetermined action (e.g., selection using the cursor), the required time for the corresponding loop animation interval is displayed, and that, upon performing a predetermined action (e.g., left-clicking the mouse), playback of the corresponding loop animation interval (motion animation) begins. Figure 20A In the example, interval 6 of video 10 is selected, and the required time for the selected interval 6 is displayed.

[0140] like Figure 19C and Figure 20A As shown, the comparison screen also includes an interval comparison bar R5. In interval comparison bar R5, the horizontal axis represents the playback time of the loop animation, showing the start and end times of each motion animation (each interval) in the baseline loop animation and the loop animation of the object being analyzed. Figure 19C and Figure 20A In the example, the start times of the reference loop animation and the loop animation of the analysis object are made consistent, the start and end times of each motion animation are shown, and a mark C10 indicating the playback position of the two loop animations in the animation display bar R4 is added. The comparison screen of this embodiment is configured such that the reference loop animation and the loop animation of the analysis object are displayed (played) synchronously (in conjunction) according to each motion animation (each interval) obtained by distinguishing each loop animation. For example, in Figure 19C and Figure 20AIn the example of FIG. 15, the required time of the section 1 of the video 115 is longer than the required time of the section 1 of the video 2, and thus, after the two-cycle animations are played, the video 2 waits until the play of the section 1 of the video 115 ends after the play of the section 1 ends, and after the play of the section 1 of the video 115 ends, the play of the section 2 of the two-cycle animations starts. Figure 20B The section comparison field R5 shows the state of the time when the play of the section 1 of the two-cycle animations ends, and the marks C10 indicate the end times of the section 1 of the respective cycle animations, and then, the play of the section 2 of the respective cycle animations starts.

[0141] Further, each section displayed in the section comparison field R5 is configured to, when a predetermined operation (for example, a left click of a mouse) is performed, make the start times of the action animations (sections) of the action being operated (selected) coincide, and show the start times and the end times of the action animations of the respective sections. In Figure 20C In the example of FIG. 15, the section 4 is selected, the start times of the section 4 are made to coincide, and the start times and the end times of the respective sections of the two-cycle animations are shown. Thus, it is possible to easily grasp the length of the required time of the selected section. At this time, it is possible to cyclically play (repeat) the action animations of the selected action (section) in the two-cycle animations in the animation display field R4. Further, when an arbitrary section is selected via the section comparison field R5, the action animations in the two-cycle animations are synchronously displayed in the animation display field R4 for the selected section. Thus, it is possible to compare the actions of the workers in the two-cycle animations for each section. Further, the comparison screen is configured to, during the display of the two-cycle animations in the animation display field R4, when a switching instruction of the analysis objects is performed using the switching buttons C8 and C9, the play of the reference cycle animation from the same section as the section being displayed and the cycle animation of the analysis object being instructed to switch starts at this time. The same applies when a switching instruction of the reference cycle animation is performed using the switching button C7, and the play of the reference cycle animation being instructed to change and the cycle animation of the analysis object being displayed starts at this time from the same section as the section being displayed.

[0142] Further, each section in the section comparison field R5 is configured to, when a predetermined operation (for example, a double click of a mouse) is performed, perform a further division process of the action animations of the section being operated (selected) into detailed action animations. Note that the action animations of the cycle animations being processed are divided into action animations of each action as a large classification by one division process, the action animations of the action of the large classification are divided into action animations of each action as a middle classification by one division process, and the action animations of the action of the middle classification are divided into action animations of each action as a small classification by one division process. In Figure 20CIn the example of FIG. 9, the interval 4 is selected, and in this case, the control section 11 performs the same processing as in step S97 using the boundary prediction model M3 to classify the action animation of the interval 4 into action animations of actions of the middle classification (interval 4-1, interval 4-2,...). Here, the control section 11 sequentially inputs each frame included in the action animation of the interval 4 into the boundary prediction model M3 and determines (predicts) the start time and the end time of the action animation of each action of the middle classification included in the action animation of the interval 4 on the basis of the output information from the boundary prediction model M3. Also in this case, the control section 11 can set or input the number of classifications (for example, 5) of the action animations of actions of the middle classification from the action animations of actions of the large classification into the boundary prediction model M3 and can perform editing processing on the boundaries of the action animations of actions of the middle classification after the classification. Figure 20D The interval comparison field R5 shows the display state after the classification of the action animations of actions of the middle classification (interval 4-1, interval 4-2,...). Note that, also in this case, the control section 11 can perform the same processing as in step S97 using the boundary prediction model M3 to classify the action animations of actions of the middle classification into action animations of actions of the small classification (not shown) in the case where any interval (for example, interval 4-1) in the interval comparison field R5 is selected. Figure 20D In the case where any interval (for example, interval 4-1) in the interval comparison field R5 is selected, the control section 11 performs the same processing as in step S97 using the boundary prediction model M3 to classify the action animation of the selected interval 4-1 into action animations of actions of the small classification (not shown). The user can perform such processing a plurality of times to segment the cyclic animation into action animations in stages and can perform the analysis processing using the action animations with the desired degree of detail. In addition, only the interval that the user wants to confirm can be segmented into action animations of more detailed actions, and thus, it is possible to suppress unnecessary segmentation processing.

[0143] The control section 11 displays the action animations of actions of the small classification (not shown) in the interval comparison field R5 in the case where the user selects the interval 4-1 in the interval comparison field R5. Figure 19C In the case where the user selects the interval 4-1 in the interval comparison field R5, the control section 11 displays the action animations of actions of the small classification (not shown) in the interval comparison field R5. Figure 20AAfter the comparison result is displayed, if the switching of the reference loop animation is instructed using the switching button C7, for example, the reference loop animation is changed to the selected other loop animation, and the contents of the comparison screen are updated. Also, if the switching of the loop animation that is the analysis target is instructed using the switching buttons C8 and C9, the control section 11 changes the analysis target to the selected other loop animation, and updates the contents of the comparison screen. Also, if the selection of each marker drawn in the graph Dl, the scatter diagram of the cycle time of each loop animation, or the scatter diagram of the required time of each motion animation (each section) displayed in the analysis information field R3 is accepted, the control section 11 displays the required time or the cycle time corresponding to the selected marker. Also, if the selection of each section shown in the cumulative graph D2 displayed in the analysis information field R3 is accepted, the control section 11 displays the required time of the selected section, and if the playback of the section is instructed, displays the motion animation of the section in the animation display field R4.

[0144] The control section 11 determines whether to end the above processing (S105). For example, if the end instruction of the above processing is accepted from the user via the input section 14, the control section 11 determines to end the processing. If the control section 11 determines not to end the above processing (S105: NO), it stands by while performing the processing corresponding to the accepted operation each time the operation on the comparison screen is accepted. If the control section 11 determines to end the above processing (S105: YES), it stores the comparison analysis result displayed in the comparison screen in the storage section 12 (S106), and ends the series of processing. For example, the control section 11 stores the animation selected in step S91, the start time and the end time of the loop animation extracted from each animation in step S94, the start time and the end time of the motion animation obtained by distinguishing each loop animation in step S97, and the like in one folder provided in the storage section 12. In this way, by reading out the folder in which the series of comparison analysis results are stored, the comparison analysis result can be confirmed at any time, and further, by further operating the displayed comparison screen, further comparison analysis processing can be performed. Note that not only the series of comparison analysis results can be stored in each folder, but also each loop animation extracted from the animation that is the processing target can be stored in each folder.

[0145] By the above processing, in the present embodiment, from the animation obtained by photographing the worker, a plurality of cycle animations are automatically extracted (segmented) based on the designated cycle animation, and each cycle animation is automatically distinguished in accordance with each action performed by the worker. In addition, among the plurality of cycle animations selected as analysis targets from the automatically drawn cycle animations, a reference cycle animation that should be a template is determined, and a comparison result of the reference cycle animation with other cycle animations is prompted. By synchronously playing in units of action animations when the reference cycle animation and the other cycle animations are displayed, comparison in units of action animations becomes easy.

[0146] As shown in Figure 20A , in the comparison screen, a collection button C11 for instructing display of a collection screen is provided, and the collection screen can read out the comparison analysis result in the folder generated by the above processing and stored in the storage section 12 by one click. In a case where the collection button C11 is operated, the control section 11 displays the collection screen R6 as shown in Figure 21A . The collection screen R6 is provided with a comparison tab for displaying a selection screen accepting selection of the comparison analysis result stored in the storage section 12, a video tab for displaying a selection screen accepting selection of the animation obtained by photographing the worker with the camera, and an all tab for displaying a selection screen accepting selection of any one of the comparison analysis result and the animation. In the example of Figure 21A , the all tab is selected, and the selection screens of the comparison analysis result and the animation are displayed. In the collection screen in the screen of Figure 21A , for example, in a case where "Comparison Analysis - Video 115 x Animation 1" is selected, the control section 11 reads out the corresponding comparison analysis result from the storage section 12, and displays the comparison screen as shown in Figure 19C and Figure 20A in the display section 15. By thus using the collection screen, the desired comparison analysis result or animation can be read out and displayed by one click.

[0147] Figure 19C , Figure 20A and Figure 21A , the animation display bar R4 in the screen is a configuration that displays only one cycle animation of an analysis target and can be switched with the switching buttons C8 and C9. In addition, as shown in Figure 21B , it can also be a configuration that the animation display bar R4 displays a plurality of cycle animations (animation group) that should be analysis targets. For example, it can be a configuration that the cycle animations of the analysis targets are displayed in the animation display bar R4 as shown in Figure 17the cycle animation selected in step S100 in the reference cycle animation and the cycle animation selected in step S100 in the analysis target cycle animation. In this case, a graph D1 showing the comparison result of the required time of each section can be displayed in the analysis information column R3 for all the cycle animations displayed in the animation display column R4, and the start time and the end time of each section of all the cycle animations can be displayed in the section comparison column R5.

[0148] (Embodiment 3)

[0149] In Embodiment 2, a configuration in which the required time of each action animation (each section) in the cycle animations to be compared (the reference cycle animation and the cycle animation of the analysis target) is compared and prompted is described. In the present embodiment, an information processing apparatus that prompts the cycle animation in which the required time of each section is longer than that of the reference cycle animation and the cycle animation in which the similarity in each section is lower than that of the reference cycle animation from among the cycle animations selected as the analysis target is described. The information processing apparatus 10 of the present embodiment has the same configuration as that of the information processing apparatus 10 of Embodiment 1 shown in Figure 2

[0150] Figure 22 is a flowchart showing an example of the comparison processing sequence of the cycle animation of Embodiment 3, Figure 23 is an explanatory diagram showing an example of a screen. Figure 22 The processing shown in Figure 17 is obtained by adding steps S111 to S114 between steps S102 and S103 in the processing shown in Figure 17 The same steps as those of Embodiment 1 are omitted from the description. In Figure 22 , the illustration of steps S91 to S101 in Figure 17 is omitted.

[0151] In the information processing apparatus 10 of the present embodiment, the control section 11 performs the same processing as that of the information processing apparatus 10 of Embodiment 1 shown in Figure 17 ​The same processing is performed as steps S91 to S102. Then, the control unit 11 extracts one loop animation selected as the analysis object in step S100 (S111), and calculates the difference in the required time of each interval (each motion animation) in the extracted loop animation and the reference loop animation (the difference of the excess part based on the reference loop animation) (S112). In addition, the control unit 11 calculates the similarity between the extracted loop animation and each interval (each motion animation) in the reference loop animation (S113). Here, similarity can also be calculated using correlation coefficient, cosine similarity, etc., or it can be constructed using a learning model that estimates the similarity between two motion animations. The control unit 11 determines whether the processing of steps S112 to S113 for all loop animations selected as the analysis object has ended (S114). If the determination is not finished (S114: NO), it returns to step S111; if the determination is finished (S114: YES), it moves to step S103. In step S103, the control unit 11 generates as follows: Figure 23 The comparison screen shown. Figure 23 The image has the same Figure 20A The same structure applies to the screen. In the cumulative chart D2, for the loop animation with the largest difference in required time for each interval calculated in step S112 (the difference from the baseline loop animation), a message indicating the largest required time for each interval is displayed. Figure 23 In the example, it is shown that intervals 3 and 9 in the loop animation of video 2 require the longest time. It should be noted that the message attached to the cumulative chart D2 is not limited to the configuration displayed when the difference from the baseline loop animation is greatest; it can also be the configuration displayed when the difference is greater than or equal to a predetermined value. Furthermore, in this embodiment, instead of... Figure 23 The comparison screen can also be composed of a cumulative chart D2 that includes a message displaying the loop animation with the lowest similarity (similarity to the baseline loop animation) among the intervals calculated in step S113. In this case, it can alert the user to loop animations where the worker's actions are dissimilar (deviation) from the baseline loop animation. For loop animations with intervals that require a long duration, and loop animations with intervals that have low similarity to the baseline loop animation, there is a high probability that the worker's work has some problems. By alerting the user to such loop animations and intervals, the user can verify whether the worker's work is appropriate.

[0152] Note that the control section 11 can calculate a score corresponding to the difference in the required time of each section (for example, the smaller the difference, the higher the score), and evaluate the loop animation by adding the scores of the sections as the score of the loop animation. In addition, the control section 11 can calculate a score corresponding to the similarity of each section (each motion animation) in the loop animation to be analyzed and the reference loop animation (for example, the greater the similarity, the higher the score), and evaluate the loop animation by adding the scores of the sections as the score of the loop animation.

[0153] (Embodiment 4)

[0154] In Embodiments 2 to 3, the reference loop animation is determined from the loop animation selected as the analysis target. In the present embodiment, an information processing apparatus that generates a reference loop animation from a loop animation extracted from a motion to be processed is described. The information processing apparatus 10 of the present embodiment has the same configuration as the information processing apparatus 10 of Embodiment 1 shown in Figure 2 , and thus the description of the configuration is omitted.

[0155] Figure 24 is a flowchart showing an example of the comparison processing sequence of the loop animation of Embodiment 4, Figure 25 is an explanatory diagram showing an example of the reference loop animation. Figure 24 The processing shown in Figure 17 is obtained by adding steps S121 to S128 between steps S99 and S100 in the processing shown in Figure 17 . The same steps as in Figure 24 are omitted. Figure 17 The illustration of steps S91 to S98 and S103 to S106 in

[0156] In the information processing apparatus 10 of the present embodiment, the control section 11 performs the same processing as steps S91 to S99 in Figure 17 . Thereby, the control section 11 displays a screen as shown in Figure 18C or Figure 18D . In the screen displayed here, a button (not shown) for accepting an instruction to generate a reference loop animation based on a loop animation extracted from a motion to be processed is provided, and the generation instruction of the reference loop animation is accepted by operating the button. The control section 11 determines whether the generation instruction of the reference loop animation is accepted (S121), and moves to step S100 in the case where it is determined that the generation instruction is not accepted (S121: NO). In this case, the control section 11 performs the same processing as Embodiment 2 shown in Figure 17 .

[0157] In a case where it is determined that the generation instruction of the reference cycle animation is received (S121: YES), the control section 11 extracts the motion animations of the same motion from the respective cycle animations distinguished as the motion animations in step S97 (S122). The control section 11 calculates the scores corresponding to the required times for the motion animations extracted from the respective cycle animations (S123). For example, the control section 11 assigns a higher score to each motion animation as the shorter the required time. Then, the control section 11 calculates the similarities of all combinations of the motion animations extracted from the respective cycle animations (S124), and calculates the scores corresponding to the calculated similarities (S125). For example, the control section 11 sequentially calculates the similarities with the other motion animations for one motion animation, assigns a higher score to the motion animation as the higher the similarity, and calculates the sum of the scores corresponding to the respective similarities as the score of the motion animation. The control section 11 calculates the scores corresponding to the similarities with the other motion animations by performing the same processing for all the motion animations.

[0158] The control section 11 determines the motion animations for the reference cycle animation on the basis of the scores calculated in step S123 and the scores calculated in step S125 for each motion (each section) (S126). For example, the control section 11 determines the motion animation for which the sum of the scores calculated in step S123 and the scores calculated in step S125 is the highest as the motion animation for the reference cycle animation. The control section 11 determines whether the processing of steps S123 to S126 is completed for the motion animations of all the motions (S127), and returns to step S122 to repeat the processing of steps S122 to S126 for the motion of which the processing is not completed in a case where it is determined that the processing is not completed (S127: NO). Note that the motion animation for the reference cycle animation is only the motion image extracted from the animation of the processing target, and can be a motion animation of a different worker.

[0159] In a case where it is determined that the processing of the motion animations of all the motions is completed (S127: YES), the control section 11 stitches the motion animations determined for the respective motions in step S126, and generates the reference cycle animation (S128). Thus, the reference cycle animation as shown in FIG. 17 is generated. Figure 25 Figure 25 ​In the example, the reference loop animation of the section 1 of the loop animation based on the video 20, the section 2 of the loop animation of the video 30,..., the section 9 of the video 40, and the section 10 of the video 60 are generated. Then, the control section 11 moves to step S100. Note that in step S102 in this case, the control section 11 determines the reference loop animation generated in step S128 as the reference loop animation that should be used as a template. Through the above processing, it is possible to generate the reference loop animation using the motion animation that is short in time required in any motion (section) and high in similarity to the motion animation in other loop animations. It is considered that the higher the similarity of the motion animation performed by the worker to other motion animations, the more standard the motion performed. Thus, it is possible to generate the most suitable reference loop animation based on the motion animation that is short in time required and in which a standard motion is performed. Note that in the above processing, the motion animation used for the reference loop animation is not limited to being determined based on the total of the scores corresponding to the time required and the scores corresponding to the similarity to the motion animation in other loop animations. For example, it is also possible to determine the motion animation that is the shortest in time required, or the motion animation that is the highest in similarity to other motion animations, or the like, as the motion animation used for the reference loop animation. In addition, it is also possible to perform determination processing of whether or not the motion animation extracted from each loop animation is a standard motion, for example, by the main factor classification model M5 used in Embodiment 1, and perform the processing of steps S123 to S126 on the motion animation determined to be a standard motion. In this case, it is possible to generate the reference loop animation based on the motion animation obtained by capturing a more suitable and more standard motion.

[0160] (Embodiment 5)

[0161] In Embodiments 2 to 4, the configuration in which the animation in which one worker is captured is used as a processing target is described. In the present embodiment, an information processing apparatus in which an animation in which a plurality of workers are captured is used as a processing target is described. The information processing apparatus 10 of the present embodiment has the same configuration as that of the information processing apparatus 10 of Embodiment 1 shown in FIG. 1, and thus, the description of the configuration is omitted. Figure 2

[0162] Figure 26 is a flowchart showing an example of a comparison processing sequence of the loop animation of Embodiment 5, Figure 27A Figure 27B is an explanatory diagram showing an example of a screen. Figure 26 The processing shown in FIG. 13 is obtained by adding steps S131 to S136 between steps S91 and S92 in the processing shown in FIG. 12. The description of the steps common to those of Figure 17 Figure 17 is omitted. In Figure 26 Figure 17 ​​​​The illustrations of steps S94 to S106 are shown.

[0163] In the information processing apparatus 10 of this embodiment, the control unit 11 receives data from, for example... Figure 18A The animation selection screen shown accepts the selection of any animation (S91). The control unit 11 performs object detection processing on the selected animation and detects the subject (object) in the animation (S131). The object detection processing can be performed using a learning model consisting of object detection algorithms such as CNN, SSD (SingleShot Multibox Detector), YOLO (You Only Look Once), or semantic segmentation algorithms such as SegNet, FCN (Fully Convolutional Network), and U-Net. The control unit 11 determines whether there are multiple detected objects (S132). If there are not multiple detected objects (S132: NO), that is, if there is only one detected object, the process moves to step S92. In this case, the control unit 11 performs the same... Figure 17 The same process is applied to Implementation Method 2 shown.

[0164] If multiple objects are detected (S132: YES), such as Figure 27A As shown, the control unit 11 displays a message prompting the user to select an analysis target from the objects in the animation (S133). (In progress) Figure 27A The animation displayed on the screen is configured such that an object can be selected through a predetermined operation (e.g., left-clicking the mouse), and the control unit 11 accepts the selection of an object through the predetermined operation on the animation (S134). It should be noted that the control unit 11 clearly indicates the detected object in the displayed animation using a bounding box, etc., and by accepting the selection of any bounding box, the selection of any object can be accepted. The control unit 11 determines the area of ​​the selected object in the animation (S135). Here, the control unit 11 can determine the area of ​​the analyzed object as an area of ​​a predetermined size centered on the point where the predetermined operation was performed, or it can determine the area of ​​the analyzed object as an area containing the point where the predetermined operation was performed within the area of ​​each object detected in step S131. Figure 27B In the example, a drop-down menu is displayed by performing a predetermined operation on the object on the right side of the animation. When "Select as object" is selected in the drop-down menu, the control unit 11 identifies the object as the analysis object.

[0165] In animations containing multiple subjects, each subject is filmed within the same area of ​​the animation. Figure 27AIn the example of FIG. 9, the subject photographed on the left side of the animation is always photographed on the left side, and the subject photographed on the right side is always photographed on the right side. Thus, in the present embodiment, the control section 11 extracts the region containing the analysis subject from each frame of the animation selected in step S91, and generates a partial animation containing only the analysis subject (S136). Then, the control section 11 executes the process after step S92 with the partial animation containing one analysis subject as the processing target. Thus, by the same process as the information processing apparatus 10 of Embodiments 2 to 4, it is possible to present the comparison result of the cyclic animation extracted from the partial animation. Note that, in the present embodiment, it is also possible to generate no partial animation containing the analysis subject from the animation of the processing target, and execute the process after step S92 based on the region of the analysis subject in the animation of the processing target.

[0166] In the above process, even in a case where a plurality of workers are included in the animation, it is possible to select an arbitrary one of the workers as the analysis subject, and by the same process as in the above Embodiments 2 to 4, it is possible to perform the comparison of the cyclic animation extracted from the animation in which the worker selected as the analysis subject is photographed. As in the above Embodiments 2 to 5, the animation of the processing target is not limited to the animation in which the worker performing the work is photographed, and can be an animation in which a robot configured to perform a predetermined work is photographed.

[0167] The matters described in the above embodiments can be combined with each other. In addition, the independent claims and the dependent claims described in the claims can be combined with each other in all combinations, regardless of the form of citation. Moreover, although the form of the claim in which the citation of two or more other claims is described is used in the claims (the form of multiple claims), it is not limited thereto. The form of the multiple claims in which the citation of at least one multiple claim is described (the form of multiple-citation multiple claim) can be used.

[0168] It should be understood that the embodiments disclosed this time are illustrative in all aspects and are not limited to the scope of the present application. The scope of the present application is shown by the scope of the claims rather than the meaning described above, and is intended to include all modifications within the meaning and scope equivalent to those of the claims.

[0169] Explanation of Reference Signs:

[0170] 10 information processing apparatus

[0171] 11 control section

[0172] 12 storage section

[0173] 13 communication section

[0174] 14 input section

[0175] 15 display section

[0176] M1 breed prediction model

[0177] M2 cycle prediction model

[0178] M3 boundary prediction model

[0179] M4 action analysis model

[0180] M5 main factor classification model

Claims

1. A storage medium, characterized by, The storage medium stores a program that causes a computer to execute the following processing: distinguishing an animation obtained by photographing an object performing a work, for each action included in the work; based on the distinguished animation, determining whether each action is a standard action; and based on the determination result for each action, analyzing whether the work is a standard work, wherein the animation is distinguished for each work element including a plurality of actions, the work element being one cycle of a work, the distinguished work element animation is distinguished for each action, a plurality of the work element animations are input to a learning model that learns to output information related to a main factor in which a work element performed by an object in the work element animation is not a standard work element, when the work element animation is input, based on the output information related to the main factor, the plurality of work element animations are grouped for each main factor.

2. A storage medium, characterized by The storage medium stores a program that causes a computer to execute the following processing: distinguishing an animation obtained by photographing an object performing a work, for each action included in the work; based on the distinguished animation, determining whether each action is a standard action; based on the determination result for each action, analyzing whether the work is a standard work; based on each distinguished animation obtained by distinguishing each animation of a plurality of animations for each action, measuring an action time of each action in each animation; and and generating a graph showing a deviation of the measured action time of each action in each animation, wherein the graph is a graph in which the action time of each action in each animation is associated with each action, in a case where a selection of a point plotted in the graph is received, outputting a distinguished animation of a corresponding action in an animation corresponding to the selected point.

3. The storage medium of claim 1 or 2, wherein, The program causes the computer to execute the following processing: inputting the animation to a learning model that learns to output information related to an action performed by an object in the distinguished animation included in the animation, when an animation obtained by photographing an object is input; and based on the output information related to the action, determining whether the action in the distinguished animation is a standard action.

4. The storage medium of claim 1 or 2, wherein, The program causes the computer to execute the following processing: based on whether an action time of each action in each distinguished animation is a standard action time, determining whether each action is a standard action.

5. The storage medium of claim 3, wherein, The program causes the computer to execute the following processing: based on the animation, determining a kind of a work object of a work performed by an object in the animation; selecting the learning model corresponding to the determined kind; and inputting the animation to the selected learning model, and outputting information related to an action performed by an object in the distinguished animation included in the animation.

6. The storage medium of claim 1 or 2, wherein, The program causes the computer to execute the following processing: inputting the animation into a learning model that learns to output information about actions performed by an object in the animation when an animation obtained by shooting an object performing a job is inputted, and outputting information about actions included in the job; and distinguishing the animation by each of the actions based on the outputted information about actions.

7. The storage medium of claim 1 or 2, wherein, The program causes the computer to execute the following processing: outputting a distinguished position at which the animation is distinguished by each of the actions; and receiving a change to each distinguished position.

8. The storage medium of claim 1 or 2, wherein, The program causes the computer to execute the following processing: receiving a setting of a distinguished position distinguished by each of the actions for any one of playback positions of the animation.

9. The storage medium of claim 1, wherein, The program causes the computer to execute the following processing: outputting an animation including the job element animation by each of the main factors.

10. The storage medium of claim 1 or 2, wherein, The program causes the computer to execute the following processing: synchronously outputting distinguished animations of each action in a plurality of animations based on distinguished animations obtained by distinguishing each of the animations by each of the actions.

11. The storage medium of claim 1, wherein, The program causes the computer to execute the following processing: generating a graph showing a job start timing and a job end timing for each of the job elements based on the job element animations obtained by distinguishing the animation.

12. A storage medium, characterized by The storage medium stores therein a program that causes a computer to execute the following processing: distinguishing an animation obtained by shooting an object performing a job by each of actions included in the job; determining whether each action is a standard action based on the distinguished animations respectively distinguished; analyzing whether the job is a standard job based on the determination results of the actions; distinguishing a plurality of the animations by each of the actions included in the job; and synchronously outputting distinguished animations of each action in a plurality of the animations based on the distinguished animations respectively distinguished, wherein a reference animation compared with other animations is selected from the plurality of the animations, synchronously outputting the selected reference animation and distinguished animations of each action in the other animations other than the reference animation, in the output of the distinguished animation of any one of the reference animation and the other animations, when a change instruction of the other animations is received, synchronously outputting the reference animation and the other animations instructed to be changed from the distinguished animations of the actions in the output.

13. A storage medium, characterized by The storage medium stores therein a program that causes a computer to execute the following processing: distinguishing an animation obtained by shooting an object performing a job by each of actions included in the job; determining whether each action is a standard action based on the distinguished animations respectively distinguished; analyzing whether the job is a standard job based on the determination results of the actions; distinguishing a plurality of the animations by each of the actions included in the job; and synchronously outputting distinguished animations of each action in a plurality of the animations based on the distinguished animations respectively distinguished, wherein a reference animation compared with other animations is selected from the plurality of the animations, synchronously outputting each action distinguishing animation of a selected reference animation and other animations than the reference animation, extracting, from each action distinguishing animation of a plurality of the other animations, an action distinguishing animation in which similarity is low between each action distinguishing animation of the reference animation and each action distinguishing animation of the other animations.

14. A storage medium, characterized by The storage medium stores a program that causes a computer to execute the following processing: distinguishing each action from a plurality of animations obtained by photographing an object performing a job, determining whether each action is a standard action based on the distinguished action, analyzing whether the job is a standard job based on the determination result of each action, distinguishing each action from a plurality of the animations, and synchronously outputting each action distinguishing animation of a plurality of the animations based on the distinguished action, wherein, from a plurality of the animations, a reference animation is selected for comparison with other animations, synchronously outputting each action distinguishing animation of a selected reference animation and other animations than the reference animation, extracting, from each action distinguishing animation of a plurality of the other animations, an action distinguishing animation in which a required time of an action related to the distinguishing animation is long.

15. A storage medium, characterized by The storage medium stores a program that causes a computer to execute the following processing: distinguishing each action from a plurality of animations obtained by photographing an object performing a job, determining whether each action is a standard action based on the distinguished action, analyzing whether the job is a standard job based on the determination result of each action, distinguishing each action from a plurality of the animations, synchronously outputting each action distinguishing animation of a plurality of the animations based on the distinguished action, from a plurality of the animations, a reference animation is selected for comparison with other animations, and synchronously outputting each action distinguishing animation of a selected reference animation and other animations than the reference animation, wherein, from each action distinguishing animation of a plurality of the animations, an action distinguishing animation that should be included in a reference animation is extracted for each action, splicing the extracted action distinguishing animation for each action to generate the reference animation.

16. The storage medium according to claim 1 or 2, wherein the animations include a plurality of cycle animations in which a time in which the object performs a series of the jobs is one cycle, the program causes the computer to execute the following processing: receiving designation of one cycle animation from the animations, extracting a cycle animation similar to the designated cycle animation from the animations, and distinguishing each cycle animation.

17. The storage medium according to claim 16, wherein a plurality of the animations obtained by photographing one or a plurality of the objects are stored in a storage section, the program causes the computer to execute the following processing: extracting a cycle animation similar to the designated cycle animation from the plurality of the animations stored in the storage section, and ​ The extracted respective cycle animations are distinguished by each of the actions.

18. The storage medium of claim 1 or 2, wherein, The program causes the computer to execute the following processing: In a case where the animation includes a plurality of objects, a selection of any one of the objects is received; and The animation is distinguished by each action performed by the selected object.

19. An information processing method characterized by comprising: The following processing is executed by a computer: An animation obtained by photographing an object performing a job is distinguished by each action included in the job; Based on the distinguished animations respectively, it is determined whether each action is a standard action; and Based on the determination results of the actions, it is analyzed whether the job is a standard job, wherein the animation is distinguished by each job element including a plurality of actions, the job element being a job of one cycle, the distinguished job element animations respectively are distinguished by each of the actions, a plurality of the job element animations are respectively input to a learning model that learns to output information related to a main factor in which a job element performed by an object in the job element animation is not a standard job, in a case where the job element animation is input, based on the output information related to the main factor, the plurality of job element animations are grouped by each of the main factors.

20. An information processing apparatus, comprising: The information processing apparatus has a control section, the control section is configured to, distinguish an animation obtained by photographing an object performing a job by each action included in the job, based on the distinguished animations respectively, determine whether each action is a standard action, based on the determination results of the actions, analyze whether the job is a standard job, distinguish the animation by each job element including a plurality of actions, the job element being a job of one cycle, distinguish the distinguished job element animations respectively by each of the actions, input a plurality of the job element animations respectively to a learning model that learns to output information related to a main factor in which a job element performed by an object in the job element animation is not a standard job, in a case where the job element animation is input, based on the output information related to the main factor, group the plurality of job element animations by each of the main factors.

21. An information processing method characterized by comprising: The following processing is executed by a computer: An animation obtained by photographing an object performing a job is distinguished by each action included in the job; Based on the distinguished animations respectively, it is determined whether each action is a standard action; Based on the determination results of the actions, it is analyzed whether the job is a standard job; A plurality of the animations are distinguished by each action included in the job; and Based on the distinguished animations respectively, the distinguished animations of each action in a plurality of the animations are output in synchronization, wherein, from a plurality of the animations, a reference animation compared with other animations is selected, the distinguished animations of each action in the selected reference animation and other animations other than the reference animation are output in synchronization, In output of the action-distinguished animation of either of the reference animation and the other animation, in a case where a change instruction of the other animation is received, the reference animation and the other animation instructed to change are synchronously output from the action-distinguished animation in the output.

22. An information processing apparatus comprising: The information processing apparatus has a control section, The control section is configured to, distinguish an animation obtained by photographing an object on which a work is performed, for each action included in the work, determine whether each action is a standard action based on the distinguished animation distinguished respectively, analyze whether the work is a standard work based on the determination result of each action, distinguish a plurality of the animations for each action included in the work, synchronously output the action-distinguished animation of each action in a plurality of the animations based on the distinguished animation distinguished respectively, select a reference animation to be compared with other animations from a plurality of the animations, synchronously output the selected reference animation and the action-distinguished animation of each action in the other animation other than the reference animation, In output of the action-distinguished animation of either of the reference animation and the other animation, in a case where a change instruction of the other animation is received, the reference animation and the other animation instructed to change are synchronously output from the action-distinguished animation in the output.

Images