MOTION ANALYSIS SYSTEM AND MOTION ANALYSIS PROGRAM

Abstract

Motion analysis system (100), including: a target image receiving unit (211) for receiving a plurality of target image data parts obtained by successively photographing a target person who is to perform a series of movements; a target pose detection unit (212) for detecting a pose of the target person shown in a target image from any of the multiple target image data parts and for generating target pose data representing the detected pose; a target trajectory extraction unit (213) to extract from a plurality of target pose data parts corresponding to the plurality of target image data parts, as target trajectory data representing a transition of the target person's pose, two or more target pose data parts, including target pose data at a target time and target pose data from a time that is an antecedent time continuing up to the target time and / or a subsequent time following the target time; and a movement specification unit (214) to obtain a movement identifier for identifying a movement of the target person based on the target trajectory data from a movement database (291), wherein the motion specification unit (214) extracts a motion identifier from motion trajectory data matching the target trajectory data from the motion database, on which a plurality of sets of a motion identifier for identifying a motion and motion trajectory data representing a transition of a pose of a person who has to perform a motion identified by the motion identifier are recorded; wherein in the motion database (291) a plurality of times, a plurality of motion pose data parts and a plurality of motion identifiers are linked together, Each motion trajectory data part is formed from two or more motion pose data parts. the target trajectory extraction unit (213) determines a target time group containing the target time and extracts two or more target pose data parts corresponding to two or more photograph times from which the target time group is formed, as target trajectory data, and the motion specification unit (214) determines a comparison time group containing a comparison time for each comparison time and extracts from the motion database (291) two or more motion pose data parts associated with two or more comparison times from which the determined comparison time group is formed, compares motion trajectory data formed by the two or more extracted motion pose data parts with the target trajectory data, selects the motion trajectory data that matches the target trajectory data based on a result of a comparison of the individual motion trajectory data parts and the target trajectory data, and extracts as a motion identifier that matches the target trajectory data a motion identifier associated with the comparison time corresponding to the selected motion trajectory data; the motion analysis system (100) further comprises: a motion identifier receiving unit (221) for multiple reception of a motion identifier group, which is a plurality of motion identifiers to which the plurality of times have been added; a motion image reception unit (222) for receiving multiple motion image data groups, which are a plurality of motion image data parts to which a plurality of times have been added and which are obtained by successively photographing an acting person who has to perform a series of movements; a unit (225) for capturing a temporary pose of the acting person shown in a motion picture from all of the majority of motion picture data parts in the motion picture data group and for generating temporary pose data representing the captured pose for each assumed motion picture data group; a unit (226) for generating a temporary table for generating a plurality of temporary tables in which a time group, a temporary pose data group corresponding to the motion image data group and the motion identifier group are linked together; a motion pose generation unit (227) for extracting temporary trajectory data, which are two or more temporary pose data parts associated with two or more motion identifiers to identify an identical motion, from each of the plurality of temporary tables, and for generating two or more motion pose data parts corresponding to two or more times, based on a plurality of temporary trajectory data parts extracted from the plurality of temporary tables, for each motion contained in the sequence of motions; and a motion registration unit (224) for mutually linking and registering the two or more times, the two or more motion pose data parts, the two or more motion identifiers for identifying the same motion in the motion database (291), for each motion contained in the series of motions.

Description

field of technology

[0001] The present invention relates to a technique for motion analysis. State of the art

[0002] In industrial fields, processes such as measuring cycle time and analyzing work content are mainly carried out manually.

[0003] Cycle time is the time a worker needs to assemble a product.

[0004] An analysis of job content is conducted to identify work that is not performed or is performed outside of a fixed work location. Non-stationary work refers to work that is not carried out at a fixed location.

[0005] Manual processing incurs high personnel costs. Furthermore, manual processing can only handle processing within a limited scope.

[0006] If it becomes easy to analyze the movements of working people, it is assumed that processing such cycle time measurements and analyzing work content will be simple.

[0007] Patent literature 1 discloses a motion analysis that is performed automatically. For a motion analysis, a camera and a three-dimensional sensor are attached to a person's head, and a certain degree of the person's movements are extracted using the camera and the three-dimensional sensor. List of citationsPatent literature Patent Literature 1: JP 2016-099982 A Patent literature 2: DE 11 2019 006 583 T5 (post-publication prior art pursuant to §3(2) PatG) Patent literature 3: EP 2 899 706 A1 Outline of the invention: Technical task

[0008] In patent literature 1, a camera and a three-dimensional sensor are attached to a person's head for automatic motion analysis.

[0009] In patent literature 2, image data of a target person are acquired and skeletal framework information is extracted, representing a body posture of the target person, such as the coordinates of a multitude of joints, from the acquired image data. A type of movement is identified.

[0010] Patent reference 3 discloses a method for behavior analysis by an intelligent system, wherein the system captures a series of images of a monitored area and generates a point set of a moving silhouette for each image, so that the movements can be tracked based on the trajectory of the points. Behavior can then be identified from the trajectory by comparison with a database.

[0011] When a method from patent literature 1 is applied to analyze the movements of a worker, an object that is not necessary for the worker's work is attached to the worker's body. Furthermore, the object attached to the worker's body can hinder the worker's work.

[0012] The present invention aims to enable an analysis of the movements of a working person without hindering work. Solution to the task

[0013] To solve the problem, a motion analysis system according to claim 1 and a motion analysis program according to claim 4 are proposed.

[0014] Further training and advantageous designs result from the sub-claims. Advantageous effects of the invention

[0015] According to the present invention, it is possible to specify the movements of a working person without attaching an object unnecessary for the work, such as a camera and a three-dimensional sensor, etc., to the body of the working person (as an example of a target person). That is, it is possible to analyze the movements of a working person without hindering their work. Brief description of the drawings Fig. Figure 1 is a configuration sketch of a motion analysis system 100 according to a first embodiment; Fig. Figure 2 is a configuration sketch of a motion analysis device 200 according to the first embodiment; Fig. Figure 3 is a flowchart of a motion analysis procedure according to the first embodiment; Fig. Figure 4 is an explanatory diagram of a target pose detection process (S120) according to the first embodiment; Fig. Figure 5 is a flowchart of a target trajectory extraction process (S130) according to the first embodiment; Fig. Figure 6 is an explanatory diagram of the target trajectory extraction process (S130) according to the first embodiment; Fig. Figure 7 is a configuration sketch of a motion database 291 according to the first embodiment; Fig. 8 is a flowchart of a motion specification process (S140) according to the first embodiment; Fig. 9 is a flowchart of a comparison process (S144) according to the first embodiment; Fig. Figure 10 is an explanatory diagram concerning the use of dynamic programming according to the first embodiment; Fig. Figure 11 is a configuration sketch of the motion analysis system 100 according to the first embodiment; Fig. Figure 12 is a configuration diagram of a registration unit 220 according to the first embodiment; Fig. 13 is a flowchart of a registration process according to the first embodiment; Fig. 14 is an explanatory diagram of the motion trajectory detection process (S103) according to the first embodiment; Fig. Figure 15 is a configuration diagram of a registration unit 220 according to a second embodiment; Fig. 16 is a flowchart of a registration process according to the second embodiment; Fig. 17 is a flowchart of a motion pose generation process (S250) according to the second embodiment; Fig. Figure 18 is a sketch that presents a concrete example of sampling or scanning according to the second embodiment; Fig. Figure 19 is a configuration sketch of a motion analysis device 200 according to a third embodiment; Fig. 20 is a configuration diagram of a learning unit 230 according to the third embodiment; Fig. 21 is a flowchart of a motion analysis procedure according to the third embodiment; Fig. Figure 22 is a flowchart of a learning process according to the third embodiment; and Fig. Figure 23 is a configuration sketch of the hardware of the motion analysis device 200 according to the embodiments. Description of embodiments

[0016] In the embodiments and drawings, identical or corresponding elements are designated by the same reference numbers. Descriptions for elements with the same reference numbers as elements already explained are omitted or simplified where appropriate. The arrows in the drawings primarily illustrate data flows or processing sequences. First embodiment

[0017] A motion analysis system 100 for automatically analyzing a person's movements is based on... Fig. 1 to Fig. 14 described. ***Explanation of the configuration***

[0018] Based on Fig. Section 1 describes a configuration of the motion analysis system 100.

[0019] The motion analysis system 100 includes a camera 101 and a motion analysis device 200.

[0020] Camera 101 photographs a target person 111. Camera 101 can photograph the target person 111 from the front or from a diagonal angle. Furthermore, camera 101 can photograph the entire body of the target person 111 or a part of it (for example, the upper half of the body).

[0021] The motion analysis device 200 analyzes the movements of the target person 111.

[0022] Target person 111 is a person who is to be the subject of a movement analysis. A concrete example of target person 111 is a person who has a job to do (a working person).

[0023] Based on Fig. Section 2 describes the configuration of the motion analysis device 200.

[0024] The motion analysis device 200 is a computer comprising hardware components such as a processor 201, a main memory 202, an auxiliary storage device 203, a communication device 204, and an input and output interface 205. These hardware components are interconnected via signal lines.

[0025] The Processor 201 is an integrated circuit (IC) that performs arithmetic processing, thereby controlling other hardware components. For example, the Processor 201 could be a CPU, a DSP, or a GPU.

[0026] IC is the abbreviation for "Integrated Circuit".

[0027] CPU is the abbreviation for "Central Processing Unit".

[0028] DSP is the abbreviation for "Digital Signal Processor".

[0029] GPU is the abbreviation for "Graphics Processing Unit", i.e., graphics processor.

[0030] The main memory (RAM) is a volatile or non-volatile storage device. It is also referred to as a main memory device or main storage. For example, RAM is a memory device. Data stored in RAM is transferred to auxiliary storage device 203 as needed.

[0031] RAM is the abbreviation for "Random Access Memory".

[0032] The auxiliary storage device 203 is a non-volatile storage device. For example, the auxiliary storage device 203 is a ROM, an HDD, or flash memory. Data stored in the auxiliary storage device 203 is loaded into the main memory 202 as needed.

[0033] ROM is the abbreviation for "Read Only Memory".

[0034] HDD is the abbreviation for "Hard Disk Drive".

[0035] The communication device 204 is a receiver and a transmitter. For example, the communication device 204 is a communication chip or a NIC.

[0036] NIC is the abbreviation for "Network Interface Card".

[0037] The input / output interface 205 is a port to which an input device and an output device are connected. For example, the input / output interface 205 is a USB port, the input device is a keyboard or a mouse, and the output device is a display.

[0038] USB stands for "Universal Serial Bus".

[0039] The motion analysis device 200 comprises elements such as a specification unit 210 and a registration unit 220. The specification unit 210 comprises elements such as a target image acquisition unit 211, a target pose detection unit 212, a target trajectory extraction unit 213, a motion specification unit 214, and a result output unit 215. These elements are implemented by software.

[0040] The auxiliary storage device 203 stores a motion analysis program that causes a computer to function as the specification unit 210 and the registration unit 220. The motion analysis program is loaded into the main memory 202 and executed by the processor 201.

[0041] Furthermore, the auxiliary storage device 203 stores an operating system. At least part of the operating system is loaded into the main memory 202 and executed by the processor 201.

[0042] The 201 processor runs the motion analysis program while running the OS.

[0043] OS is the abbreviation for "Operating System".

[0044] The input and output data of the motion analysis program are stored in a storage unit 290.

[0045] The main memory 202 functions as the memory unit 290. However, memory devices such as the auxiliary memory device 203, a register in the processor 201, and a cache memory in the processor 201, etc., can function as the memory unit 290 instead of the main memory 202 or together with the main memory 202.

[0046] The storage unit 290 stores a movement database 291, etc.

[0047] The motion analysis device 200 can include multiple processors that replace the processor 201. These multiple processors share the functions of the processor 201.

[0048] The motion analysis program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or flash memory, etc. ***Explanation of the operation***

[0049] One operating sequence of the motion analysis system 100 corresponds to one motion analysis procedure. Furthermore, one operating sequence of the motion analysis device 200 corresponds to one processing sequence performed by a motion analysis program.

[0050] A motion analysis method is based on Fig. 3 discussed.

[0051] In step S110, the target image receiving unit 211 receives a plurality of target image data parts.

[0052] The majority of target image data is obtained by successively photographing the target person 111, who has to perform a series of movements.

[0053] Each target image data segment represents a target image. Target person 111 is shown in the target image.

[0054] For example, the majority of target image data segments are entered into the motion analysis device 200 by a user. The target image reception unit 211 receives the majority of the entered target image data segments.

[0055] For example, the majority of target image data parts are sent by camera 101 and received by motion analysis device 200. Target image receiving unit 211 accepts the received majority of target image data parts.

[0056] In step S120, the target pose detection unit 212 detects a pose of the target person 111, which is shown in the target image by each of the target image data parts, and generates target pose data that express the detected pose.

[0057] In this way, it is possible to obtain the majority of target pose data parts that correspond to the majority of target image data parts.

[0058] A target pose detection process (S120) is based on Fig. 4 described. [m-1], [m], and [m+1] indicate three consecutive photography times. The photography time is the period during which a photograph is taken. Target image data 121 at the respective photography times represent target images at the respective photography times. The target person 111 is shown in the target images at the respective photography times.

[0059] The target pose detection unit 212 detects a pose of the target person 111 at all photographic times by performing an image process on the target image at all photographic times.

[0060] More precisely, the target pose detection unit 212 captures a skeletal frame of the target person 111. Based on the skeletal frame of the target person 111, a pose of the target person 111 is specified.

[0061] More precisely, the target pose detection unit 212 determines the positions of the individual joints of the target person 111. The position of each joint can be an absolute position in a specific coordinate system or a relative position with respect to other joints. The position of each joint is identified by a coordinate value. The skeletal frame is specified by a plurality of joints, such as a neck joint and a shoulder joint.

[0062] The target pose detection unit 212 generates target pose data 122, which represents a pose of the target person 111 at the respective time of photography.

[0063] In the target pose data 122 at the respective photography time, each black dot represents a position of a joint of the target person 111.

[0064] It will be revisited Fig. 3 Reference is made, and the descriptions continue from step S130.

[0065] Steps S130 and S140 are performed for each photograph within a photographic time slot. A photographic time slot is a time slot in which a photograph is taken.

[0066] In step S130, the target trajectory extraction unit 213 extracts target trajectory data from a plurality of target pose data parts, which correspond to the plurality of target image data parts.

[0067] The target trajectory data is formed from two or more target pose data segments that represent a transition of the pose of target person 111. Within the target trajectory data, these two or more target pose data segments are ordered in a time series manner. However, the target trajectory data can also be a polynomial function represented by approximating two or more target pose data segments.

[0068] The following is an additional explanation of the target trajectory data. Trajectory data consists of data from successive times. The target trajectory data consists of target pose data from successive times. The successive times are the target time and the times that continue up to and follow the target time. If the target time is [t], the successive times are [t-1], [t], and [t+1].

[0069] The target trajectory extraction process (S130) is based on Fig. 5 described.

[0070] In step S131, the target trajectory extraction unit 213 selects a previously unselected photography time from a photography time slot.

[0071] The photography time slot is a time slot in which a photograph is taken.

[0072] The selected photography time is referred to as the "target time".

[0073] In step S132, the target trajectory extraction unit 213 determines a photograph time group for the target time.

[0074] A photography time group consists of two or more consecutive photography times, including the target time. It is possible to predetermine the number of photography times included in the photography time group.

[0075] The target time is denoted by "m". For example, the target trajectory extraction unit 213 determines a photography time group consisting of a photography time (m-1), a photography time (m) and a photography time (m+1) as the photography time group for the target time.

[0076] The specific photography time group is referred to as the "target time group".

[0077] In step S133, the target trajectory extraction unit 213 extracts target trajectory data corresponding to the target time from the majority of target pose data parts.

[0078] For example, if the target time group consists of time (m-1), time (m), and time (m+1), the target trajectory extraction unit 213 extracts target pose data at time (m-1), target pose data at time (m), and target pose data at time (m+1). Three extracted target pose data fragments form target trajectory data corresponding to the target time.

[0079] The target trajectory extraction process (S130) is based on Fig. 6 described. [m-1], [m] and [m+1] denote three consecutive photography times.

[0080] It is assumed that a target time group of [m-1], [m] and [m+1] is formed.

[0081] The target trajectory extraction unit 213 extracts the target pose data 122 at [m-1], the target pose data 122 at [m] and the target pose data 122 at [m+1].

[0082] Three extracted target pose data 122 form target trajectory data 123 at [m].

[0083] It will be revisited Fig. 3 Reference is made, and the descriptions continue for step S140 and thereafter.

[0084] In step S140, the motion specification unit 214 receives a motion identifier to identify a movement of the target person 111 based on the target trajectory data.

[0085] More precisely, the motion specification unit 214 extracts a motion identifier from motion trajectory data that matches the target trajectory data from the motion database 291.

[0086] The motion database 291 is a database in which one or more sets consisting of a motion identifier and a motion trajectory data part are registered.

[0087] The motion identifier identifies a movement.

[0088] The motion trajectory data represents a transition of positions of a person who has to perform movements that can be identified by motion identifiers.

[0089] Now the configuration of the movement database 291 is based on Fig. 7 described. [t-1], [t] and [t+1] denote three consecutive photography times.

[0090] In the movement database 291, a plurality of times, a plurality of movement pose data parts and a plurality of movement identifiers are linked together.

[0091] The motion trajectory data 133 are formed from two or more motion pose data parts 132. More precisely, the motion trajectory data 133 are formed from two or more motion pose data parts 132 that are linked to two or more consecutive times.

[0092] The movement pose data 132 represent poses of a person who has to perform movements.

[0093] The motion trajectory data 133 at [t] are formed from motion pose data 132 at [t-1], motion pose data 132 at [t], and motion pose data 132 at [t+1]. The motion trajectory data 133 include data from successive times before and after [t].

[0094] The motion trajectory data 133 at [t] represent a transition of poses of a person who has to perform movements that are identified by a motion identifier “work A” at time t.

[0095] The motion specification process (S140) is based on Fig. 8 described.

[0096] In step S141, the motion specification unit 214 selects an unselected time from "Time" in the motion database 291. More precisely, the motion specification unit 214 selects times in ascending order.

[0097] The selected time is referred to as the "reference time".

[0098] In step S142, the motion specification unit 214 determines a comparison time group with respect to the comparison time.

[0099] The comparison time group consists of two or more consecutive times, including the comparison time itself. It is possible to predetermine the number of times included in the comparison time group.

[0100] The reference time is denoted by "t". For example, the motion specification unit 214 defines a time group formed by time (t-1), time (t) and time (t+1) as the reference time group for the reference time.

[0101] In step S143, the motion specification unit 214 extracts motion trajectory data corresponding to the comparison time from the motion database 291.

[0102] For example, when the comparison time group is formed from time (t-1), time (t), and time (t+1), the motion specification unit 214 extracts motion pose data at time (t-1), the motion pose data at time (t), and the motion pose data at time (t+1). These three extracted motion pose data sets are formed from the motion trajectory data corresponding to the comparison time.

[0103] In step S144, the motion specification unit 214 compares the motion trajectory data with the target trajectory data. The target trajectory data is extracted in step S130 (see Fig. 3).

[0104] More precisely, the motion specification unit 214 calculates a distance between the motion trajectory data and the target trajectory data.

[0105] The distance between the motion trajectory data and the target trajectory data corresponds to the magnitude of the difference between a transition of poses represented by the target trajectory data and a transition of poses represented by the motion trajectory data. A comparison process (S144) is described based on Fig. 9.

[0106] Target pose data contained in the target trajectory data are denoted by I(m) or I(m+k).

[0107] Motion pose data contained in the motion trajectory data are denoted by R(t) or R(t+k).

[0108] "k" is an integer equal to or greater than -i, but equal to or less than +i. "i" is an integer equal to or greater than 1. If the number of target pose data parts contained in the target trajectory data is "3", then "i" equals "1".

[0109] In step S1441, the motion specification unit 214 calculates a distance between target pose data I(m+k) and target pose data I(m). The calculated distance is denoted by I'(m+k).

[0110] The distance I'(m+k) corresponds to the size of the difference between the pose represented by the target pose data I(m) and the pose represented by the target pose data I(m+k).

[0111] It is possible to represent the distance I'(m+k) by an expression (1). l'(m+k)=I(m+k)−I(m)

[0112] More precisely, the motion specification unit 214 calculates the distance I'(m+k) in the following way.

[0113] The target pose data specifies the respective positions of a plurality of joints.

[0114] The motion specification unit 214 calculates a distance for each joint between a joint position in the target pose data I(m) and a joint position in the target pose data I(m+k). It may be necessary for the motion specification unit 214 to weight each joint and calculate the distance for each joint according to that weight. In this case, the distance for a significant joint will be a large value.

[0115] Otherwise, the motion specification unit 214 calculates a difference between the joint spacing in the target pose data I(m) and the joint spacing in the target pose data I(m+k) as the distance relative to a set of joints. It may be indicated that the motion specification unit 214 weights each set of joints and calculates the distance relative to each set according to the weight of each set of joints. In this case, the distance relative to a significant set of joints will be a large value.

[0116] The motion specification unit 214 calculates the sum of the calculated distances. The calculated sum becomes a distance I'(m+k).

[0117] In step S1442, the motion specification unit 214 calculates a distance between motion pose data R(t+k) and motion pose data R(t). The calculated distance is denoted by R'(t+k).

[0118] The distance R'(t+k) corresponds to the size of the difference between the pose represented by the motion pose data R(t) and the pose represented by the motion pose data R(t+k).

[0119] It is possible to represent the distance R'(t+k) by an expression (2). R'(t+k)=R(t+k)−R(t)

[0120] More precisely, the motion specification unit 214 calculates the distance R'(t+k) in the following way.

[0121] The movement pose data specifies the respective positions of a plurality of joints.

[0122] The motion specification unit 214 calculates, for each joint, the distance between a joint position in the motion pose data R(t) and a joint position in the motion pose data R(t+k). It may be indicated that the motion specification unit 214 weights each joint and calculates the distance for each joint according to that weight. In this case, the distance for a significant joint will be a large value.

[0123] Otherwise, the motion specification unit 214 calculates the difference between a joint distance in the motion pose data R(m) and a joint distance in the motion pose data I(m+k) as a distance with respect to a set of joints. It may be appropriate for the motion specification unit 214 to weight each set of joints and calculate a distance with respect to each set of joints according to the weight of each set of joints. In this case, a distance with respect to a significant set of joints will be a large value.

[0124] The motion specification unit 214 calculates the sum of the calculated distances. The calculated sum becomes the distance R'(t+k).

[0125] In step S1443, the motion specification unit 214 calculates the sum of the differences between the distance I'(m+k) and the distance R'(t+k).

[0126] The calculated sum represents a distance between motion trajectory data and target trajectory data.

[0127] It is possible to express the distance between motion trajectory data and target trajectory data using expression (3). [Formula 1] ∑k=−il|I'(m+k)−R'(t+k)|

[0128] It may be advisable to calculate the distance between the motion trajectory data and the target trajectory data using dynamic programming or a hidden Markov model, etc. These methods calculate the smallest possible distance by dynamically lengthening or shortening time series data.

[0129] In these methods, for example, if the distance calculated by I'(m+k)-R' (m+(k+a)) is smaller than the distance calculated by I'(m+k)-R' (t+k), the distance between "k" and "k+a" is added.

[0130] The use of dynamic programming is based on Fig. 10 described.

[0131] The numerical values ​​in the table represent values ​​of |I'(m+a)-R'(t+b)|, with the proviso that -2≤a and b≤2. A solid arrow indicates a path (a correspondence between I' and R').

[0132] In dynamic programming, the distances between the individual grids are calculated by changing "a" and "b". Then, the path (see the solid arrow) is selected that minimizes the total sum of the distances.

[0133] It will be revisited Fig. 8 Reference is made, and the description continues with step S145.

[0134] In step S145, the motion specification unit 214 decides whether there is a comparison time that has not yet been selected.

[0135] If the comparison time that has not yet been selected exists, processing continues to step S141.

[0136] If the comparison time that has not yet been selected does not exist, processing continues to step S146.

[0137] In step S146, the motion specification unit 214 selects motion trajectory data that matches target trajectory data based on the result of a comparison between the individual motion trajectory data and the target trajectory data.

[0138] More precisely, the motion specification unit 214 selects motion trajectory data that match target trajectory data based on the distance of the individual motion trajectory data in relation to the target trajectory data.

[0139] The motion trajectory data that match the target trajectory data are determined in the following way.

[0140] The motion specification unit 214 selects the smallest distance from those calculated in step S144. Then, the motion specification unit 214 selects motion trajectory data that corresponds to the selected distance. The selected motion trajectory data is motion trajectory data that matches the target trajectory data.

[0141] However, if the selected distance (the smallest distance) is greater than a threshold, the motion specification unit 214 has the possibility of determining that the motion trajectory data matching the target trajectory data do not exist.

[0142] In step S147, the motion specification unit 214 extracts a motion identifier from the motion trajectory data matching the target trajectory data from the motion database 291.

[0143] More precisely, the motion specification unit 214 selects a comparison time that corresponds to the motion trajectory data selected in step S146 and extracts the motion identifier associated with the selected comparison time. The extracted motion identifier is a motion identifier of the motion trajectory data that matches the target trajectory data.

[0144] For example, in step S146 it is assumed that the motion trajectory data 133 of [t] (see Fig. 7) can be selected. In this case, the motion specification unit 214 extracts the motion identifier “Work A”, which is linked to [t], from the motion database 291.

[0145] However, if it is determined that there is no motion trajectory data matching the target trajectory data (see step S146), no motion identifier is extracted. If the motion identifier is not extracted, the motion specification unit 214 provides an identifier indicating work (non-stationary work) that is not generally performed.

[0146] It will be revisited Fig. 3 Reference is made to describe the processing after step S130 and step S140.

[0147] By performing step S130 and step S140 for each photographic time in a photographic time slot, a motion identifier is obtained with respect to the individual photographic times.

[0148] After steps S130 and S140 have been performed for each photographic time in the photographic time slot, processing continues to step S150.

[0149] In step S150, the result output unit 215 outputs result data.

[0150] For example, the results output unit 215 displays results data on a screen.

[0151] The results data includes a motion identifier for each photography time. ***Supplement to preprocessing***

[0152] A procedure for generating the motion database 291 is described.

[0153] Fig. Figure 11 represents a configuration of the motion analysis system 100 at the time of the creation of the motion database 291.

[0154] The motion analysis system 100 includes a camera 102 and a motion analysis device 200.

[0155] Camera 102 photographs a person 112. Camera 102 can be the same as camera 101 (see Fig. 1) or it can be a different camera than camera 101.

[0156] The acting person 112 is a person who performs actions (a target movement) that are a target for registration in the movement database 291. The acting person 112 can be the same person as the target person 111 (see Fig. 1) be or can be another person different from the target person 111.

[0157] If the person acting is a worker, they will perform tasks that are generally carried out (stationary tasks) as a series of movements. However, the person acting can also perform a series of movements that include tasks that are not generally carried out (non-stationary tasks).

[0158] The configuration of the recording unit 220, which is provided in the motion analysis device 200, is described in Fig. 12 described.

[0159] The registration unit 220 includes elements such as a motion detection receiving unit 221, a motion image receiving unit 222, a motion pose detection unit 223 and a motion registration unit 224.

[0160] A registration process will now be initiated by registration unit 220 based on... Fig. 13 described.

[0161] The registration process will add the movement database 291 (see Fig. 7) generated.

[0162] In step S101, the motion identifier acceptance unit 221 receives a plurality of motion identifiers to which a plurality of times have been added.

[0163] For example, the motion identification acceptance unit 221 receives a motion identification list. The motion identification list receives multiple records from a time and a motion identifier.

[0164] For example, a user enters the motion identification list into the motion analysis device 200. The motion identification acceptance unit 221 receives the motion identification list.

[0165] For movements that are not included in stationary work, identifiers are added to indicate non-stationarity.

[0166] In step S102, the motion image reception unit 222 receives a plurality of motion image data parts to which a plurality of times have been added. The plurality of times added to the plurality of motion image data parts coincide with a plurality of times added to the plurality of motion identifiers.

[0167] The majority of target image data is obtained by successively photographing the person in question (112) performing a series of movements.

[0168] Each motion image data segment represents a motion image. The acting person 112 is shown in the motion image.

[0169] For example, the motion image reception unit 222 receives a motion image file. The motion image file contains multiple sets of time and motion image data.

[0170] For example, the motion image file is entered by users into the motion analysis device 200. The motion image reception unit 222 receives the entered motion image file.

[0171] In step S103, the motion pose detection unit 223 captures a pose of the acting person 112, which is shown in the motion images of all motion image data, and generates motion pose data that express the captured pose.

[0172] In this way, the majority of motion pose data parts are obtained, which correspond to the majority of motion image data parts.

[0173] Based on Fig. Section 14 discusses a motion pose detection process (S103). [t-1], [t] and [t+1] indicate three consecutive times.

[0174] The motion image data 131 at a given time represents a motion image at that particular time. The acting person 112 is shown in the motion image at that particular time.

[0175] The motion pose detection unit 223 detects the pose of the acting person 112 at the respective time by performing an image process on the motion image at the respective time.

[0176] More precisely, the motion pose detection unit 223 captures a skeletal frame of the acting person 112. The pose of the acting person 112 is specified by the skeletal frame of the acting person 112.

[0177] More precisely, the motion pose detection unit 223 recognizes the position of each joint of the acting person 112. It is possible to specify the positions of individual joints using coordinate values ​​or relative positions with respect to other joints. It is possible to specify the skeletal frame using multiple joints, such as a neck joint and a shoulder joint, etc.

[0178] The motion pose detection unit 223 generates motion pose data 132, which represents the pose of the acting person 112 at the respective time.

[0179] In the movement pose data 132 at the respective times, each black dot represents a joint of the acting person 112.

[0180] It will be revisited Fig. 13 Referenced to describe step S104.

[0181] In step S104, the motion registration unit 224 registers a plurality of times, a plurality of motion pose data parts, a plurality of motion identifiers in the motion database 291 by linking them together. ***Explanation of an example***

[0182] The motion specification unit 214 can specify a time slot in which stationary work is performed and a time slot in which non-stationary work is performed.

[0183] More precisely, the motion specification unit 214 measures a time slot between times to which identifiers are appended indicating non-stationarity. The measured time slot is a time slot in which the non-stationary work is performed. The motion specification unit 214 then detects a time slot that is not the time slot in which the non-stationary work is performed. The detected time slot is a time slot in which stationary work is performed. ***Effect of the first embodiment***

[0184] According to the first embodiment, it becomes possible to specify the movements of a person without having to attach objects such as a camera and a three-dimensional sensor, etc., which are not necessary for the work, to the person's body (an example of a target person). That is, it is possible to analyze the person's movements without hindering their work.

[0185] Furthermore, it becomes possible to specify time slots in which the individual movements are carried out by the target person 111. That is, it is possible to specify a time slot in which stationary work is carried out and a time slot in which non-stationary work is carried out.

[0186] Furthermore, by comparing trajectory data (successive time data) of target trajectory data and motion trajectory data, the following effects can be achieved. The work can encompass multiple movements. By comparing not only movements at a single time, but also trajectory data that represent successive movements, it is possible to determine a more suitable work. Second embodiment

[0187] Based on Fig. 15 to Fig. Section 18 mainly describes points that differ from those of the first embodiment with regard to an embodiment for improving the accuracy of a motion analysis. ***Explanation of the configuration***

[0188] The configuration of the motion analysis system 100 is the same as the configuration in the first embodiment (see Fig. 1 and Fig. 11).

[0189] However, in Fig. 11 the acting person 112 performs a series of movements several times, and the camera 102 photographs the series of movements by the acting person 112 several times.

[0190] The configuration of the motion analysis device 200 is the same as in the first embodiment (see Fig. 2) same.

[0191] However, part of the configuration in the registration unit 220 differs from the configuration in the first embodiment (see Fig. 12) different.

[0192] Now the configuration of the registration unit 220 is based on Fig. 15 described.

[0193] The registration unit 220 includes elements such as the motion identification acceptance unit 221, the motion image acceptance unit 222, a unit 225 for the temporary recording of poses, a unit 226 for the temporary generation of tables, a motion pose generation unit 227 and the motion registration unit 224. ***Explanation of the operation***

[0194] A registration process will now be initiated by registration unit 220 based on... Fig. 16 described.

[0195] The movement database 291 (see Fig. 7) is generated by the registration process.

[0196] Steps S210 and S240 are performed multiple times for each of the movements in the sequence. That is, if the sequence of movements is performed three times, steps S210 through S240 are performed three times.

[0197] In step S210, the motion identification acceptance unit 221 receives a motion identification group.

[0198] The motion identifier group is a plurality of motion identifiers to which a plurality of times have been added.

[0199] Step S210 corresponds to step S101 in the first embodiment (see Fig. 13).

[0200] In step S220, the motion image reception unit 222 receives a motion image data group.

[0201] The motion image data group is a plurality of motion image data pieces to which a plurality of times have been added and which were obtained by photographing an acting person performing a series of movements.

[0202] Step S220 corresponds to step S202 in the first embodiment (see Fig. 13).

[0203] In step S230, unit 225 temporarily captures poses of the acting person 12, which are shown in the motion images of the individual motion image data in the motion image data group, and generates temporary pose data that express the captured poses.

[0204] In this way, a temporary pose data group corresponding to the motion image data group is obtained. The temporary pose data group is a plurality of temporary pose data parts, which correspond to a plurality of motion image data parts.

[0205] A method for generating temporary pose data is the same as the method for generating motion pose data in the first embodiment (see step S103 in Fig. 13).

[0206] In step S240, unit 226 creates a temporary table.

[0207] The temporary table is a table in which a time group, a temporary pose data group, and a movement identifier group are linked together.

[0208] The time group is a plurality of times that are attached to a plurality of motion identifiers and a plurality of motion images.

[0209] By performing steps S210 and S240 for each of the series of movements for a plurality of times, a plurality of temporary tables are generated.

[0210] After steps S210 and S240 have been performed for each of the series of movements for a plurality of times, the process continues to step S250.

[0211] In step S250, the motion pose generation unit 227 generates two or more motion pose data parts, corresponding to two or more times for each motion contained in a series of motions.

[0212] Two or more motion pose data pieces corresponding to two or more times are generated in the following manner.

[0213] The motion pose generation unit 227 extracts trajectory data from all of a plurality of temporary tables, which are two or more temporary pose data parts that are linked to two or more motion identifiers that identify the same motion.

[0214] The motion pose generation unit 227 generates two or more motion pose data pieces corresponding to two or more times, based on a plurality of temporary trajectory data extracted from a plurality of temporary tables.

[0215] A motion pose generation process (S250) is based on Fig. 17 described.

[0216] Steps S251 and S253 are performed for each movement included in the sequence of movements.

[0217] In step S251, the motion pose generation unit 227 extracts temporary trajectory data from each of a plurality of temporary tables. In this way, a plurality of temporary trajectory data fragments are extracted.

[0218] In step S252, the motion pose generation unit 227 determines the number of motion pose data parts from which the motion trajectory data is formed. This determined number is referred to as the "number of configuration data".

[0219] For example, the number of configuration data points is an average of the number of temporary pose data points in a plurality of temporary trajectory data points. Let's assume that the number of temporary pose data points forming temporary trajectory data for a first time is "3", the number of temporary pose data points forming temporary trajectory data for a second time is "4", and the number of temporary pose data points forming temporary trajectory data for a third time is "5". In this case, the number of configuration data points is "4".

[0220] In step S253, the motion pose generation unit 227 performs sampling on all temporary trajectory data parts.

[0221] In this way, the number of temporary pose data parts, each forming the temporary trajectory data parts, becomes a number equal to the number of configuration data.

[0222] Fig. 18 represents a concrete example of sampling.

[0223] "A n “ represents temporary pose data contained in temporary trajectory data from the nth time before sampling.

[0224] "A n “ represents temporary pose data contained in temporary trajectory data of the nth time after sampling.

[0225] “*” denotes multiplication and “+” denotes addition.

[0226] Each temporary pose data segment specifies the respective position of a plurality of joints. Multiplication and addition are performed for each joint on coordinate values ​​that identify its position.

[0227] It is assumed that the number of configuration data points is "4".

[0228] Before sampling, the temporary trajectory data for the first time period are generated from three temporary pose data parts A1(1-3). Furthermore, the temporary trajectory data for the second time period are generated from four temporary pose data parts A2(1-4). Finally, the temporary trajectory data for the third time period are generated from five temporary pose data parts A3(1-5).

[0229] By sampling the temporary trajectory data of the first time, four temporary pose data parts A1'(1-4) are generated based on three temporary pose data parts A1(1-3).

[0230] By sampling the temporary trajectory data of the second time, four temporary pose data parts A2'(1-4) are generated based on four temporary pose data parts A2(1-4).

[0231] By sampling the temporary trajectory data of the third time, four temporary pose data parts A3'(1-4) are generated based on five temporary pose data parts A3(1-5).

[0232] After sampling, the temporary trajectory data for the first time period are generated from four temporary pose data parts A1'(1-4). Furthermore, the temporary trajectory data for the second time period are generated from four temporary pose data parts A2'(1-4). Finally, the temporary trajectory data for the third time period are generated from four temporary pose data parts A3'(1-4).

[0233] It will be revisited Fig. 17 Referenced to describe step S254.

[0234] In step S254, the motion pose generation unit 227 generates a number of motion pose data parts equal to the number of configuration data.

[0235] The motion pose generation unit 227 generates motion pose data at all times as follows.

[0236] The motion pose generation unit 227 calculates the sum of a plurality of temporary pose data parts linked to the same time. The sum of these multiple temporary pose data parts is calculated for each joint using coordinate values ​​that identify its position.

[0237] The motion pose generation unit 227 divides the sum of the multiple temporary pose data parts by the number of temporary pose data parts. This calculates the average of the multiple temporary pose data parts associated with the same time. The average of the multiple temporary pose data parts becomes the motion pose data.

[0238] However, the motion pose generation unit 227 can calculate a distribution of a plurality of temporary pose data parts together with the average of the plurality of temporary pose data parts.

[0239] It will be revisited Fig. 16 Referenced to describe step S260.

[0240] In step S260, the motion registration unit 224 links and registers for each motion contained in a series of motions two or more times, two or more motion pose data parts, two or more motion identifiers to identify the same motion in the motion database 291. ***Effect of the second embodiment***

[0241] The second embodiment makes it possible to generate the motion database 291 based on a series of movements performed at multiple times. As a result, it is possible to increase the accuracy of the motion analysis. Third embodiment

[0242] With reference to an embodiment in which motion analysis is performed using a learning model, the following are based on Fig. 19 to Fig. 22 main points are described that differ from the first embodiment and the second embodiment. ***Explanation of the configuration***

[0243] One configuration of the motion analysis system 100 is the configuration in the first embodiment (see Fig. 1 and Fig. 11) same.

[0244] Now the configuration of the motion analysis device 200 is based on... Fig. 19 described.

[0245] The motion analysis device 200 includes elements such as a specification unit 210 and a learning unit 230. These elements are implemented through software.

[0246] The motion analysis program performs a computer function as the specification unit 210 and the learning unit 230. Learning unit 230 generates a learning model 292.

[0247] A memory unit 290 stores the learning model 292.

[0248] The learning model 292 outputs a motion identifier in response to input of motion trajectory data.

[0249] Now the configuration of learning unit 230 is based on... Fig. 20 described.

[0250] The learning unit 230 includes elements such as a movement recognition acceptance unit 231, a movement image acceptance unit 232, a movement pose detection unit 233 and a movement learning unit 234. ***Explanation of a movement***

[0251] A motion analysis method is based on Fig. 21 described.

[0252] In step S310, a target image receiving unit 211 receives a plurality of target image data parts.

[0253] Step S310 is identical to step S110 in the first embodiment (see Fig. 3).

[0254] In step S320, a target pose detection unit 212 detects a pose of the target person 111, which is shown in a target image, from all target image data and generates target pose data which represent the detected pose.

[0255] In this way, a plurality of target pose data parts are obtained, which correspond to a plurality of target image data parts.

[0256] Step S320 is identical to step S120 in the first embodiment (see Fig. 3).

[0257] In step S330, a target trajectory extraction unit extracts 213 target trajectory data from a plurality of target pose data parts, which correspond to a plurality of target image data parts.

[0258] Step S330 is identical to step S130 in the first embodiment (see Fig. 3).

[0259] In step S340, a motion specification unit 214 receives a motion identifier to identify the movement of the target person 111 based on target trajectory data.

[0260] The motion identifier is obtained in the following way.

[0261] The motion specification unit 214 inputs the target trajectory data into the learning model.

[0262] The learning model outputs a movement identifier from learning trajectory data that matches the input target trajectory data.

[0263] The motion specification unit 214 receives the motion identifier from the learning model.

[0264] Steps S330 and S340 are performed for each photograph within a photograph time slot. This ensures that a motion indicator is obtained for each photograph.

[0265] After steps S330 and S340 have been performed for each photographic time in a photographic time slot, processing continues to step S350.

[0266] In step S350, the result output unit 215 outputs result data.

[0267] Step S350 is identical to step S150 in the first embodiment (see Fig. 3). ***Explanation of a preprocessing process***

[0268] Based on Fig. 22 describes a learning process through learning unit 230.

[0269] The learning model 292 is generated by the learning process.

[0270] In step S301, the motion identifier acceptance unit 231 receives a plurality of motion identifiers to which a plurality of times have been added.

[0271] Step S301 is identical to step S101 in the first embodiment (see Fig. 13).

[0272] In step S302, the motion image reception unit 232 receives a plurality of motion image data parts, to which a plurality of times are added.

[0273] Step S302 is identical to step S102 in the first embodiment (see Fig. 13).

[0274] In step S303, the motion pose detection unit 233 captures a pose of the acting person 112, which is shown in a motion image, from all motion image data and generates motion pose data which represent the captured pose.

[0275] In this way, a plurality of motion pose data parts are obtained, which correspond to a plurality of motion image data parts.

[0276] Step S303 is identical to step S103 in the first embodiment (see Fig. 13).

[0277] In step S304, the movement learning unit 234 performs a learning process by converting a set of movement pose data and a movement identifier in relation to respective times into learning data.

[0278] For example, the motion learning unit 234 performs machine learning using an existing machine learning model. More precisely, the motion learning unit 234 inputs learning data into the machine learning model at any given time. The machine learning model then performs machine learning on this data at any given time. However, the motion learning unit 234 can also perform learning using a different method.

[0279] In this way, learning model 292 is generated. If learning model 292 exists, it is updated. ***Effect of the third embodiment***

[0280] The third embodiment makes it possible to perform a motion analysis using the learning model 292. As a result, it is possible to increase the accuracy of the motion analysis. *** Supplement to the embodiments ***

[0281] A hardware configuration of the motion analysis device 200 is based on Fig. 23 described.

[0282] The motion analysis device 200 has a processing circuit 209.

[0283] The processing circuit 209 is a hardware component for the realization of the specification unit 210, the registration unit 220 and the learning unit 230.

[0284] The processing circuit 209 can be a dedicated piece of hardware or it can be a processor 201 that executes a program stored in memory 202.

[0285] If the processing circuit 209 is a dedicated hardware component, then the processing circuit 209 is, for example, a single integrated circuit, a compound integrated circuit, a programmed processor, a parallel programmed processor, an ASIC or an FPGA, or a combination thereof.

[0286] ASIC is the abbreviation for "application specific integrated circuit".

[0287] FPGA is an abbreviation for "Field-Programmable Gate Array".

[0288] The motion analysis device 200 can have multiple processing circuits that replace the processing circuit 209. These multiple processing circuits share the functions of the processing circuit 209.

[0289] In the processing circuit 209, some of the functions can be implemented by a dedicated hardware component, and the remaining functions can be implemented by software or firmware.

[0290] As described above, it is possible to implement each function of the motion analysis device 200 through a hardware component, through software, firmware or a combination thereof.

[0291] Each of the present embodiments is an example of a preferred embodiment and is not intended to limit the technical scope of the present invention. Each of the present embodiments can also be implemented only partially or in combination with another embodiment. Methods described by means of flowcharts, etc., can be modified as appropriate.

[0292] “Unit” as an element of the motion analysis device 200 can be replaced by “Process” or “Step”.

[0293] The motion analysis device 200 can be implemented by a plurality of devices.

[0294] Cameras 101 and 102 can be replaced by a depth sensor or another type of sensor. Three-dimensional data is obtained from these sensors.

[0295] The learning model 292 can be implemented by a hardware component or by a combination of software and a hardware component. Reference symbol list

[0296] 100: Motion analysis system; 101: Camera; 102: Camera; 111: Target person; 112: Acting person; 121: Target image data; 122: Target pose data; 123: Target trajectory data; 131: Motion image data; 132: Motion pose data; 133: Motion trajectory data; 200: Motion analysis device; 201: Processor; 202: Memory; 203: Auxiliary memory device; 204: Communication device; 205: Input and output interface; 209: Processing circuit; 210: Specification unit; 211: Target image acquisition unit; 212: Target pose acquisition unit; 213: Target trajectory extraction unit; 214: Motion specification unit; 215: Result output unit; 220: Registration unit; 221: Motion recognition acquisition unit; 222: Motion image acquisition unit; 223: Motion pose acquisition unit; 224: Motion registration unit; 225: Temporary pose acquisition unit; 226: Temporary table generation unit; 227: Motion pose generation unit; 230: Learning unit;231: Motion recognition acquisition unit; 232: Motion image acquisition unit; 233: Motion pose detection unit; 234: Motion learning unit; 290: Storage unit; 291: Motion database; 292: Learning model;

Claims

[1] Motion analysis system (100), comprising: a target image receiving unit (211) for receiving a plurality of pieces of target image data obtained by sequentially photographing a target person who is to perform a series of movements; a target pose detecting unit (212) for detecting a pose of the target person shown in a target image from each of the plurality of target image data pieces and generating target pose data representing the detected pose; a target trajectory extraction unit (213) for extracting, from a plurality of pieces of target pose data corresponding to the plurality of pieces of target image data, as target trajectory data representing a transition of the pose of the target person, two or more pieces of target pose data including target pose data at a target time and target pose data from a time that is a preceding time continuing up to the target time and / or a following time following the target time; and a movement specifying unit (214) for obtaining a movement identifier for identifying a movement of the target person based on the target trajectory data from a movement database (291), wherein the motion specifying unit (214) extracts a motion identifier of motion trajectory data matching the target trajectory data from the motion database on which a plurality of sets of a motion identifier for identifying a motion and motion trajectory data representing a transition of a pose of a person to perform a motion identified by the motion identifier are recorded; wherein in the motion database (291), a plurality of times, a plurality of motion pose data pieces and a plurality of motion identifiers are linked together, each motion trajectory data piece is formed by two or more motion pose data pieces, the target trajectory extraction unit (213) determines a target time group containing the target time and extracts two or more pieces of target pose data corresponding to two or more photographing times constituting the target time group as target trajectory data, and the motion specifying unit (214) determines a comparison time group containing a comparison time for each comparison time and extracts from the motion database (291) two or more pieces of motion pose data associated with two or more comparison times constituting the determined comparison time group, compares motion trajectory data formed by the two or more extracted pieces of motion pose data with the target trajectory data, selects the motion trajectory data that matches the target trajectory data based on a result of a comparison of the individual pieces of motion trajectory data and the target trajectory data, and extracts, as a motion identifier that matches the target trajectory data, a motion identifier associated with the comparison time corresponding to the selected motion trajectory data; wherein the motion analysis system (100) further comprises: a motion ID accepting unit (221) for accepting a plurality of motion IDs group which is a plurality of motion IDs to which the plurality of times have been added; a motion image receiving unit (222) for receiving a plurality of motion image data groups which are a plurality of pieces of motion image data to which a plurality of times have been added and which are obtained by successively photographing an actor who is to perform a series of movements; a temporary pose detecting unit (225) for detecting a pose of the actor shown in a motion image from all of the plurality of motion image data pieces in the motion image data group and generating temporary pose data representing the detected pose for each assumed motion image data group; a temporary table generating unit (226) for generating a plurality of temporary tables in which a time group, a temporary pose data group corresponding to the motion image data group, and the motion ID group are linked together; a motion pose generation unit (227) for extracting temporary trajectory data, which are two or more pieces of temporary pose data associated with two or more motion identifiers for identifying a same motion, from each of the plurality of temporary tables, and for generating two or more pieces of motion pose data corresponding to two or more times based on a plurality of pieces of temporary trajectory data extracted from the plurality of temporary tables for each motion included in the series of motions; and a motion registration unit (224) for linking together and registering the two or more times, the two or more pieces of motion pose data, the two or more motion identifiers for identifying the same motion in the motion database (291), for each motion included in the series of motions. [2] The motion analysis system (100) of claim 1, wherein each piece of target pose data indicates the respective position of a plurality of joints of the target person. [3] Motion analysis program to cause a computer to perform: a target image accepting process for accepting a plurality of pieces of target image data obtained by sequentially photographing a target person who is to perform a series of movements; a target pose detection process for detecting a pose of the target person shown in a target image from each of the plurality of target image data pieces and generating target pose data representing the detected pose; a target trajectory extraction process for extracting, from a plurality of pieces of target pose data corresponding to the plurality of pieces of target image data, as target trajectory data representing a transition of the pose of the target person, two or more pieces of target pose data including target pose data at a target time and target pose data of at least one time being a preceding time continuing up to the target time and a following time following the target time; and a motion specification process for obtaining a motion identifier for identifying a movement of the target person based on the target trajectory data from a motion database (291), wherein the motion specifying process extracts a motion identifier of motion trajectory data matching the target trajectory data from a motion database (291) on which a plurality of sets of a motion identifier for identifying a motion and motion trajectory data representing a transition of a pose of a person to perform a motion identified by the motion identifier are recorded, wherein in the motion database (291), a plurality of times, a plurality of motion pose data pieces and a plurality of motion identifiers are linked together, each motion trajectory data piece is formed by two or more motion pose data pieces, the target trajectory extraction process determines a target time group containing the target time and extracts two or more pieces of target pose data corresponding to two or more photographing times constituting the target time group as target trajectory data, and the motion specifying process determines a comparison time group containing a comparison time for each comparison time and extracts from the motion database (291) two or more pieces of motion pose data associated with two or more comparison times constituting the determined comparison time group, compares motion trajectory data constituted by the two or more extracted pieces of motion pose data with the target trajectory data, selects the motion trajectory data that matches the target trajectory data based on a result of a comparison of the individual pieces of motion trajectory data and the target trajectory data, and extracts, as a motion identifier that matches the target trajectory data, a motion identifier associated with the comparison time corresponding to the selected motion trajectory data; The motion analysis program causes the computer to to carry out: a motion ID accepting process for accepting a motion ID group that is a plurality of motion IDs to which the plurality of times have been added, in multiples; a motion image receiving process for receiving a plurality of times a motion image data group which is a plurality of pieces of motion image data to which a plurality of times have been added and which are obtained by successively photographing an actor who is to perform a series of movements; a temporary pose detection process for detecting a pose of the actor shown in a motion image from all of the plurality of motion image data pieces in the motion image data group and generating temporary pose data representing the detected pose for each assumed motion image data group; a temporary table generation process for generating a plurality of temporary tables in which a time group, a temporary pose data group corresponding to the motion image data group, and the motion ID group are linked together; a motion pose generation process for extracting temporary trajectory data, which are two or more pieces of temporary pose data associated with two or more motion identifiers for identifying a same motion, from each of the plurality of temporary tables, and for generating two or more pieces of motion pose data corresponding to two or more times based on a plurality of pieces of temporary trajectory data extracted from the plurality of temporary tables for each motion included in the series of motions; and a motion registration process for linking together and registering the two or more times, the two or more pieces of motion pose data, the two or more motion identifiers for identifying the same motion in the motion database (291), for each motion included in the series of motions.

Images