Analysis device, analysis method, analysis program, and generation device

Abstract

To provide an analysis device in which a degree where each of multiple structures contributes to operation can be reflected to feature information with high accuracy.SOLUTION: An analysis device 20 analyzes target operation having multiple structures connected to each other. The analysis device 20 includes an angular momentum information acquisition part 202 and a feature information acquisition part 203. The angular momentum information acquisition part 202 acquires angular momentum information for representing a time series of each angular momentum of the multiple structures. The feature information acquisition part 203 acquires feature information being the information based on the first singular vector having the element corresponding to a first singular value being the maximum in the singular values of a first matrix where the angular momentum information to each of the multiple structures is the element and corresponding to each of the multiple structures, and being the information for representing the features of the target operation.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an analysis device, an analysis method, an analysis program, and a generation device. [Background technology]

[0002] There is known an analysis device that analyzes the motion of an object having a plurality of mutually connected components. For example, the analysis device described in Patent Document 1 analyzes human walking. The analysis device acquires the amount of sway of each of the lower back and thorax and back as feature information representing the characteristics of walking, and displays the acquired feature information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2015 / 129883 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, humans may have paralysis in the left or right half of the body. In this case, the greater the degree of disability due to paralysis, the greater the difference in the degree to which the components included in the left half of the body and the components included in the right half of the body contribute to the movement during walking. For example, the components are the thigh, the lower leg, the foot, the upper arm, or the forearm. Therefore, if the degree to which the components included in the left half of the body and the components included in the right half of the body contribute to the movement can be known with high accuracy, it is easy to evaluate, for example, the progress of rehabilitation.

[0005] However, the above-mentioned analysis device has a problem that the degree to which the components included in the left half of the body and the components included in the right half of the body contribute to the movement is not easily reflected in the feature information. This type of problem can occur in movements other than walking as well.

[0006] One object of the present invention is to reflect the degree to which each of a plurality of components contributes to an operation in feature information with high accuracy. [Means for solving the problem]

[0007] In one aspect, an analysis device analyzes a motion of a target having a plurality of components connected to each other. The analysis device includes an angular momentum information acquisition unit and a feature information acquisition unit. The angular momentum information acquisition unit acquires angular momentum information that represents a time series of the angular momentum of each of the plurality of components. A feature information acquisition unit acquires feature information, which corresponds to a first singular value that is the largest among the singular values ​​of a first matrix whose elements are angular momentum information for each of the multiple components, and is information based on a first singular vector having elements corresponding to each of the multiple components, and which is information that represents the features of the target's motion.

[0008] In another aspect, the analysis method analyzes the motion of a subject having multiple entities linked together. The analysis method is as follows: acquiring angular momentum information representing a time series of the angular momentum of each of the plurality of components; The method includes acquiring characteristic information, which corresponds to a first singular value that is the largest among the singular values ​​of a first matrix whose elements are angular momentum information for each of the multiple components, and is information based on a first singular vector having elements corresponding to each of the multiple components, and which is information that represents characteristics of the target's motion.

[0009] In another aspect, the analysis program causes a computer to execute a process for analyzing the behavior of an object having a plurality of interconnected entities. The process is acquiring angular momentum information representing a time series of the angular momentum of each of the plurality of components; The method includes acquiring characteristic information, which corresponds to a first singular value that is the largest among the singular values ​​of a first matrix whose elements are angular momentum information for each of the multiple components, and is information based on a first singular vector having elements corresponding to each of the multiple components, and which is information that represents characteristics of the target's motion.

[0010] In another aspect, a generator generates a trained model used to analyze the behavior of an object having multiple interconnected constructs. The generation device includes an angular momentum information acquisition unit, a first singular vector acquisition unit, a second singular vector acquisition unit, and a model generation unit.

[0011] The angular momentum information acquisition unit acquires, for each of the plurality of learning objects, angular momentum information that represents a time series of the angular momentum of each of the plurality of constituents. The first singular vector acquisition unit acquires, for each of the multiple learning objects, a first singular vector that corresponds to a first singular value that is the largest among the singular values ​​of a first matrix whose elements are angular momentum information for each of the multiple constituents, and has elements that correspond to each of the multiple constituents.

[0012] The second singular vector acquisition unit acquires second singular vectors that correspond to a second largest second singular value among singular values ​​of a second matrix whose elements are the first singular vectors for each of the multiple learning objects, and that have elements corresponding to each of the multiple learning objects. The model generation unit generates a trained model for each of a plurality of learning objects by learning teacher data including angular momentum information of the learning object and information based on an element of the second singular vector corresponding to the learning object. Effect of the Invention

[0013] The degree to which each of the multiple components contributes to the operation can be reflected in the feature information with high accuracy. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram illustrating a configuration of an analysis system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating a configuration of a generating device according to the first embodiment. [Diagram 3] FIG. 1 is a block diagram illustrating a configuration of an analysis device according to a first embodiment. [Figure 4] FIG. 2 is a block diagram illustrating the functions of a generating device according to the first embodiment. [Diagram 5] FIG. 2 is a block diagram illustrating the functions of the analysis device according to the first embodiment. [Figure 6] 4 is a flowchart showing a process executed by a generating device according to the first embodiment. [Figure 7] 4 is a flowchart showing a process executed by the analysis device of the first embodiment. [Figure 8] 4 is a graph illustrating an example of a result of singular value decomposition of a first matrix calculated by the analysis system of the first embodiment. [Figure 9] 4 is a graph illustrating an example of a result of singular value decomposition of a first matrix calculated by the analysis system of the first embodiment. [Figure 10] 4 is a graph illustrating an example of a result of singular value decomposition of a second matrix calculated by the analysis system of the first embodiment. [Figure 11] 4 is a graph illustrating an example of a result of singular value decomposition of a second matrix calculated by the analysis system of the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments of an analysis device, an analysis method, an analysis program, and a generation device of the present invention will be described with reference to FIGS.

[0016] First Embodiment (overview) The analysis device according to the first embodiment analyzes the motion of an object having a plurality of components connected to each other. The analysis device includes an angular momentum information acquisition unit and a feature information acquisition unit. The angular momentum information acquisition unit acquires angular momentum information that represents a time series of the angular momentum of each of the plurality of components. The feature information acquisition unit acquires feature information, which corresponds to a first singular value that is the largest among the singular values ​​of a first matrix whose elements are angular momentum information for each of the multiple components, and is information based on a first singular vector having elements corresponding to each of the multiple components, and which is information that represents the features of the target's motion.

[0017] According to this, the degree to which each of the multiple constituents contributes to the movement is reflected in the first singular vector. Therefore, the degree to which each of the multiple constituents contributes to the movement can be reflected in the feature information with high accuracy. As a result, the degree to which the constituents included in the left half of the body and the constituents included in the right half of the body contribute to the movement can be known with high accuracy. Therefore, for example, the progress of rehabilitation can be appropriately evaluated. Next, the analysis system of the first embodiment will be described in more detail.

[0018] (composition) 1, the analysis system 1 includes a generating device 10 and an analyzing device 20. The generating device 10 and the analyzing device 20 are communicatively connected to each other via a communication line NW. The communication line NW may include a transmission path for wireless communication.

[0019] Each of the generating device 10 and the analyzing device 20 is an information processing device or a computer. For example, the computer may be at least a part of a stationary game machine, a portable game machine, a television receiver, a smartphone, or the like. For example, the generating device 10 may be a server computer, a desktop computer, a laptop computer, a tablet computer, a smartphone, or the like. For example, the analyzing device 20 may be a desktop computer, a laptop computer, a tablet computer, a smartphone, or the like. Each of the generating device 10 and the analyzing device 20 may be composed of a plurality of devices connected to each other so as to be able to communicate with each other.

[0020] As shown in FIG. 2, the generation device 10 includes a processing device 11, a storage device 12, an input device 13, an output device 14, and a communication device 15, which are connected to each other via a bus BU1. The processing device 11 executes a program stored in the storage device 12 to control the storage device 12, the input device 13, the output device 14, and the communication device 15. In this way, the processing device 11 realizes functions that will be described later.

[0021] In this example, the processing device 11 is a central processing unit (CPU). Note that the processing device 11 may include a micro processing unit (MPU), a graphics processing unit (GPU), or a digital signal processor (DSP) instead of or in addition to the CPU.

[0022] In this example, the storage device 12 includes a volatile memory and a non-volatile memory. For example, the storage device 12 includes at least one of a random access memory (RAM), a read only memory (ROM), a semiconductor memory, an organic memory, a hard disk drive (HDD), and a solid state drive (SSD).

[0023] The input device 13 receives information from outside the generating device 10. In this example, the input device 13 includes a keyboard and a mouse. The input device 13 may also include a microphone. The input device 13 is connected to a photographing device 50.

[0024] The photographing device 50 includes a first photographing section and a second photographing section. The first photographing section photographs an object and obtains a visible light image representing the photographed object. The visible light image is an image representing the intensity of visible light reflected by the object for each of a plurality of pixels. In this example, the plurality of pixels included in the visible light image are arranged in a lattice pattern.

[0025] In this example, the first image capturing unit includes a color camera or an RGB (Red Green Blue) camera. Note that the first image capturing unit may be a black and white camera. In this example, the first image capturing unit acquires visible light video information representing a time series of visible light images.

[0026] The second image capturing unit captures an image of an object and obtains a distance image representing the captured object. The distance image is an image representing the distance between a reference plane and the object for each of a plurality of pixels. In this example, the reference plane forms a vertical plane. The reference plane may be inclined with respect to the vertical plane. In this example, the plurality of pixels included in the distance image have a lattice-like arrangement.

[0027] In this example, the second image capturing unit includes a TOF (Time Of Flight) camera. The second image capturing unit may acquire distance images by using a stereo camera instead of the TOF camera. The second image capturing unit may also acquire distance images by using a distance measuring sensor that measures the distance between a reference plane and an object for each of a plurality of pixels arranged in a lattice pattern. In this example, the second image capturing unit acquires distance video information that represents a time series of distance images.

[0028] The input device 13 receives the visible light video information and the distance video information acquired by the image capturing device 50 . The imaging device 50 may constitute a part of the generating device 10.

[0029] The output device 14 outputs information to the outside of the generation device 10. In this example, the output device 14 includes a display. The output device 14 may also include a speaker. In addition, the generation device 10 may be equipped with a touch panel display that constitutes both the input device 13 and the output device 14.

[0030] The communication device 15 communicates with devices external to the generating device 10. In this example, the communication device 15 comprises a card-type or on-board type network adapter or network interface.

[0031] 3, the analysis device 20 includes a processing device 21, a storage device 22, an input device 23, an output device 24, and a communication device 25, which are connected to each other via a bus BU2. The processing device 21, the storage device 22, the input device 23, the output device 24, and the communication device 25 have similar functions to the processing device 11, the storage device 12, the input device 13, the output device 14, and the communication device 15 of the generation device 10, respectively.

[0032] The input device 23 is connected to the photographing device 60. The photographing device 60 has a similar configuration to the photographing device 50. To the input device 23, visible light video information and distance video information acquired by the photographing device 60 are input. The imaging device 60 may constitute a part of the analysis device 20.

[0033] (function) The generation device 10 generates a trained model. The trained model is used to analyze the behavior of an object having multiple components connected to each other. In this example, the subject is a human with hemiplegia on the left or right side of the body. The subject may be a human without paralysis. The subject may also be an animal other than a human (e.g., a dog, a cat, or a horse).

[0034] In this example, the multiple components are 11 components consisting of a pelvis, a forearm, an upper arm, a thigh, a lower leg, and a foot, each of which is included in the half of the body on the paralyzed side, and a forearm, an upper arm, a thigh, a lower leg, and a foot, each of which is included in the half of the body on the non-paralyzed side. The multiple components may be a part of the above 11 components. The multiple components may also include other components in addition to the above 11 components.

[0035] In this example, the motion is walking. However, the motion may be a motion other than walking (for example, getting up, sitting down, standing up, running, pitching, swimming, etc.).

[0036] As shown in FIG. 4, the functions of the generation device 10 include an angular momentum information acquisition unit 101, a first singular vector acquisition unit 102, a second singular vector acquisition unit 103, and a model generation unit 104.

[0037] The angular momentum information acquisition unit 101 acquires angular momentum information for each of a plurality of (in this example, K) learning objects based on the visible light video information and the distance video information acquired by the image capturing device 50. K represents an integer of 2 or greater.

[0038] The angular momentum information represents a time series of the angular momentum of each of the multiple constituents during a certain motion period. For example, if the motion is cyclical, the motion period corresponds to one cycle. In this example, the motion period corresponds to a period of two steps, one step on each side. In this example, the motion period starts when the foot included in the paralyzed half of the body touches the ground, and ends when the foot touches the ground again after it leaves the ground.

[0039] In this example, the angular momentum information is obtained as follows. The angular momentum information acquisition section 101 estimates a time series of a plurality of (14 in this example) bone structure positions during a motion period based on the visible light video information and the distance video information. Each of the multiple skeletal positions corresponds to either end of the 11 components. In this example, the 14 skeletal positions are the wrist, elbow, shoulder, hip, knee, ankle, and toe that are included in the right half of the body, and the wrist, elbow, shoulder, hip, knee, ankle, and toe that are included in the left half of the body.

[0040] The buttocks included in the right half of the body and the buttocks included in the left half of the body form the two ends of the pelvis. The wrist and elbow form the two ends of the forearm. The elbow and shoulder form the two ends of the upper arm. The buttocks and knee form the two ends of the thigh. The knee and ankle form the two ends of the lower leg. The ankle and toe form the two ends of the foot.

[0041] The angular momentum information acquisition unit 101 calculates a time series R of the position of the overall center of gravity of the subject during the motion period based on the estimated time series of the multiple skeletal configuration positions. (k) w (t j ), the time series of the velocity of the object's overall center of mass V (k) w (t j ), the time series of the positions of the centers of gravity of multiple constructs R (k) i (t j ), the time series of the velocity of the center of gravity of each of the multiple components V (k) i (t j ), and the time series Ω of the angular velocity of each of the multiple components (k) i (t j ) is estimated. For example, the above estimation may be performed using the technique described in Non-Patent Document 1 below. (Non-Patent Document 1) Jun Inagaki and 4 others, "Detection and Accuracy of the Body's Combined Center of Gravity Using a Kinect Sensor," Journal of Clinical Gait Analysis Research, Clinical Gait Analysis Research, 2017, Vol. 4, No. 1, pp. 21-27

[0042] k represents an integer from 1 to K. In this example, the superscript "(k)" on a variable indicates that the variable is a value for the kth learning object. i represents an integer from 1 to I. I represents an integer equal to or greater than 2. In this example, I represents 11. In this example, the subscript "i" for a variable represents a value for the i-th component. In this example, the first component corresponds to the pelvis. The second to sixth components correspond to the upper arm, forearm, thigh, lower leg, and foot, respectively, which are included in the half of the body on the side not affected by paralysis. The seventh to eleventh components correspond to the foot, lower leg, thigh, forearm, and upper arm, respectively, which are included in the half of the body on the side affected by paralysis.

[0043] t j represents time. j represents an integer from 1 to J. J represents an integer of 2 or more. In this example, J represents 100. In this example, t1 to t Jcorrespond to J time points that equally divide the operating period.

[0044] The angular momentum information acquisition unit 101 calculates a time series L of the angular momentum of each of the plurality of components during the operation period based on the estimated time series group and the following formulas 1 and 2. (k) i (t j ) is estimated. In this way, the angular momentum information acquisition unit 101 estimates the time series L of the angular momentum of each of the multiple constituent bodies. (k) i (t j ) to obtain angular momentum information.

number

number

[0045] B (k) i (t j ) represents the angular momentum vector of the i-th component. D (k) represents a reference vector, which is a unit vector that represents the reference rotation direction. In this example, the reference vector D (k) represents a unit vector that is perpendicular to the frontal plane (in other words, the coronal plane) and has a direction toward the rear (from ventral to dorsal) of the subject when the right half of the body is paralyzed, and represents a unit vector that has a direction toward the front (from dorsal to ventral) of the subject when the left half of the body is paralyzed. Therefore, in this example, the angular momentum L (k) i (t j ) is the angular momentum vector B (k) i (t j ) represents the component of rotation about an axis perpendicular to the frontal plane.

[0046] m (k) i represents the mass of the i-th component. M (k) i represents the moment of inertia of the i-th component. For example, mass m(k) i , and moment of inertia M (k) i is a predetermined value. Note that mass m (k) i , and moment of inertia M (k) i At least one of the above may be a value input by a user of generation device 10, or a value estimated based on the visible light video information and the distance video information. In this example, angular momentum information is acquired in this manner.

[0047] The first singular vector acquisition unit 102 calculates a first matrix A based on the angular momentum information acquired by the angular momentum information acquisition unit 101. (k) Generate 1. First matrix A (k) 1, as expressed in the following Equation 3, has angular momentum information for each of the multiple constituent bodies as elements.

number

[0048] The first singular vector acquisition unit 102 calculates the generated first matrix A (k) 1 is subjected to singular value decomposition as shown in the following Equations 4 to 7 to obtain a first singular vector z (k) Get 1. The first singular vector z (k) 1 is the first matrix A (k) The first singular value λ, which is the largest among the singular values ​​of 1 (k) 1 and has elements corresponding to each of the multiple constructs (in this example, a right singular vector). In this example, the superscript "T" on the variables denotes a transposed matrix.

number

number

number

number

[0049] The first singular vector acquisition unit 102 acquires a first singular vector z (k) Obtain 1.

[0050] The second singular vector acquisition unit 103 acquires the first singular vector z (k) 1, a second matrix A2 represented by the following Equation 8 is generated. As represented in Equation 8, the second matrix A2 is obtained by dividing the first singular vector z (k) 1 is an element.

number

[0051] The second singular vector acquisition unit 103 acquires a second singular vector p2 by performing singular value decomposition on the generated second matrix A2 as expressed in the following Formula 9 to Formula 12. The second singular vector p2 corresponds to the second largest second singular value γ2 among the singular values ​​of the second matrix A2, and is a singular vector (in this example, a left singular vector) having elements corresponding to each of the multiple learning objects.

number

number

number

number

[0052] In this example, the k-th element p of the second singular vector p2 k,2represents a characteristic feature specific to the motion of the kth learning object relative to the average motion of a group of learning objects consisting of K learning objects. In this example, the kth element p of the second singular vector p2 becomes larger as the difference between the average motion of the group of learning objects and the motion of the kth learning object becomes larger. k,2 The size of becomes larger.

[0053] The model generation unit 104 generates a trained model based on the second singular vector p2 acquired by the second singular vector acquisition unit 103. In this example, the trained model is generated by inputting the angular momentum information L of each of the multiple training objects. (k) i (t j ) and the element p2 of the second singular vector p2 corresponding to the learning object k,2 This is done by learning training data including:

[0054] The trained model is generated by inputting the angular momentum information L of each of the multiple training objects. (k) i (t j ) and the element p2 of the second singular vector p2 corresponding to the learning object k,2 This may be performed by learning training data including a value obtained by multiplying

[0055] In addition, the generation of the trained model is performed by inputting the angular momentum information L of each of the multiple training objects. (k) i (t j ) and the element p2 of the second singular vector p2 corresponding to the learning object k,2 For example, the code information may be obtained by learning training data including the code information of the element p k,2 If p is a positive value, it represents "+1" and the element p k,2 If is a negative value, it represents "-1".

[0056] In addition, the generation of the trained model is performed by inputting the angular momentum information L of each of the multiple training objects.(k) i (t j ) and the element p2 of the second singular vector p2 corresponding to the learning object k,2 For example, the interval information may be obtained by learning teacher data including the element p k,2 The interval information indicates the interval (in other words, the class) in which the value of p is included. k,2 It may also represent an interval in which the magnitude of the value of

[0057] In addition, the angular momentum information L (k) i (t j ) may correspond to only a portion of the operating period. For example, the generation of the trained model may be performed by performing cross-validation.

[0058] The trained model includes a neural network. In this example, the trained model includes a regression type (in other words, a recurrent type) neural network (RNN; Recurrent Neural Network). In this example, the trained model includes a Long Short Term Memory (LSTM). In this example, the LSTM is a Deep Bidirectional LSTM (DBLSTM). Note that the trained model may include an RNN other than an LSTM.

[0059] The model generation unit 104 transmits model information representing the generated trained model to the analysis device 20.

[0060] The analysis device 20 analyzes the motion of the target object based on the trained model generated by the generation device 10 and the visible light video information and the distance video information acquired for the target object.

[0061] As shown in FIG. 5, the functions of the analysis device 20 include a model information storage unit 201, an angular momentum information acquisition unit 202, a feature information acquisition unit 203, and a feature information output unit 204.

[0062] The model information storage unit 201 receives the model information generated by the generation device 10 from the generation device 10 , and stores the received model information in the storage device 22 .

[0063] The angular momentum information acquisition unit 202 acquires angular momentum information for the target object based on the visible light video information and the distance video information acquired by the image capturing device 60. The angular momentum information acquisition unit 202 acquires the angular momentum information using a method similar to that used by the angular momentum information acquisition unit 101.

[0064] The feature information acquisition unit 203 acquires feature information for the target of interest based on the trained model represented by the model information stored in the model information storage unit 201 and the angular momentum information for the target of interest acquired by the angular momentum information acquisition unit 202. The feature information represents the features of the motion of the target of interest.

[0065] In this example, the feature information represents a value estimated as an element of the second singular vector p2 obtained for the multiple learning objects, corresponding to the target object of interest among the multiple learning objects, when it is assumed that the target object of interest is included among the multiple learning objects that served as the basis for generating the trained model. Therefore, in this example, the feature information corresponds to information based on the element of the second singular vector p2 that corresponds to the target object of interest among the multiple objects. In addition, the feature information represents a value estimated as an element of the first singular vector z (k) 1. Therefore, in this example, the feature information is the first singular vector z (k) Corresponds to information based on 1.

[0066] The characteristic information output unit 204 outputs the characteristic information acquired by the characteristic information acquisition unit 203 via the output device 24 (in this example, displays it on a display). In addition to or instead of the feature information, the feature information output unit 204 may output output information associated with the feature information. For example, the output information may represent the progress of rehabilitation or the normality of movement. In this case, the output information may represent that the smaller the value represented by the feature information is, the higher the progress of rehabilitation or the higher the normality of movement is.

[0067] The analysis device 20 may have the functions of the generation device 10 in addition to the functions of the analysis device 20. In this case, the analysis system 1 does not need to include the generation device 10 and the imaging device 50.

[0068] (operation) Next, the operation of the analysis system 1 will be described with reference to FIGS. In this example, the generation device 10 executes the process shown in FIG. 6 to generate a trained model. Specifically, the generating device 10 executes a first loop process (steps S101 to S104) in which each of the K learning objects is used as a processing object one by one in turn.

[0069] In the first loop process, the generating device 10 acquires angular momentum information for the learning object to be processed based on the visible light video information and the distance video information acquired by the image capturing device 50 (step S102). Next, the generating device 10 calculates a first singular vector z (k) 1 is obtained (step S103).

[0070] Then, the generating device 10 executes the first loop process for all of the learning objects, and then calculates the first singular vectors z (k) 1, a second singular vector p2 is obtained (step S105).

[0071] Next, the generating device 10 executes a second loop process (steps S106 to S108) in which each of the K learning objects is used as a processing object in turn, one by one.

[0072] In the second loop process, the generating device 10 calculates the angular momentum information L (k) i (t j ) and the element p2 of the second singular vector p2 acquired in step S105, which corresponds to the learning object. k,2 The training data including the above is learned (step S107).

[0073] Then, after executing the second loop process for all of the learning objects, the generation device 10 transmits model information representing the generated trained model to the analysis device 20 (Step S109). In this manner, the generating device 10 executes the process shown in FIG.

[0074] Furthermore, the analysis device 20 executes a process not shown in the figure to receive model information from the generation device 10 and store the received model information in the storage device 22 .

[0075] In this example, the analysis device 20 executes the process shown in FIG. 7 to analyze the motion of the target object based on the trained model generated by the generation device 10 and the visible light video information and distance video information acquired for the target object.

[0076] Specifically, analysis device 20 acquires angular momentum information for the target of interest based on the visible light video information and distance video information acquired by image capture device 60 (step S201). Next, analysis device 20 acquires feature information for the target of interest based on the trained model represented by the model information stored in storage device 22 and the angular momentum information acquired in step S201 (step S202).

[0077] Next, the analysis device 20 outputs the characteristic information acquired in step S202 via the output device 24 (step S203). In this manner, the analysis device 20 executes the process shown in FIG.

[0078] As described above, the analysis device 20 of the first embodiment analyzes the motion of a subject having a plurality of components connected to each other. The analysis device 20 includes an angular momentum information acquisition unit 202 and a feature information acquisition unit 203. The angular momentum information acquisition unit 202 acquires angular momentum information that represents a time series of the angular momentum of each of the multiple constituents. The feature information acquisition unit 203 acquires feature information that corresponds to a first singular value that is the largest among singular values ​​of a first matrix whose elements are the angular momentum information for each of the multiple constituents and is based on a first singular vector that has elements corresponding to each of the multiple constituents, and that represents a feature of the target's motion.

[0079] According to this, the degree to which each of the multiple constituents contributes to the movement is reflected in the first singular vector. Therefore, the degree to which each of the multiple constituents contributes to the movement can be reflected in the feature information with high accuracy. As a result, the degree to which the constituents included in the left half of the body and the constituents included in the right half of the body contribute to the movement can be known with high accuracy. Therefore, for example, the progress of rehabilitation can be appropriately evaluated.

[0080] Furthermore, in the analysis device 20 of the first embodiment, the feature information corresponds to the second largest singular value among the singular values ​​of a second matrix whose elements are the first singular vectors for each of the multiple objects, and is information based on an element of the second singular vector corresponding to a target object of interest among the multiple objects, the second singular vector having elements corresponding to each of the multiple objects.

[0081] According to this, in a group of objects consisting of multiple objects, the characteristics specific to the motion of each object are reflected in the second singular vector. Therefore, in the group of objects, the characteristics specific to the motion of a target object can be reflected in the feature information with high accuracy. As a result, for example, the characteristics specific to the motion of the target object relative to the average motion of the group of objects can be known with high accuracy.

[0082] Furthermore, in the analysis device 20 of the first embodiment, the feature information acquisition unit 203 acquires feature information based on the trained model and angular momentum information for the target of interest. The trained model is generated by acquiring a second singular vector that corresponds to the second largest second singular value among the singular values ​​of a second matrix in which the first singular vector for each of the multiple learning objects is an element and has elements corresponding to each of the multiple learning objects, and by learning, for each of the multiple learning objects, teacher data including the angular momentum information of the learning object and information based on the element of the second singular vector corresponding to the learning object.

[0083] According to this, in a group of objects consisting of multiple objects, the characteristics specific to the motion of each object are reflected in the second singular vector. Therefore, in the group of objects, the characteristics specific to the motion of a target object can be reflected in the feature information with high accuracy. As a result, for example, the characteristics specific to the motion of the target object relative to the average motion of the group of objects can be known with high accuracy. Furthermore, feature information for the target object of interest can be acquired more quickly than in the case where the second singular vector is calculated for a group of objects including the target object of interest.

[0084] Furthermore, in the analysis device 20 of the first embodiment, the subject is a human being who has paralysis on the left or right side of the body. The multiple components include a pelvis, a forearm, an upper arm, a thigh, a lower leg, and a foot, which are included in the half of the body on the paralyzed side, and a forearm, an upper arm, a thigh, a lower leg, and a foot, which are included in the half of the body on the non-paralyzed side.

[0085] According to this, the elements of the first matrix correspond to the components included in the half of the body on the paralyzed side and the components included in the half of the body on the non-paralyzed side, respectively. As a result, the degree to which the components contribute to the movement between the components included in the half of the body on the paralyzed side and the components included in the half of the body on the non-paralyzed side can be known with high accuracy. Therefore, for example, the progress of rehabilitation can be appropriately evaluated.

[0086] Furthermore, in the analysis device 20 of the first embodiment, the movement is walking. The angular momentum information corresponds to a period of two steps, one step to the left and one step to the right.

[0087] In a healthy person, it is expected that the degree to which the components in the left half of the body and the components in the right half of the body contribute to the movement is equal during walking. Therefore, the analysis device 20 can evaluate the degree to which walking is performed appropriately with high accuracy. Therefore, for example, the progress of rehabilitation can be appropriately evaluated.

[0088] Furthermore, the generation device 10 of the first embodiment generates a trained model used to analyze the behavior of an object having multiple components connected to each other. The generation device 10 includes an angular momentum information acquisition unit 101 , a first singular vector acquisition unit 102 , a second singular vector acquisition unit 103 , and a model generation unit 104 .

[0089] The angular momentum information acquiring unit 101 acquires, for each of a plurality of learning objects, angular momentum information that indicates a time series of the angular momentum of each of a plurality of constituents. The first singular vector acquisition unit 102 acquires, for each of the multiple learning objects, a first singular vector that corresponds to a first singular value that is the largest among the singular values ​​of a first matrix whose elements are angular momentum information for each of the multiple constituents, and has elements that correspond to each of the multiple constituents.

[0090] The second singular vector acquisition unit 103 acquires second singular vectors that correspond to the second largest second singular value among the singular values ​​of a second matrix whose elements are the first singular vectors for each of the multiple learning objects, and that have elements corresponding to each of the multiple learning objects. The model generation unit 104 generates a trained model for each of a plurality of learning objects by learning teacher data including angular momentum information of the learning object and information based on an element of the second singular vector corresponding to the learning object.

[0091] According to this, in a learning object group consisting of multiple learning objects, the characteristics specific to the movement of each learning object are reflected in the second singular vector. Therefore, the characteristics specific to the movement of each learning object relative to the average movement of the learning object group can be reflected in the trained model with high accuracy. As a result, by acquiring the characteristic information based on the trained model, the characteristics specific to the movement of the target object can be reflected in the characteristic information with high accuracy. In addition, the characteristic information for the target object can be acquired more quickly than when the second singular vector is calculated for a learning object group including the target object.

[0092] 8 and 9 are graphs showing an example of the result of singular value decomposition of the first matrix A1 calculated by the analysis system 1 of the first embodiment. Fig. 8(A) shows an example of a right singular vector (in this example, the first singular vector) z1 corresponding to the largest singular value (in this example, the first singular value) λ1 of the first matrix A1. Fig. 8(B) shows an example of a vector λ1u1 obtained by multiplying the largest singular value (in this example, the first singular value) λ1 of the first matrix A1 by the left singular vector u1 corresponding to the singular value λ1.

[0093] Fig. 9(A) shows an example of a right singular vector z2 corresponding to the second largest singular value λ2 of the first matrix A1. Fig. 9(B) shows an example of a vector λ2u2 obtained by multiplying the second largest singular value λ2 of the first matrix A1 by the left singular vector u2 corresponding to the singular value λ2.

[0094] 10 and 11 are graphs showing an example of the result of singular value decomposition of the second matrix A2 calculated by the analysis system 1 of the first embodiment. Fig. 10(A) shows an example of a right singular vector y1 corresponding to the maximum singular value γ1 of the second matrix A2. Fig. 10(B) shows an example of a vector γ1p1 obtained by multiplying the maximum singular value γ1 of the second matrix A2 by the left singular vector p1 corresponding to the singular value γ1.

[0095] Fig. 11(A) shows an example of a right singular vector y2 corresponding to the second largest singular value (in this example, the second singular value) γ2 of the second matrix A2. Fig. 11(B) shows an example of a vector γ2p2 obtained by multiplying the second largest singular value (in this example, the second singular value) γ2 of the second matrix A2 by a left singular vector (in this example, the second singular vector) p2 corresponding to the singular value γ2.

[0096] Meanwhile, the generating device 10 of the first embodiment acquires visible light video information and distance video information by inputting the information from the imaging device 50. The generating device 10 of the modified example of the first embodiment may acquire the visible light video information and distance video information by receiving the information from another device communicably connected to the generating device 10. The generating device 10 of the modified example of the first embodiment may acquire the visible light video information and distance video information by reading the information from a recording medium.

[0097] Meanwhile, the analysis device 20 of the first embodiment acquires visible light video information and distance video information by inputting the information from the image capture device 60. The analysis device 20 of the modified example of the first embodiment may acquire the visible light video information and distance video information by receiving the information from another device communicably connected to the analysis device 20. The analysis device 20 of the modified example of the first embodiment may acquire the visible light video information and distance video information by reading the information from a recording medium.

[0098] Meanwhile, each of the generating device 10 and the analyzing device 20 of the first embodiment acquires angular momentum information based on both the visible light video information and the distance video information. Note that each of the generating device 10 and the analyzing device 20 of the modified example of the first embodiment may acquire angular momentum information based on either the visible light video information or the distance video information.

[0099] Furthermore, each of the generating device 10 and the analyzing device 20 in the modified example of the first embodiment may acquire angular momentum information based on detection information instead of or in addition to the visible light video information and the distance video information. For example, the detection information may be information acquired using a sensor or a marker attached to each component.

[0100] The analysis device 20 of the first embodiment acquires feature information for the target of interest based on the trained model generated by the generation device 10. The analysis device 20 of the modified example of the first embodiment may acquire feature information for the target of interest without using a trained model. In this case, the analysis device 20 may acquire first singular vectors for each of the multiple learning objects and the target of interest, and may acquire second singular vectors based on the acquired first singular vectors, thereby acquiring elements of the second singular vectors for the target of interest as feature information for the target of interest. The analysis device 20 of the modified example of the first embodiment may also acquire first singular vectors for the target of interest based on angular momentum information, and use the acquired first singular vectors as feature information.

[0101] The present invention is not limited to the above-described embodiment. For example, various modifications that can be understood by a person skilled in the art may be made to the above-described embodiment without departing from the spirit of the present invention. [Explanation of symbols]

[0102] 1. Analysis system 10 Generator 11 Processing equipment 12 Storage device 13 Input Devices 14 Output Devices 15. Communications Equipment 101 Angular momentum information acquisition unit 102 First singular vector acquisition unit 103 Second singular vector acquisition unit 104 Model Generation Unit 20 Analyzer 21 Processing equipment 22 Storage device 23 Input Devices 24 Output Devices 25 Communication Equipment 201 Model information storage unit 202 Angular momentum information acquisition unit 203 Feature information acquisition unit 204 Feature information output unit 50 Imaging Equipment 60 Imaging Equipment BU1 Bus BU2 bus Network communication line

Claims

1. An analysis device for analyzing the motion of a subject having a plurality of interconnected components, an angular momentum information acquisition unit that acquires angular momentum information representing a time series of the angular momentum of each of the plurality of constituent bodies; a feature information acquisition unit that acquires feature information, the feature information being information based on a first singular vector having elements corresponding to each of the plurality of constituent bodies and corresponding to a first singular value that is a maximum among singular values ​​of a first matrix whose elements are the angular momentum information for each of the plurality of constituent bodies; and An analysis device comprising:

2. The analysis device according to claim 1 , an analysis device, wherein the feature information is information based on an element of a second singular vector that corresponds to a second largest singular value among singular values ​​of a second matrix whose elements are the first singular vectors for each of the multiple objects, and that has elements corresponding to each of the multiple objects, and that corresponds to an object of interest among the multiple objects.

3. The analysis device according to claim 1 or 2, The feature information acquisition unit acquires the feature information based on a trained model and the angular momentum information for an object of interest; the trained model is generated by acquiring a second singular vector that corresponds to a second largest second singular value among singular values ​​of a second matrix in which the first singular vector for each of a plurality of learning objects is an element, and that has elements corresponding to each of the plurality of learning objects, and by learning, for each of the plurality of learning objects, teacher data including the angular momentum information of the learning object and information of the second singular vector based on the element corresponding to the learning object.

4. The analysis device according to any one of claims 1 to 3, The subject is a human being with left or right hemiplegia, The multiple components of the analysis device include a pelvis, a forearm, an upper arm, a thigh, a lower leg, and a foot that are each included in the half of the body on the paralyzed side, and a forearm, an upper arm, a thigh, a lower leg, and a foot that are each included in the half of the body on the non-paralyzed side.

5. The analysis device according to any one of claims 1 to 4, the movement is walking, The angular momentum information corresponds to a period of two steps, one step to the left and one step to the right.

6. 1. A method for analyzing the behavior of a target having a plurality of interconnected components, comprising: acquiring angular momentum information representing a time series of the angular momentum of each of the plurality of components; acquiring feature information, the feature information being information based on a first singular vector having elements corresponding to each of the plurality of constituent bodies, the first singular value being the largest among singular values ​​of a first matrix whose elements are the angular momentum information for each of the plurality of constituent bodies; and the first singular vector having elements corresponding to each of the plurality of constituent bodies; The analytical method includes:

7. An analysis program that causes a computer to execute a process for analyzing the behavior of an object having a plurality of mutually connected components, The process comprises: acquiring angular momentum information representing a time series of the angular momentum of each of the plurality of components; acquiring feature information, the feature information being information based on a first singular vector having elements corresponding to each of the plurality of constituent bodies, the first singular value being the largest among singular values ​​of a first matrix whose elements are the angular momentum information for each of the plurality of constituent bodies; and the first singular vector having elements corresponding to each of the plurality of constituent bodies; (c) analyzing the data;

8. A generating device for generating a trained model used to analyze the behavior of an object having a plurality of interconnected components, comprising: an angular momentum information acquisition unit that acquires, for each of a plurality of learning objects, angular momentum information that represents a time series of the angular momentum of each of the plurality of constituents; a first singular vector acquisition unit that acquires, for each of the plurality of learning objects, a first singular vector that corresponds to a first singular value that is maximum among singular values ​​of a first matrix whose elements are the angular momentum information for each of the plurality of constituents, and has elements that correspond to each of the plurality of constituents; a second singular vector acquisition unit that acquires second singular vectors corresponding to a second largest second singular value among singular values ​​of a second matrix whose elements are the first singular vectors for each of the multiple learning objects, and having elements corresponding to each of the multiple learning objects; a model generation unit that generates the trained model by learning teacher data including, for each of the plurality of learning objects, the angular momentum information of the learning object and information of the second singular vector based on an element corresponding to the learning object; A generating device comprising:

Images