Analysis device, analysis method, generation device, and generation method
By generating a learned model using angular momentum information and singular value decomposition, the problem of the difficulty in reflecting the degree of contribution of structural movements in existing technologies is solved, and high-precision rehabilitation training assessment is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TOHOKU UNIV
- Filing Date
- 2022-02-11
- Publication Date
- 2026-08-04
AI Technical Summary
Existing analytical devices are unable to accurately reflect the contribution of multiple structures to movement, especially when hemiplegic patients walk, as the difference in contribution of the left and right hemiplegic structures to movement is difficult to reflect accurately.
By employing an angular momentum information acquisition unit and a feature information acquisition unit, the first singular vector and the second singular vector are obtained through singular value decomposition, a learned model is generated, and the contribution of multiple structures to the action is analyzed.
It achieves high-precision reflection of the degree of contribution of movement between the left and right half of the body structures, and can appropriately evaluate the progress of rehabilitation training, especially providing accurate structural contribution assessment in walking movements.
Smart Images
Figure CN117750906B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to analytical apparatus, analytical methods, analytical procedures, and generating apparatus. Background Technology
[0002] Analysis devices are known to analyze the movements of objects with multiple interconnected structures. For example, the analysis device described in Patent Document 1 analyzes a person's walking. The analysis device acquires the swaying amounts of the waist and chest / back as feature information representing the characteristics of walking, and displays the acquired feature information.
[0003] Existing technical documents
[0004] Patent documents
[0005] Patent Document 1: International Publication No. 2015 / 129883 Summary of the Invention
[0006] The problem that the invention aims to solve
[0007] However, sometimes a person has paralysis on one side of their body, either the left or right. In this case, the greater the degree of impairment caused by the paralysis, the greater the difference in the contribution of structures to movement between the left and right sides of the body during walking. These structures might include, for example, the thigh, calf, foot, upper arm, or forearm. Therefore, if the contribution of structures to movement between the left and right sides of the body can be accurately determined, for example, the progress of rehabilitation training can be easily evaluated.
[0008] However, the aforementioned analytical apparatus has the following problem: the contribution of structures contained in the left half of the body to movement is difficult to reflect in the feature information between the structures contained in the right half of the body and the structures contained in the left half of the body. This problem also arises in movements other than walking.
[0009] One of the objectives of this invention is to accurately reflect the contribution of each of multiple structures to the action in the feature information.
[0010] Methods for solving problems
[0011] In one aspect, the analysis device analyzes the motion of an object having multiple interconnected structures. The analysis device includes an angular momentum information acquisition unit and a feature information acquisition unit.
[0012] The angular momentum information acquisition unit acquires the time-series angular momentum information representing the angular momentum of each of the multiple structures.
[0013] The feature information acquisition unit acquires feature information, which is information based on the first singular vector and is information representing the characteristics of the action. The first singular vector corresponds to the largest singular value among the singular values of the first matrix that takes the angular momentum information for each of the multiple structures as elements, and has elements corresponding to each of the multiple structures.
[0014] On the other hand, the analysis method analyzes the actions of objects with multiple interconnected structures.
[0015] The analysis method includes the following steps:
[0016] Obtain angular momentum information representing the time series of the angular momentum of each of the multiple structures; and
[0017] The feature information is obtained based on the first singular vector and is information representing the characteristics of the action. The first singular vector corresponds to the largest singular value among the singular values of the first matrix that takes the angular momentum information for each of the multiple structures as elements, and has elements corresponding to each of the multiple structures.
[0018] On the other hand, the analysis program enables the computer to perform analysis of actions on objects that have multiple interconnected structures.
[0019] The process includes the following steps:
[0020] Obtain angular momentum information representing the time series of the angular momentum of each of the multiple structures; and
[0021] The feature information is obtained based on the first singular vector and is information representing the characteristics of the action. The first singular vector corresponds to the largest singular value among the singular values of the first matrix that takes the angular momentum information for each of the multiple structures as elements, and has elements corresponding to each of the multiple structures.
[0022] On the other hand, the generating device generates a learned model for analyzing the actions of objects with multiple interconnected structures.
[0023] The generation device includes an angular momentum information acquisition unit, a first singular vector acquisition unit, a second singular vector acquisition unit, and a model generation unit.
[0024] The angular momentum information acquisition unit acquires the time-series angular momentum information representing the angular momentum of each of the multiple learning objects.
[0025] The first singular vector acquisition unit acquires a first singular vector for each of the plurality of learning objects. The first singular vector corresponds to the largest first singular value among the singular values of a first matrix that takes the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures.
[0026] The second singular vector acquisition unit acquires a second singular vector, which corresponds to the second largest second singular value among the singular values of a second matrix that takes the first singular vector as an element for each of the plurality of learning objects, and has an element corresponding to each of the plurality of learning objects.
[0027] The model generation unit generates a learned model for each of the multiple learning objects by learning training data. The training data includes the angular momentum information of the learning object and information about the elements corresponding to the learning object based on the second singular vector.
[0028] Invention Effects
[0029] It can accurately reflect the contribution of each of multiple structures to the action in the feature information. Attached Figure Description
[0030] Figure 1 This is a block diagram showing the structure of the analysis system according to the first embodiment.
[0031] Figure 2 This is a block diagram showing the structure of the generating apparatus according to the first embodiment.
[0032] Figure 3 This is a block diagram showing the structure of the analysis apparatus according to the first embodiment.
[0033] Figure 4 This is a block diagram illustrating the function of the generating apparatus in the first embodiment.
[0034] Figure 5 This is a block diagram illustrating the function of the analysis apparatus in the first embodiment.
[0035] Figure 6 This is a flowchart illustrating the processing performed by the generation apparatus of the first embodiment.
[0036] Figure 7 This is a flowchart illustrating the processing performed by the analysis device in the first embodiment.
[0037] Figure 8 This is a diagram showing an example of the result of the singular value decomposition of the first matrix calculated by the analysis system of the first embodiment.
[0038] Figure 9This is a diagram showing an example of the result of the singular value decomposition of the first matrix calculated by the analysis system of the first embodiment.
[0039] Figure 10 This is a diagram showing an example of the result of the singular value decomposition of the second matrix calculated by the analysis system of the first embodiment.
[0040] Figure 11 This is a diagram showing an example of the result of the singular value decomposition of the second matrix calculated by the analysis system of the first embodiment. Detailed Implementation
[0041] The following is for reference Figures 1 to 11 Various embodiments of the present invention related to the analysis apparatus, analysis method, analysis program, and generation apparatus will be described.
[0042] <First Implementation>
[0043] (summary)
[0044] The analysis apparatus of the first embodiment analyzes the motion of an object having multiple interconnected structures. The analysis apparatus includes an angular momentum information acquisition unit and a feature information acquisition unit.
[0045] The angular momentum information acquisition unit acquires the time-series angular momentum information representing the angular momentum of each of the multiple structures.
[0046] The feature information acquisition unit acquires feature information based on a first singular vector and is information representing the action features of an object. The first singular vector corresponds to the largest first singular value among the singular values of a first matrix that takes the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures.
[0047] Therefore, the contribution of each of the multiple structures to the movement is reflected in the first singular vector. Thus, the contribution of each of the multiple structures to the movement can be accurately reflected in the feature information. As a result, the contribution of structures to the movement can be accurately determined between structures contained in the left and right halves of the body. Therefore, for example, the progress of rehabilitation training can be appropriately evaluated.
[0048] Next, the analysis system of the first embodiment will be described in more detail.
[0049] (structure)
[0050] like Figure 1As shown, the analysis system 1 includes a generation device 10 and an analysis device 20. The generation device 10 and the analysis device 20 are communicatively connected to each other via a communication line NW. The communication line NW may also include a wireless communication transmission path.
[0051] The generation device 10 and the analysis device 20 are respectively information processing devices or computers. For example, the computer may be at least a part of a stationary game console, a portable game console, a television receiver, or a smartphone. For example, the generation device 10 may be a server computer, a desktop computer, a laptop computer, a tablet computer, or a smartphone. For example, the analysis device 20 may be a desktop computer, a laptop computer, a tablet computer, or a smartphone. Furthermore, the generation device 10 and the analysis device 20 may each be composed of multiple devices that are interconnected in a communicable manner.
[0052] like Figure 2 As shown, the generation device 10 includes a processing device 11, a storage device 12, an input device 13, an output device 14, and a communication device 15 interconnected via a bus BU1.
[0053] The processing device 11 controls the storage device 12, the input device 13, the output device 14, and the communication device 15 by executing a program stored in the storage device 12. Thus, the processing device 11 performs the functions described later.
[0054] In this example, the processing device 11 is a CPU (Central Processing Unit). Alternatively, the processing device 11 may replace the CPU or include, in addition to, an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor).
[0055] In this example, storage device 12 includes volatile memory and non-volatile memory. For example, storage device 12 includes at least one of RAM (Random Access Memory), ROM (Read Only Memory), semiconductor memory, organic memory, HDD (Hard Disk Drive), and SSD (Solid State Drive).
[0056] The input device 13 receives external information from the generating device 10. In this example, the input device 13 includes a keyboard and a mouse. Additionally, the input device 13 may also include a microphone. The input device 13 is connected to the imaging device 50.
[0057] The imaging device 50 includes a first imaging unit and a second imaging unit. The first imaging unit captures an image of the object, obtaining a visible light image representing the captured object. The visible light image is an image representing the intensity of visible light reflected by the object for each of a plurality of pixels. In this example, the plurality of pixels included in the visible light image are arranged in a grid pattern.
[0058] In this example, the first imaging unit includes a color camera or an RGB (Red, Green, Blue) camera. Alternatively, the first imaging unit could also be a monochrome camera. In this example, the first imaging unit acquires visible light dynamic image information representing a time series of visible light images.
[0059] The second imaging unit captures an image of the object and obtains a distance image representing the captured object. The distance image is an image representing the distance between a reference plane and the object for each of multiple pixels. In this example, the reference plane is a vertical plane. Furthermore, the reference plane may also be tilted relative to the vertical plane. In this example, the multiple pixels contained in the distance image are arranged in a grid pattern.
[0060] In this example, the second imaging unit includes a TOF (Time of Flight) camera. Alternatively, the second imaging unit can also use a stereo camera instead of a TOF camera to acquire distance images. Furthermore, the second imaging unit can also be equipped with a range sensor that measures the distance between a reference plane and an object for each of multiple pixels arranged in a grid pattern, and use the range sensor to acquire distance images. In this example, the second imaging unit acquires distance dynamic image information representing a time series of distance images.
[0061] The input device 13 receives visible light dynamic image information and distance dynamic image information acquired by the imaging device 50.
[0062] In addition, the shooting device 50 can also be part of the generating device 10.
[0063] The output device 14 outputs information to the external environment of the generating device 10. In this example, the output device 14 includes a display. Alternatively, the output device 14 may also include a speaker.
[0064] In addition, the generating device 10 may also have a touch panel-type display that constitutes both the input device 13 and the output device 14.
[0065] The communication device 15 communicates with external devices of the generation device 10. In this example, the communication device 15 has a card-type or onboard network adapter or network interface.
[0066] like Figure 3 As shown, the analysis device 20 includes a processing device 21, a storage device 22, an input device 23, an output device 24, and a communication device 25 interconnected via a bus BU2. The processing device 21, storage device 22, input device 23, output device 24, and communication device 25 each have the same functions as the processing device 11, storage device 12, input device 13, output device 14, and communication device 15 of the generation device 10.
[0067] The input device 23 is connected to the imaging device 60. The imaging device 60 is configured similarly to the imaging device 50. The input device 23 receives visible light dynamic image information and distance dynamic image information acquired by the imaging device 60.
[0068] In addition, the imaging device 60 can also be part of the analysis device 20.
[0069] (Function)
[0070] The generation device 10 generates a learned model. The learned model is used to analyze the actions of objects with multiple interconnected structures.
[0071] In this example, the subject is a person who is paralyzed on either the left or right side of their body. However, the subject could also be a person who is not paralyzed. Additionally, the subject could be an animal other than a human (e.g., a dog, cat, or horse).
[0072] In this example, the multiple structures consist of 11 structures: the pelvic region, the forearm, upper arm, thigh, lower leg, and foot respectively included on the paralyzed side of the body, and the forearm, upper arm, thigh, lower leg, and foot respectively included on the non-paralyzed side of the body. Furthermore, the multiple structures may also be part of the aforementioned 11 structures. Additionally, the multiple structures may include other structures besides the aforementioned 11 structures.
[0073] In this example, the action is walking. However, the action can also be anything other than walking (e.g., getting up, sitting down, standing up, running, throwing a ball, or swimming).
[0074] like Figure 4 As shown, the functions of the generation device 10 include an angular momentum information acquisition unit 101, a first singular vector acquisition unit 102, a second singular vector acquisition unit 103, and a model generation unit 104.
[0075] The angular momentum information acquisition unit 101 acquires angular momentum information for each of a plurality of learning objects (K in this example) based on visible light dynamic image information and distance dynamic image information acquired by the imaging device 50. K represents an integer greater than 2.
[0076] Angular momentum information represents the time sequence of the angular momentum of each of multiple structures within a specified period of motion. For example, in the case of periodic motion, the period of motion corresponds to one cycle. In this example, the period of motion corresponds to the two-step period of one step to the left and one step to the right. In this example, the period of motion begins at the point when the foot contained in the half of the body on the paralyzed side touches the ground, and ends at the point when the foot touches the ground again after it leaves the ground.
[0077] In this example, the angular momentum information is obtained as follows.
[0078] The angular momentum information acquisition unit 101 estimates the time series of multiple (14 in this example) skeletal structure positions during the action based on visible light dynamic image information and distance dynamic image information.
[0079] Each of the multiple skeletal structural locations corresponds to either end of one of the 11 structures. In this example, the 14 skeletal structural locations are comprised of the wrist, elbow, shoulder, hip, knee, ankle, and toe, which are respectively located on the right side of the body, and the wrist, elbow, shoulder, hip, knee, ankle, and toe, which are respectively located on the left side of the body.
[0080] The hips included in the right half of the body and the hips included in the left half of the body form the two ends of the pelvic region. The wrist and elbow form the two ends of the forearm. The elbow and shoulder form the two ends of the upper arm. The hip and knee form the two ends of the thigh. The knee and ankle form the two ends of the lower leg. The ankle and toes form the two ends of the foot.
[0081] The angular momentum information acquisition unit 101 estimates the time sequence R of the overall center of gravity position of the object during the action based on the time sequence of multiple estimated skeletal structure positions. (k) w (t j The time series V of the velocity of the center of gravity of the entire object. (k) w (t j The time series R of the centroid position of each of the multiple structures. (k) i (t j The time series V of the velocity of the center of gravity of each of the multiple structures. (k) i (t j ), and the time series Ω of the angular velocity of each of the multiple structures.(k) i (t j ).
[0082] For example, the above presumption can be made using the technology described in Non-Patent Document 1 below.
[0083] (Non-patent literature 1) Jun Inagaki et al., “Detection and accuracy of body composite center of gravity using Kinect sensor”, Journal of Clinical Walking Analysis Research, Clinical Walking Analysis Research Association, 2017, Vol. 4, No. 1, pp. 21-27.
[0084] k represents an integer from 1 to K. In this example, the superscript character "(k)" for the variable represents the value for the k-th learning object.
[0085] i represents an integer from 1 to 1. I represents an integer greater than 2. In this example, 1 represents 11. In this example, the subscript character "i" for a variable represents the value of the i-th structure. In this example, the first structure corresponds to the pelvis. The second to sixth structures correspond to the upper arm, forearm, thigh, lower leg, and foot, respectively, on the non-paralyzed side of the body. The seventh to eleventh structures correspond to the foot, lower leg, thigh, forearm, and upper arm, respectively, on the paralyzed side of the body.
[0086] t j This represents time. j represents an integer from 1 to J. J represents an integer greater than 2. In this example, J represents 100. In this example, t1 to t... J These correspond to J time points that divide the action period into equal segments.
[0087] The angular momentum information acquisition unit 101 estimates the time series L of the angular momentum of each of the multiple structures during the action based on the estimated time series set and the following mathematical formulas 1 and 2. (k) i (t j Therefore, the angular momentum information acquisition unit 101 acquires the time series L representing the angular momentum of each of the multiple structures. (k) i (t j ) angular momentum information.
[0088] [Formula 1]
[0089]
[0090] [Formula 2]
[0091]
[0092] B (k) i (t j ) represents the angular momentum vector of the i-th structure. D (k) Let D represent the reference vector, which is a unit vector indicating the direction of rotation that serves as the reference. In this example, the reference vector D... (k) Orthogonal to the frontal plane (in other words, the coronal plane) and in the case of paralysis on the right side of the body, it represents a unit vector having a direction toward the rear of the object (from the ventral side to the dorsal side), while in the case of paralysis on the left side of the body, it represents a unit vector having a direction toward the front of the object (from the dorsal side to the ventral side).
[0093] Therefore, in this example, the angular momentum L (k) i (t j ) represents the angular momentum vector B (k) i (t j The component in ) is the component about a rotation axis orthogonal to the forehead.
[0094] m (k) i M represents the mass of the i-th structure. (k) i This represents the moment of inertia of the i-th structure. For example, mass m (k) i and inertial torque M (k) i It is a predetermined value. Furthermore, the mass m... (k) i and the moment of inertia M (k) i At least one of the values can be a value input by the user of the generating device 10, or a value estimated based on visible light dynamic image information and distance dynamic image information.
[0095] In this example, angular momentum information is obtained in this way.
[0096] The first singular vector acquisition unit 102 generates the first matrix A based on the angular momentum information acquired by the angular momentum information acquisition unit 101. (k) 1. First matrix A (k) As shown in the following mathematical formula 3, the angular momentum information for each of the multiple structures is an element.
[0097] [Formula 3]
[0098]
[0099] The first singular vector acquisition unit 102 is shown in mathematical formulas 4 to 7 below, for the generated first matrix A (k) 1. Perform singular value decomposition to obtain the first singular vector z. (k) 1. The first singular vector z (k) 1 corresponds to the first matrix A (k) The largest first singular value λ among the singular values of 1 (k) 1, and is a singular vector with each corresponding element of the multiple structures (in this case, a right singular vector).
[0100] In this example, the superscript character for the variable, "T", indicates the transpose of the matrix.
[0101] [Formula 4]
[0102]
[0103] [Formula 5]
[0104]
[0105] [Formula 6]
[0106]
[0107] [Formula 7]
[0108]
[0109] The first singular vector acquisition unit 102 acquires a first singular vector z for each of the multiple learning objects. (k) 1.
[0110] The second singular vector acquisition unit 103 is based on the first singular vector z acquired by the first singular vector acquisition unit 102. (k) 1. Generate a second matrix A2 represented by the following mathematical expression 8. As shown in mathematical expression 8, in the second matrix A2, the first singular vector z for each of the multiple learning objects... (k) 1 is an element.
[0111] [Formula 8]
[0112]
[0113] The second singular vector acquisition unit 103, as shown in mathematical formulas 9 to 12 below, performs singular value decomposition on the generated second matrix A2 to obtain the second singular vector p2. The second singular vector p2 corresponds to the second largest second singular value γ2 among the singular values of the second matrix A2, and is a singular vector (in this example, a left singular vector) having an element corresponding to each of the multiple learning objects.
[0114] [Formula 9]
[0115] A2=PΓY T
[0116] [Formula 10]
[0117]
[0118] [Formula 11]
[0119]
[0120] [Number 12]
[0121]
[0122] In this example, the k-th element p of the second singular vector p2 k，2 This represents the inherent characteristic of the action of the k-th learner relative to the average action of the group of k learners. In this example, regarding the second singular vector p2, the greater the difference between the average action of the group of learners and the action of the k-th learner, the stronger the k-th element p of the second singular vector p2. k，2 The larger the size.
[0123] The model generation unit 104 generates a learned model based on the second singular vector p2 obtained by the second singular vector acquisition unit 103.
[0124] In this example, for each of the multiple learning objects, a learned model is generated by learning training data, where the training data includes the angular momentum information L of that learning object. (k) i (t j The element p corresponding to the learning object in the second singular vector p2. k，2 .
[0125] Furthermore, for each of multiple learning objects, the learned model can be generated by learning training data, where the training data includes the angular momentum information L of that learning object. (k) i (t j ) and the element p corresponding to the learning object of the second singular vector p2. k，2 The value obtained by multiplying by the second singular value γ2.
[0126] Alternatively, for each of multiple learning objects, the learned model can be generated by learning training data, where the training data includes the angular momentum information L of that learning object. (k) i (t jThe element p corresponding to the learning object in the second singular vector p2. k，2 Symbolic information. For example, symbolic information in element p. k，2 A positive value represents "+1", and in the element p k，2 A negative value represents "-1".
[0127] Alternatively, for each of multiple learning objects, the learned model can be generated by learning training data, where the training data includes the angular momentum information L of that learning object. (k) i (t j The element p corresponding to the learning object in the second singular vector p2. k，2 The interval information. For example, interval information represents the interval containing the element p from multiple predetermined intervals. k，2 The range of values (in other words, the hierarchy). Furthermore, range information can also represent a range containing the element p from multiple predetermined ranges. k，2 The range of values.
[0128] In addition, the angular momentum information L contained in the training data (k) i (t j It can also correspond to only a part of the action. For example, the generation of a learned model can be performed by cross-validation.
[0129] The learned model includes neural networks. In this example, the learned model includes a recurrent neural network (RNN). In this example, the learned model includes an LSTM (Long Short-Term Memory). In this example, the LSTM is a Deep Bidirectional LSTM (DBLSTM). Alternatively, the learned model can also contain RNNs other than LSTMs.
[0130] The model generation unit 104 sends model information representing the generated learned model to the analysis device 20.
[0131] The analysis device 20 analyzes the action of the object of interest based on the learned model generated by the generation device 10, the visible light dynamic image information obtained for the object of interest, and the distance dynamic image information.
[0132] like Figure 5 As shown, the analysis device 20 includes a model information storage unit 201, an angular momentum information acquisition unit 202, a feature information acquisition unit 203, and a feature information output unit 204.
[0133] The model information storage unit 201 receives model information generated by the generation device 10 from the generation device 10 and stores the received model information in the storage device 22.
[0134] The angular momentum information acquisition unit 202 acquires angular momentum information for the object of interest based on visible light dynamic image information and distance dynamic image information acquired by the imaging device 60. The angular momentum information acquisition unit 202 acquires angular momentum information using the same method as the angular momentum information acquisition unit 101.
[0135] The feature information acquisition unit 203 acquires feature information for the object of interest based on the learned model represented by the model information stored in the model information storage unit 201 and the angular momentum information acquired by the angular momentum information acquisition unit 202 for the object of interest. The feature information represents the characteristics of the action of the object of interest.
[0136] In this example, the feature information represents the value of the element corresponding to the object of interest among the multiple learning objects, which is presumed to be among the learning objects that form the basis for generating the learned model, taken from the second singular vector p2. Therefore, in this example, the feature information corresponds to the information of the element corresponding to the object of interest among the multiple objects, based on the second singular vector p2. Furthermore, the feature information is based on the first singular vector z. (k) 1. The generated information. Therefore, in this example, the feature information corresponds to the information based on the first singular vector z. (k) Information 1.
[0137] The feature information output unit 204 outputs the feature information acquired by the feature information acquisition unit 203 via the output device 24 (in this example, it is displayed on the display).
[0138] Furthermore, the feature information output unit 204 can also output output information corresponding to the feature information, in addition to or in place of the feature information. For example, the output information can also represent the progress of rehabilitation training or the normality of movement. In this case, the output information can indicate that the smaller the value represented by the feature information, the higher the progress of rehabilitation training or the higher the normality of movement.
[0139] In addition to its own functions, the analysis device 20 may also have the functions of the generation device 10. In this case, the analysis system 1 may not have the generation device 10 and the imaging device 50.
[0140] (action)
[0141] Next, refer to Figure 6 and Figure 7 The operation of analysis system 1 is explained.
[0142] In this example, the generation device 10 performs [operations] to generate the learned model. Figure 6 The processing shown.
[0143] Specifically, the generation device 10 performs a first loop process (steps S101 to S104) in which each of the K learning objects is used as a processing object in turn.
[0144] In the first loop processing, the generation device 10 obtains angular momentum information for the learning object being processed, based on the visible light dynamic image information and distance dynamic image information obtained by the imaging device 50 (step S102). Next, the generation device 10 obtains a first singular vector z based on the obtained angular momentum information. (k) 1 (Step S103).
[0145] Then, after performing the first loop processing on all learning objects, the generation device 10, based on the first singular vector z obtained for each of the K learning objects, (k) 1. Obtain the second singular vector p2 (step S105).
[0146] Next, the generation device 10 performs a second loop process (steps S106 to S108) in which each of the K learning objects is used as a processing object in turn.
[0147] In the second cycle of processing, the generation device 10 learns, for the learning object being processed, the angular momentum information L obtained in step S102. (k) i (t j ) and the element p corresponding to the learning object of the second singular vector p2 obtained in step S105. k，2 Training data (step S107).
[0148] Then, after performing the second loop processing on all learning objects, the generation device 10 sends the model information representing the generated learned model to the analysis device 20 (step S109).
[0149] Thus, the generating device 10 performs... Figure 6 The processing shown.
[0150] In addition, the analysis device 20 receives model information from the generation device 10 by performing a process not shown, and stores the received model information in the storage device 22.
[0151] In this example, the analysis device 20 performs the following steps to analyze the action of the object of interest based on the learned model generated by the generation device 10 and the visible light dynamic image information and distance dynamic image information obtained for the object of interest. Figure 7The processing shown.
[0152] Specifically, the analysis device 20 obtains angular momentum information for the object of interest based on the visible light dynamic image information and distance dynamic image information obtained by the imaging device 60 (step S201). Next, the analysis device 20 obtains feature information for the object of interest based on the learned model represented by the model information stored in the storage device 22 and the angular momentum information obtained in step S201 (step S202).
[0153] Next, the analysis device 20 outputs the feature information obtained in step S202 via the output device 24 (step S203).
[0154] Thus, the analysis device 20 performs... Figure 7 The processing shown.
[0155] As explained above, the analysis device 20 of the first embodiment analyzes the actions of an object having multiple interconnected structures.
[0156] The analysis device 20 includes an angular momentum information acquisition unit 202 and a feature information acquisition unit 203. The angular momentum information acquisition unit 202 acquires angular momentum information representing a time series of the angular momentum of each of the plurality of structures. The feature information acquisition unit 203 acquires feature information based on a first singular vector, which is information representing the characteristics of the object's action. This first singular vector corresponds to the largest singular value among the singular values of a first matrix that uses the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures.
[0157] Therefore, the contribution of each of the multiple structures to the movement is reflected in the first singular vector. Thus, the contribution of each of the multiple structures to the movement can be accurately reflected in the feature information. As a result, the contribution of structures to the movement can be accurately determined between structures contained in the left and right halves of the body. Therefore, for example, the progress of rehabilitation training can be appropriately evaluated.
[0158] Furthermore, in the analysis apparatus 20 of the first embodiment, the feature information is information about the elements corresponding to the objects of interest among a plurality of objects based on the second singular vector, which corresponds to the second largest second singular value in the singular values of a second matrix that takes the first singular vector for each of the plurality of objects as an element, and has an element corresponding to each of the plurality of objects.
[0159] Therefore, in an object group consisting of multiple objects, the inherent characteristics of each object's actions are reflected in the second singular vector. Thus, within the object group, the inherent characteristics of the actions of the object of interest can be accurately reflected in the feature information. As a result, for example, the inherent characteristics of the actions of the object of interest relative to the average action of the object group can be known with high precision.
[0160] Furthermore, in the analysis apparatus 20 of the first embodiment, the feature information acquisition unit 203 acquires feature information based on the learned model and the angular momentum information of the object of interest. The learned model is generated in the following manner: acquiring a second singular vector that corresponds to the second largest second singular value among the singular values of a second matrix that takes the first singular vector as an element for each of the plurality of learning objects, and having an element corresponding to each of the plurality of learning objects; and learning training data for each of the plurality of learning objects that includes the angular momentum information of that learning object and information about the element corresponding to that learning object based on the second singular vector.
[0161] Therefore, in an object group consisting of multiple objects, the inherent characteristics of each object's actions are reflected in the second singular vector. Thus, within the object group, the inherent characteristics of the actions of the object of interest can be accurately reflected in the feature information. As a result, for example, the inherent characteristics of the actions of the object of interest relative to the average action of the object group can be known with high precision.
[0162] In addition, compared to calculating the second singular vector for a group of objects containing the object of interest, feature information for the object of interest can be obtained quickly.
[0163] Furthermore, in the analysis device 20 of the first embodiment, the object is a person with paralysis on the left or right side of the body. The multiple structures include: a pelvic region, a forearm, upper arm, thigh, lower leg and foot respectively included in the paralyzed side of the body, and a forearm, upper arm, thigh, lower leg and foot respectively included in the non-paralyzed side of the body.
[0164] Therefore, the elements of the first matrix correspond to structures included in the paralyzed half of the body and structures included in the non-paralyzed half of the body, respectively. As a result, it is possible to know with high precision the contribution of structures to movement between structures included in the paralyzed half of the body and structures included in the non-paralyzed half of the body. Thus, for example, it is possible to appropriately evaluate the progress of rehabilitation training.
[0165] Furthermore, in the analysis device 20 of the first embodiment, the action is walking. The angular momentum information corresponds to the duration of two steps, one step to the left and one step to the right.
[0166] In healthy individuals, it is expected that walking involves structures in the left and right halves of the body contributing equally to movement. Therefore, according to the analysis device 20, the degree of appropriate walking can be evaluated with high precision. Thus, for example, the progress of rehabilitation training can be appropriately evaluated.
[0167] In addition, the generation device 10 of the first embodiment generates a learned model for analyzing the actions of an object having multiple interconnected structures.
[0168] The generation device 10 includes an angular momentum information acquisition unit 101, a first singular vector acquisition unit 102, a second singular vector acquisition unit 103, and a model generation unit 104.
[0169] The angular momentum information acquisition unit 101 acquires angular momentum information of a time series representing the angular momentum of each of the multiple learning objects for each of the multiple structures.
[0170] The first singular vector acquisition unit 102 acquires a first singular vector for each of the plurality of learning objects. The first singular vector corresponds to the largest first singular value among the singular values of a first matrix that takes the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures.
[0171] The second singular vector acquisition unit 103 acquires a second singular vector, which corresponds to the second largest singular value among the singular values of a second matrix that takes the first singular vector for each of the plurality of learning objects as an element, and has an element corresponding to each of the plurality of learning objects.
[0172] The model generation unit 104 generates a learned model for each of the multiple learning objects by learning training data, which includes the angular momentum information of the learning object and information of the elements corresponding to the learning object based on the second singular vector.
[0173] Therefore, in a learning object group consisting of multiple learning objects, the inherent features of each learning object's actions are reflected in the second singular vector. Thus, the inherent features of each learning object's actions relative to the average action of the learning object group can be accurately reflected in the learned model. As a result, by obtaining feature information based on the learned model, the inherent features of the actions of the object of interest can be accurately reflected in the feature information. Furthermore, compared to calculating the second singular vector for a learning object group containing the object of interest, feature information specific to the object of interest can be obtained more quickly.
[0174] Figure 8 and Figure 9This is a chart showing an example of the result of the singular value decomposition of the first matrix A1 calculated by the analysis system 1 of the first embodiment.
[0175] Figure 8 (A) represents an example of the right singular vector (the first singular vector in this example) z1 corresponding to the largest singular value (the first singular value in this example) λ1 among the singular values of the first matrix A1. Figure 8 (B) represents an example of vector λ1u1, which is obtained by multiplying the largest singular value (the first singular value in this example) λ1 of the singular values of the first matrix A1 by the left singular vector u1 corresponding to the singular value λ1.
[0176] Figure 9 (A) represents an example of the right singular vector z2 corresponding to the second largest singular value λ2 among the singular values of the first matrix A1. Figure 9 (B) represents an example of vector λ2u2, which is obtained by multiplying the second largest singular value λ2 in the singular values of the first matrix A1 by the left singular vector u2 corresponding to the singular value λ2.
[0177] Figure 10 and Figure 11 This is a diagram showing an example of the result of the singular value decomposition of the second matrix A2 calculated by the analysis system 1 of the first embodiment.
[0178] Figure 10 (A) represents an example of the right singular vector y1 corresponding to the largest singular value γ1 among the singular values of the second matrix A2. Figure 10 (B) represents an example of a vector γ1p1, which is obtained by multiplying the largest singular value γ1 in the singular values of the second matrix A2 by the left singular vector p1 corresponding to that singular value γ1.
[0179] Figure 11 (A) represents an example of the right singular vector y2 corresponding to the second largest singular value (the second singular value in this example) γ2 among the singular values of the second matrix A2. Figure 11 (B) represents an example of vector γ2p2, which is obtained by multiplying the second largest singular value (the second singular value in this example) γ2 of the singular values of the second matrix A2 by the left singular vector (the second singular vector in this example) p2 corresponding to that singular value γ2.
[0180] Furthermore, the generation apparatus 10 of the first embodiment obtains this information by inputting visible light dynamic image information and distance dynamic image information from the imaging device 50. Additionally, the generation apparatus 10 of the modified first embodiment can also obtain this information by receiving visible light dynamic image information and distance dynamic image information from other devices communicatively connected to the generation apparatus 10. Furthermore, the generation apparatus 10 of the modified first embodiment can also obtain this information by reading visible light dynamic image information and distance dynamic image information from a recording medium.
[0181] Furthermore, the analysis device 20 of the first embodiment acquires this information by inputting visible light dynamic image information and distance dynamic image information from the imaging device 60. Additionally, the analysis device 20 of the modified first embodiment can also acquire this information by receiving visible light dynamic image information and distance dynamic image information from another device communicatively connected to the analysis device 20. Furthermore, the analysis device 20 of the modified first embodiment can also acquire this information by reading visible light dynamic image information and distance dynamic image information from a recording medium.
[0182] Furthermore, in the first embodiment, the generation apparatus 10 and the analysis apparatus 20 obtain angular momentum information based on both visible light dynamic image information and range dynamic image information. Additionally, in a modified embodiment of the first embodiment, the generation apparatus 10 and the analysis apparatus 20 may also obtain angular momentum information based on either visible light dynamic image information or range dynamic image information.
[0183] Furthermore, in the modified embodiment of the first embodiment, the generation apparatus 10 and the analysis apparatus 20 may replace the visible light dynamic image information and the distance dynamic image information, respectively, or, in addition to the visible light dynamic image information and the distance dynamic image information, angular momentum information may be obtained based on detection information. For example, the detection information may be information obtained using sensors or markers attached to each structure.
[0184] Furthermore, the analysis apparatus 20 of the first embodiment obtains feature information for the object of interest based on the learned model generated by the generation apparatus 10. Alternatively, the analysis apparatus 20 of a variant of the first embodiment may obtain feature information for the object of interest without using the learned model. In this case, the analysis apparatus 20 may obtain first singular vectors for multiple learned objects and the object of interest respectively, and obtain second singular vectors based on the obtained first singular vectors, thereby obtaining the elements of the second singular vectors for the object of interest as feature information for the object of interest. Alternatively, the analysis apparatus 20 of a variant of the first embodiment may also obtain the first singular vector for the object of interest based on angular momentum information, and use the obtained first singular vector as feature information.
[0185] Furthermore, the present invention is not limited to the embodiments described above. For example, various modifications that can be understood by those skilled in the art can be made to the above embodiments without departing from the spirit of the present invention.
[0186] Explanation of reference numerals in the attached figures
[0187] 1 Analysis System
[0188] 10 Generating Device
[0189] 11 processing devices
[0190] 12 storage devices
[0191] 13 Input Devices
[0192] 14 Output Devices
[0193] 15 communication devices
[0194] 101 Angular Momentum Information Acquisition Department
[0195] 102 First Singular Vector Acquisition Part
[0196] 103 Second Singular Vector Acquisition Part
[0197] Model Generation Department 104
[0198] 20 Analytical Devices
[0199] 21 processing unit
[0200] 22 storage devices
[0201] 23 Input Devices
[0202] 24 output devices
[0203] 25 communication devices
[0204] 201 Model Information Storage Department
[0205] 202 Angular Momentum Information Acquisition Department
[0206] 203 Feature Information Acquisition Department
[0207] 204 Feature Information Output Section
[0208] 50 shooting devices
[0209] 60 shooting device
[0210] BU1 bus
[0211] BU2 bus
[0212] NW communication lines.
Claims
1. An analysis apparatus for analyzing the actions of an object having multiple interconnected structures, characterized in that, The analytical device includes: An angular momentum information acquisition unit acquires angular momentum information representing a time series of the angular momentum of each of the plurality of structures; and The feature information acquisition unit acquires feature information based on a first singular vector, which represents the characteristics of the action. The first singular vector corresponds to the largest singular value among the singular values of a first matrix that takes the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures. The feature information is information about the elements corresponding to the objects of interest among a plurality of objects based on the second singular vector, the second singular vector corresponding to the second largest second singular value in the singular values of a second matrix that takes the first singular vector as an element for each of the plurality of objects, and has an element corresponding to each of the plurality of objects.
2. The analytical apparatus according to claim 1, characterized in that, The feature information acquisition unit obtains the feature information based on the learned model and the angular momentum information of the object of interest. The learned model is generated by obtaining a second singular value that corresponds to the second largest singular value in the singular values of a second matrix that takes the first singular vector as an element for each of the plurality of learning objects, and has a second singular vector with an element corresponding to each of the plurality of learning objects; and for each of the plurality of learning objects, learning training data that includes the angular momentum information of the learning object and information based on the second singular vector corresponding to the learning object.
3. The analytical apparatus according to claim 1, characterized in that, The subjects are people who are paralyzed on the left or right side of their body. The plurality of structures include: a pelvic region, a forearm, upper arm, thigh, lower leg and foot respectively contained in the paralyzed half of the body, and a forearm, upper arm, thigh, lower leg and foot respectively contained in the non-paralyzed half of the body.
4. The analytical apparatus according to claim 1, characterized in that, The action described is walking. The angular momentum information corresponds to the period of two steps, one step to the left and one step to the right.
5. An analysis method for analyzing the actions of an object having multiple interconnected structures, characterized in that, The analytical method includes the following steps: Obtain angular momentum information representing the time series of the angular momentum of each of the plurality of structures; and The feature information is obtained based on a first singular vector and represents the characteristics of the action. The first singular vector corresponds to the largest singular value among the singular values of a first matrix that takes the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures. The feature information is information about the elements corresponding to the objects of interest among a plurality of objects based on the second singular vector, the second singular vector corresponding to the second largest second singular value in the singular values of a second matrix that takes the first singular vector as an element for each of the plurality of objects, and has an element corresponding to each of the plurality of objects.
6. A generation apparatus for generating a learned model, the learned model being used to analyze the actions of an object having multiple interconnected structures, characterized in that, The generating apparatus includes: An angular momentum information acquisition unit acquires, for each of the plurality of learning objects, time-series angular momentum information representing the angular momentum of each of the plurality of structures; The first singular vector acquisition unit acquires a first singular vector for each of the plurality of learning objects. The first singular vector corresponds to the largest first singular value among the singular values of a first matrix that takes the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures. The second singular vector acquisition unit acquires a second singular vector, which corresponds to the second largest singular value among the singular values of a second matrix that takes the first singular vector for each of the plurality of learning objects as an element, and has an element corresponding to each of the plurality of learning objects. as well as The model generation unit generates the learned model for each of the plurality of learning objects by learning training data, wherein the training data includes the angular momentum information of the learning object and information of the element corresponding to the learning object based on the second singular vector.
7. A generation method for generating a learned model, said learned model being used to analyze the actions of an object having multiple interconnected structures, characterized in that, The generation method includes the following steps: For each of the multiple learning objects, obtain the time-series angular momentum information representing the angular momentum of each of the multiple structures; For each of the plurality of learning objects, a first singular vector is obtained, which corresponds to the largest first singular value among the singular values of a first matrix that takes the angular momentum information for each of the plurality of structures as elements, and has elements corresponding to each of the plurality of structures. Obtain a second singular vector, which corresponds to the second largest singular value among the singular values of a second matrix that takes the first singular vector for each of the plurality of learning objects as an element, and has an element corresponding to each of the plurality of learning objects; as well as For each of the plurality of learning objects, the learned model is generated by learning training data, wherein the training data includes the angular momentum information of the learning object and information of the element corresponding to the learning object based on the second singular vector.