Physical function estimation system, physical function estimation method, and program

The system estimates gait features from video to evaluate physical functions, identifying underlying causes of issues like balance and muscle strength, enhancing intervention effectiveness.

JP7591743B2Active Publication Date: 2024-11-29PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD

Patent Information

Application Number
JP2023524062
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-05-27
Filing Date
2022-03-29
Publication Date
2024-11-29
Estimated Expiration
2042-03-29

AI Technical Summary

Technical Problem

Existing methods for evaluating physical functions, such as fall risk, can determine the presence of a problem but fail to identify its underlying cause.

Method used

A system and method that estimates gait feature amounts from video data to evaluate physical functions, using a control device to analyze skeletal models and estimate scores for multiple evaluation items, including balance, flexibility, and muscle strength, to determine the cause of physical function issues.

Benefits of technology

Enables the identification of the cause of physical function problems, allowing for targeted interventions and improving the accuracy of evaluations without requiring specialized equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591743000001
    Figure 0007591743000001
  • Figure 0007591743000002
    Figure 0007591743000002
  • Figure 0007591743000003
    Figure 0007591743000003
Patent Text Reader

Abstract

A physical-ability estimation system (1) is provided with: an analysis unit (32) that estimates gait features of a user (U) from a video in which the user (U) was captured while walking; and an estimation unit (33) that estimates, on the basis of the gait features, individual evaluation results concerning two or more evaluation items for evaluating physical ability.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a body function estimation system, a body function estimation method, and a program. [Background technology]

[0002] Conventionally, methods for evaluating or determining physical functions such as the risk of falling based on image data have been proposed. For example, Patent Document 1 discloses a technology for analyzing image data to calculate walking parameters and compute information related to the evaluation of walking movements. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-140591 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method described in Patent Document 1 can evaluate physical functions (for example, evaluating walking movements), but does not identify the cause of a problem in physical function, if any.

[0005] Therefore, the present invention provides a physical function estimation system, a physical function estimation method, and a program that are capable of estimating the cause of a problem in a physical function when such a problem exists. [Means for solving the problem]

[0006] A physical function estimation system according to one embodiment of the present invention includes a first estimation unit that estimates gait features of a user from a video captured of the user walking, and a second estimation unit that estimates evaluation results of two or more evaluation items for evaluating physical function based on the gait features.

[0007] A physical function estimating method according to one embodiment of the present invention estimates gait features of a user from a video captured of the user walking, and estimates evaluation results for two or more evaluation items for evaluating physical function based on the gait features.

[0008] A program according to one aspect of the present invention is a program for causing a computer to execute the above-described physical function estimation method. Effect of the Invention

[0009] According to one aspect of the present invention, it is possible to realize a physical function estimation system or the like that is capable of estimating the cause of a problem in a physical function when such a problem occurs. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a body function estimating system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the body function estimating system according to the embodiment. [Diagram 3] FIG. 3 is a flowchart showing the operation of the body function estimation system according to the embodiment. [Figure 4] FIG. 4 is a diagram showing the correlation in skeletal estimation between motion capture and a video according to the embodiment. [Diagram 5] FIG. 5 is a diagram showing a subsystem that details the balance ability. [Figure 6] FIG. 6 is a diagram showing the correlation in hip joint angle estimation between motion capture and video according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating the correlation in the estimation of the knee joint angle between motion capture and a video according to the embodiment. [Figure 8] FIG. 8 is a diagram showing the accuracy rate when the normal walking range of motion is estimated from a video according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating the correlation in the estimation of FRT between motion capture and a video according to the embodiment. [Figure 10] FIG. 10 is a diagram showing the correlation in estimating the time of standing on one leg with eyes open between the motion capture and a video according to the embodiment. [Figure 11] FIG. 11 is a diagram showing the correlation in estimating the proportion of one-legged standing with eyes open / closed between motion capture and video according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating the correlation in TUG estimation between motion capture and video according to the embodiment. [Figure 13] FIG. 13 is a diagram showing an evaluation result in estimation of a healthy subject or MCI (gait feature amount: 10 items) using motion capture and video according to the embodiment. [Figure 14] FIG. 14 is a diagram showing an evaluation result in estimation of a healthy subject or MCI (gait feature amount: walking speed only) using motion capture and video according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiment will be specifically described with reference to the drawings.

[0012] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, components, component arrangement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims are described as optional components.

[0013] In addition, each figure is a schematic diagram and is not necessarily illustrated precisely. Therefore, for example, the scales in each figure do not necessarily match. In addition, in each figure, substantially the same configurations are given the same reference numerals, and duplicated explanations are omitted or simplified.

[0014] Furthermore, in this specification, terms indicating relationships between elements such as the same, as well as numerical values ​​and numerical ranges, are not expressions that only indicate a strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about a few percent (e.g., about 5%).

[0015] (Embodiment) Hereinafter, the body function estimating system according to the present embodiment will be described with reference to FIGS.

[0016] [1. Configuration of the physical function estimation system] First, the configuration of a body function estimating system according to the present embodiment will be described with reference to Figures 1 and 2. Figure 1 is a diagram showing a schematic configuration of a body function estimating system 1 according to the present embodiment.

[0017] As shown in FIG. 1, the physical function estimation system 1 includes an imaging device 10, a hub 20, a control device 30, a router 40, and a terminal device 50.

[0018] The imaging device 10 captures an image of a walking user U. The imaging device 10 captures a video (image) by capturing an image of the user U walking a predetermined distance, for example. The predetermined distance is, for example, 4 m or more, but is not limited to this.

[0019] The imaging device 10 is installed in a building such as a nursing home, a hospital, an office, or a public institution, but may be installed in a home, for example. Specifically, the imaging device 10 is a security camera (monitoring camera), but may also be a door phone (intercom) camera. The number of imaging devices 10 provided in the physical function estimation system 1 is not particularly limited, and multiple imaging devices 10 may be provided. The imaging devices 10 may be multiple cameras provided in a motion capture system.

[0020] The imaging device 10 may be a USB (Universal Serial Bus) camera, a camera included in the terminal device 50 (for example, a camera mounted on a tablet or a camera mounted on a smartphone), or the like. The imaging device 10 may be any camera capable of capturing moving images. The imaging device 10 may be a fixed camera or a portable camera. The physical function estimation system 1 may include a sensor capable of acquiring gait features in place of the imaging device 10 or in addition to the imaging device 10. The sensor capable of acquiring gait features is, for example, a distance sensor, a Wifi sensor, an acceleration sensor, or the like, but is not limited thereto.

[0021] A gait is a state of physical movement when a person is walking, and gait features indicating a gait include a walking cycle, walking speed, stride length, step length, step width, etc. The gait features may further include values ​​obtained by dividing each of the stride length, step length, and step width by height, etc. The gait features may also include values ​​obtained by dividing the stride length by the step length, etc.

[0022] The hub 20 connects the imaging device 10, the control device 30, and the router 40, and relays communications between them.

[0023] The control device 30 estimates the physical function of the user U based on the video captured by the imaging device 10. The control device 30 estimates the gait feature of the user U based on the video captured by the imaging device 10, and estimates the scores of two or more evaluation items for evaluating the physical function based on the estimated gait feature. The two or more evaluation items are evaluation items related to the functional level of the body, although details of the two or more evaluation items will be described later. The two or more evaluation items include, for example, an evaluation item for at least one of the balance system, the flexibility system, and the muscle strength system. In the following, an example will be described in which each of the two or more evaluation items is an evaluation item for the balance system. Note that the score is, for example, a numerical value from 0 to 100, but is not limited thereto. The score may be a normalized value for each evaluation item.

[0024] The router 40 relays communication between the network connected by the hub 20 and the terminal device 50 .

[0025] The terminal device 50 has a display unit 51 and a receiving unit (not shown), and displays the estimation results of physical functions to the user U, etc., or receives input of predetermined information from the user U, etc. The predetermined information may be, for example, biometric information of the user U. The biometric information includes at least one of the sex, height, weight, and age of the user U. It can also be said that the biometric information includes physical information of the user U, such as the height and weight.

[0026] The display unit 51 is an example of a presentation unit that is realized by, for example, a liquid crystal panel, an organic EL (Electro Luminescence) panel, or the like, and presents predetermined information to the user. The presentation unit is not limited to the display unit 51, and may be realized by a sound output unit such as a speaker. The reception unit is a touch panel, a button, or the like, but may have a configuration that allows information to be obtained from the user by voice, gestures, or the like.

[0027] The terminal device 50 is a mobile terminal such as a tablet terminal or a smartphone, but may also be a stationary terminal.

[0028] As described above, the physical function estimation system 1 estimates gait features based on a video captured of a walking user U, and estimates scores for two or more evaluation items for evaluating physical function based on the estimated gait features. The physical function estimation system 1 estimates scores for evaluation items related to a physical function level from the gait features, that is, estimates the frailty state of the user U (e.g., an elderly person) from the gait. The physical function estimation system 1 can estimate the physical function of, for example, a healthy elderly person whose frailty state is difficult to see.

[0029] Note that communication between the devices included in the body function estimation system 1 is not limited to communication via the hub 20, the router 40, etc., and the communication method between the devices is not particularly limited.

[0030] Fig. 2 is a block diagram showing the functional configuration of the body function estimation system 1 according to this embodiment. Note that in Fig. 2, the hub 20 and the router 40 shown in Fig. 1 are omitted.

[0031] As shown in FIG. 2, the control device 30 includes an acquisition unit 31, an analysis unit 32, an estimation unit 33, a proposal unit , and an output unit .

[0032] The acquisition unit 31 acquires a moving image of a walking user U from the imaging device 10. The acquisition unit 31 is configured to include a communication circuit.

[0033] The analysis unit 32 estimates gait feature amounts of the user U based on the video acquired by the acquisition unit 31. Specifically, the analysis unit 32 estimates a skeleton of the user U based on the video, and estimates the gait feature amounts of the user U based on the estimated skeleton (skeleton estimation data).

[0034] The analysis unit 32 estimates the gait feature of the user U from a video of the user U walking. For example, the analysis unit 32 estimates the skeleton of the user U from the user U shown in the video, and estimates the gait feature based on the estimated skeleton. For example, the analysis unit 32 identifies a three-dimensional skeleton model of the user U shown in the video (three-dimensional coordinate data of each joint such as a person's knee joint, hip joint, ankle, etc.) using an existing algorithm, and identifies (estimates) the gait feature using a position change of each skeleton point of the three-dimensional skeleton model. The analysis unit 32 may estimate a two-dimensional skeleton model of the user U shown in the video. The analysis unit 32 is an example of a first estimation unit.

[0035] The existing algorithm is an algorithm used in a machine learning model. In this embodiment, the existing algorithm is an existing skeleton estimation model, for example, a trained model in which a CNN (Convolutional Neural Network) is given time series. The skeleton estimation model is trained using, for example, the data set "Human 3.6". In addition, the skeleton estimation model is trained so that the skeleton can be detected with an error of, for example, about 25 mm to 30 mm. The skeleton estimation model may be trained using, for example, a video of 27 frames.

[0036] The analysis unit 32 may estimate the gait feature amount based on a video (e.g., a plurality of distance images) acquired by motion capture. In this case, the user U wears a marker (e.g., a reflective member), and the video is a video capturing the user U walking while wearing the marker. The analysis unit 32 may estimate the gait feature amount based on the marker (e.g., a time-series change in the position of the marker) captured in the video.

[0037] The estimation unit 33 estimates evaluation results of two or more evaluation items for evaluating physical functions based on the gait feature amount estimated by the analysis unit 32. The estimation unit 33 estimates the scores of the two or more evaluation items of the user U based on, for example, the gait feature amount estimated by the analysis unit 32. Specifically, the estimation unit 33 acquires the output of a trained model obtained by inputting the gait feature amount into the trained model as the scores of the two or more evaluation items. The estimation unit 33 may further estimate the scores of the two or more evaluation items of the user U based on at least one of the biometric information (biometric feature amount) of the user U and the environmental information (environmental feature amount) indicating the walking environment of the user U. The estimation unit 33 is an example of a second estimation unit.

[0038] The suggestion unit 34 determines the content of a suggestion to be made to the user U in order to suppress the decline in physical function, based on the estimation result of the estimation unit 33. The suggestion unit 34 determines the content of the suggestion according to the scores of each of the two or more evaluation items. The content of the suggestion includes exercise, diet, medical examination, etc. The suggestion unit 34 determines the content of the suggestion based on a table in which the scores of each of the two or more evaluation items are associated with the content of the suggestion. This enables the physical function estimation system 1 to suggest to the user U a method with high intervention efficiency.

[0039] The output unit 35 outputs the estimation result of the estimation unit 33, the proposal content of the proposal unit 34, etc. to the terminal device 50. The output unit 35 outputs, for example, information to be displayed on the display unit 51 of the terminal device 50. The output unit 35 is configured to include a communication circuit.

[0040] [2. Operation of the physical function estimation system] Next, the operation of the physical function estimating system 1 configured as above will be described with reference to Figures 3 to 5. Figure 3 is a flowchart showing the operation of the physical function estimating system 1 according to this embodiment.

[0041] 3, the terminal device 50 acquires biometric information and environmental information via the reception unit (S11). The terminal device 50 acquires biometric information from, for example, a user U via the reception unit. The terminal device 50 also acquires environmental information such as the state of the walking surface (e.g., road surface) on which the user U walks and the user U's belongings (e.g., whether or not he / she has a bag).

[0042] Next, the terminal device 50 outputs the acquired biological information and environmental information to the control device 30, and the control device 30 acquires the biological information and environmental information via the acquisition unit 31 (S12).

[0043] Next, the terminal device 50 acquires an operation instruction for the imaging device 10 from the user U via the reception unit (S13). The operation instruction includes, for example, an instruction to start imaging.

[0044] Next, the terminal device 50 outputs the acquired operation instruction to the imaging device 10, and the imaging device 10 acquires the operation instruction (S14).

[0045] Next, the imaging device 10 captures an image of the walking user U (S15). The imaging device 10 captures an image of the user U walking, for example, 4 meters or more. The imaging device 10 captures an image of the user U so that, for example, the entire body of the user U is captured.

[0046] Next, the imaging device 10 outputs the moving image acquired by imaging to the control device 30, and the control device 30 acquires the moving image via the acquisition unit 31 (S16). The control device 30 stores the acquired moving image in a storage unit (not shown) (S17).

[0047] Next, the analysis unit 32 of the control device 30 estimates the skeleton of the user U based on the acquired video (S18). The control device 30 inputs the acquired video into an existing skeleton estimation model, and acquires the output obtained as the position of each skeleton point of the three-dimensional skeleton model. The analysis unit 32 may further acquire a two-dimensional skeleton model of the user U shown in the video (two-dimensional coordinate data of each joint such as a person's knee joint, hip joint, ankle, etc.) using an existing algorithm.

[0048] Next, the analysis unit 32 of the control device 30 extracts gait feature amounts of the user U using the position change of each skeleton point of the three-dimensional skeletal model (S19). It can also be said that the analysis unit 32 estimates the gait feature amounts of the user U using the position change of each skeleton point of the three-dimensional skeletal model.

[0049] Fig. 4 is a diagram showing the correlation in skeletal estimation between motion capture and video according to this embodiment. In Fig. 4, points for 108 people are plotted with one point representing the gait feature of user U acquired by motion capture and the other point representing the gait feature of user U acquired based on a three-dimensional skeletal model from a video, and the correlation coefficient ("R" in the figure) is shown. In the following, it is determined that there is a correlation when the correlation coefficient is 0.5 or more.

[0050] 4A shows the correlation between the walking speed of the user U acquired by motion capture and the walking speed of the user U based on the three-dimensional skeletal model. The correlation coefficient R is 0.86, and there is a correlation between the walking speed acquired by motion capture and the walking speed acquired based on the three-dimensional skeletal model.

[0051] 4(b) shows the correlation between the stride length of user U acquired by motion capture and the stride length of user U based on a three-dimensional skeletal model. The correlation coefficient R is 0.74, and there is a correlation between the stride length acquired by motion capture and the stride length acquired based on the three-dimensional skeletal model.

[0052] Fig. 4(c) shows the correlation between the step width of user U acquired by motion capture and the step width of user U based on the three-dimensional skeletal model. The correlation coefficient R is 0.48, and the correlation between the step width acquired by motion capture and the step width acquired based on the three-dimensional skeletal model is weak.

[0053] FIG. 4(d) shows the correlation between the step length of user U acquired by motion capture and the step length of user U based on a two-dimensional skeletal model. The correlation coefficient R is 0.85, and there is a correlation between the step length acquired by motion capture and the step length acquired based on a two-dimensional skeletal model. For this reason, when estimating the stride length, it is advisable to use a two-dimensional skeletal model.

[0054] In this way, since there is a correlation between gait feature amounts based on motion capture and gait feature amounts based on a video, if gait feature amounts acquired by motion capture are considered to be correct, it can be said that gait feature amounts can be estimated based on a video.

[0055] In step S19, the analysis unit 32 may further calculate information indicating a variation in the estimated gait feature amounts. The information indicating the variation is, for example, a standard deviation. The analysis unit 32 calculates the information indicating the variation based on time-series data of the values ​​of the gait feature amounts estimated based on the video. The analysis unit 32 calculates the variation of each gait feature amount. The information indicating the variation is also included in the gait feature amounts. In other words, the gait feature amounts in this specification may include information indicating the variation in the basic feature amounts of a gait (variation feature amounts) in addition to basic feature amounts of a gait such as a walking cycle.

[0056] The analysis unit 32 outputs the estimated gait feature amount of the user U to the estimation unit 33.

[0057] 3 again, next, the estimation unit 33 of the control device 30 estimates scores of evaluation items of the physical functions of the user U using the gait feature amount (S20). The estimation unit 33 estimates the score for each evaluation item using a trained model generated for each evaluation item.

[0058] Here, the evaluation items estimated by the estimation unit 33 will be described with reference to FIG. 5. FIG. 5 is a diagram showing a subsystem that details balance ability. FIG. 5 shows the relationship between the items of the subsystem and the evaluation method for each item. Balance ability is involved in falls, which are an important factor in the need for nursing care and assistance. By evaluating the balance ability shown in FIG. 5, effective intervention for fall prevention becomes possible, and it is expected to have an effect on extending healthy life expectancy. Also, the dashed frame in FIG. 5 indicates items that can be estimated from gait features.

[0059] The six items shown in Figure 5 represent the six elements proposed by Dr Horak et al. (Source: Horak FB., Wrisley DM., et al.: "The balance evaluation systems test (BESTest) to differentiate balance deficits. Phys Ther, 89.5: 484-498, 2009").

[0060] As shown in Figure 5, the items of the subsystem that detail balance ability (an example of the first item) include biomechanical constraints, stability limits and verticality, posture change, predictive posture control, sensory function, and walking stability. The results of the verification will be described later, but the following can be said from Figure 5.

[0061] Among the evaluation methods for biomechanical constraints, the range of motion of a walking joint can be estimated based on gait features. Specifically, the hip joint angle and knee joint angle can be estimated as the range of motion of the joint. That is, a score related to a biomechanical constraint can be estimated based on gait features. For example, when the walking speed changes, the hip joint angle or knee joint angle changes accordingly. For example, the slower the walking speed, the smaller the hip joint angle or knee joint angle tends to be. Therefore, it is considered that there is a correlation between the gait features and the hip joint angle or knee joint angle. The hip joint angle or knee joint angle is an example of the range of motion of the ankle joint.

[0062] The estimation unit 33 inputs the gait features of the user U into a first trained model that has been trained by machine learning to input the gait features and output the hip joint angle, and acquires the output as an estimated value of the hip joint angle. The input gait features include at least a walking speed. The input gait features may further include at least one of a walking cycle, a stride length, a step length, a step width, a value obtained by dividing each of the stride length, the step length, and the step width by height, and a value obtained by dividing the stride length by the step width. In addition, at least one of biological information and environmental information may be further input to the first trained model.

[0063] Furthermore, the estimation unit 33 inputs the gait features of the user U into a second trained model that has been trained by machine learning to input the gait features and output the knee joint angle, and acquires the output as an estimated value of the knee joint angle. The input gait features include at least a walking speed. The input gait features may further include at least one of a walking cycle, a stride length, a step length, a step width, a value obtained by dividing each of the stride length, the step length, and the step width by the height, and a value obtained by dividing the stride length by the step width. Furthermore, at least one of biological information and environmental information may further be input into the second trained model.

[0064] The information input to the first trained model and the second trained model (input information corresponding to the hip joint angle or knee joint angle) is set in advance and stored in, for example, a storage unit. The estimation unit 33 extracts the preset information as input information from the bioinformation and environmental information acquired in step S12 and the gait feature amount acquired in step S19, and inputs the extracted input information to the first trained model and the second trained model. The information input to the first trained model and the second trained model may be the same or different from each other.

[0065] Next, the stability limit and verticality will be described. Among the evaluation methods for the stability limit and verticality, the forward functional reach test (FRT) can be estimated based on gait features. In other words, the scores related to the stability limit and verticality can be estimated based on the gait features. For example, height and walking speed are considered to be important for estimating the forward functional reach.

[0066] The estimation unit 33 inputs the biometric information (e.g., body information) of the user U and the gait feature of the user U into a third trained model that has been trained by machine learning to input the biometric information and the gait feature and output the value of the forward functional reach, and acquires an output obtained as an estimated value of the forward functional reach. The input biometric information includes at least height. The input biometric information may further include at least one of gender, weight, and age. The input gait feature includes at least walking speed. The input gait feature may further include at least one of a walking cycle, a stride length, a step length, a step width, a value obtained by dividing each of the stride length, the step length, and the step width by the height, and a value obtained by dividing the stride length by the step width. Environmental information may also be input to the third trained model.

[0067] The information input to the third trained model (input information corresponding to the forward functional reach) is set in advance and stored in, for example, a storage unit. The estimation unit 33 extracts the preset information as input information from the bioinformation and environmental information acquired in step S12 and the gait feature amount acquired in step S19, and inputs the extracted input information to the third trained model.

[0068] Next, posture change will be described. Among the methods for evaluating posture change, standing on one leg (for example, 20 seconds or more is normal) can be estimated based on gait features. That is, a score regarding posture change can be estimated based on gait features. In this embodiment, the time spent standing on one leg with eyes open is estimated as an index for posture change. For example, height and walking speed are considered to be important for estimating the time spent standing on one leg with eyes open.

[0069] The estimation unit 33 inputs the biometric information (e.g., body information) of the user U and the gait features of the user U into a fourth trained model trained by machine learning to input the biometric information and gait features and output the eyes-open one-leg standing time, and acquires the output obtained as an estimate of the eyes-open one-leg standing time. The input biometric information includes at least height. The input biometric information may further include at least one of gender, weight, and age. The input gait features include at least walking speed. The input gait features may further include at least one of a walking cycle, a stride length, a step length, a step width, a value obtained by dividing each of the stride length, the step length, and the step width by height, and a value obtained by dividing the stride length by the step width.

[0070] The input gait feature may further include information indicating a variation in the gait feature. The information indicating the variation in the gait feature includes information indicating a variation in an item according to the item of the input gait feature, and includes, for example, at least information indicating a variation in the walking speed. The information indicating the variation in the gait feature may further include information indicating a variation in at least one of a walking cycle, a stride length, a step length, a step width, a value obtained by dividing each of the stride length, the step length, and the step width by a height, and a value obtained by dividing the stride length by the step width. Furthermore, environmental information may be further input to the fourth trained model.

[0071] The information to be input to the fourth trained model (input information corresponding to the eyes-open one-legged standing time) is set in advance and stored, for example, in a storage unit. The estimation unit 33 extracts the preset information as input information from the bio-information and environmental information acquired in step S12 and the gait feature amount acquired in step S19, and inputs the extracted input information to the fourth trained model.

[0072] Next, the sensory function will be described. Among the methods for evaluating the sensory function, standing on an inclined platform with eyes closed can be estimated based on gait features. That is, a score related to the sensory function can be estimated based on gait features. In this embodiment, the ratio of standing on one leg with eyes open / closed is estimated as an index for the sensory function. The ratio of standing on one leg with eyes open / closed is a ratio based on the time of standing on one leg with eyes open and the time of standing on one leg with eyes closed, and may be, for example, the time (standing on one leg time) logarithmized by log2 (time of standing on one leg with eyes open / time of standing on one leg with eyes closed). For example, height and walking speed are considered to be important for estimating the ratio of standing on one leg with eyes open / closed.

[0073] The estimation unit 33 inputs the biometric information (e.g., body information) of the user U and the gait feature of the user U into a fifth trained model trained by machine learning to input the biometric information and gait feature and output the ratio of standing on one leg with eyes open / closed, and acquires the output obtained as an estimate of the ratio of standing on one leg with eyes open / closed. The input biometric information includes at least height. The input biometric information may further include at least one of gender, weight, and age. The input gait feature includes at least walking speed. The input gait feature may further include at least one of walking cycle, stride length, step length, step width, values ​​obtained by dividing each of the stride length, step length, and step width by height, and values ​​obtained by dividing the stride length by step width.

[0074] The input gait feature may further include information indicating a variation in the gait feature. The information indicating the variation in the gait feature includes information indicating a variation in an item according to the item of the input gait feature, and includes, for example, at least information indicating a variation in the walking speed. The information indicating the variation in the gait feature may further include information indicating a variation in at least one of a walking cycle, a stride length, a step length, a step width, a value obtained by dividing each of the stride length, the step length, and the step width by a height, and a value obtained by dividing the stride length by the step width. Furthermore, environmental information may be further input to the fifth trained model.

[0075] The information input to the fifth trained model (input information according to the ratio of eyes open / eyes closed one-legged standing) is preset and stored in, for example, a storage unit. The estimation unit 33 extracts preset information as input information from the biometric information and environmental information acquired in step S12 and the gait feature amount acquired in step S19, and inputs the extracted input information to the fifth trained model. The input information input to the fifth trained model may be the same as the input information input to the fourth trained model.

[0076] In addition, the estimation unit 33 may estimate (e.g., calculate) the ratio of eyes-open / eyes-closed one-legged standing based on an estimated value of eyes-open one-legged standing time, which is the output of the fourth trained model, and an estimated value of eyes-closed one-legged standing time, which is the output obtained by inputting user U's biometric information (e.g., body information) and user U's gait features acquired based on a video into a trained model trained to output eyes-closed one-legged standing time using biometric information and gait features as input.

[0077] Next, walking stability will be described. Among the methods for evaluating walking stability, TUG (Timed Up to Go) can be estimated based on gait features. Also, based on the gait features, it is possible to estimate whether or not a person has mild cognitive impairment (MCI). In this embodiment, a score regarding walking stability is estimated based on the estimated value of the TUG time and the estimated result of MCI. That is, it is possible to estimate a score regarding walking stability based on the gait features. For example, age, step length, and stride length are considered to be important for estimating the TUG time. For example, walking speed is considered to be important for estimating MCI, but age is also presumed to be important. Note that the estimated result of MCI is a result of estimating whether a person is a healthy person or has MCI.

[0078] The estimation unit 33 inputs the biometric information (e.g., body information) of the user U and the gait feature of the user U acquired based on a video into a sixth trained model that has been trained by machine learning to input the biometric information and gait feature and output the TUG time, and acquires an output obtained as an estimated value of the TUG. The input biometric information includes at least age. The input biometric information may further include at least one of gender, height, and weight. The input gait feature includes at least a step length and a stride length. The input gait feature may further include at least one of a walking cycle, a walking speed, a step width, a value obtained by dividing each of the stride length, the step width, and the step width by the height, and a value obtained by dividing the stride length by the step width. Environmental information may also be input into the sixth trained model.

[0079] The information input to the sixth trained model (input information according to the TUG) is preset and stored in, for example, a storage unit. The estimation unit 33 extracts preset information as input information from the bioinformation and environmental information acquired in step S12 and the gait feature amount acquired in step S19, and inputs the extracted input information to the sixth trained model. The input information input to the sixth trained model may be the same as the input information input to the third trained model.

[0080] Furthermore, the estimation unit 33 inputs the biometric information (e.g., body information) of the user U and the gait feature of the user U acquired based on a video into a seventh trained model that has been trained by machine learning to input the biometric information and gait feature and output a determination result of MCI, and acquires an output obtained as an estimation result of MCI. The input biometric information includes at least one of gender, height, weight, and age. The input gait feature includes at least walking speed. The input gait feature may further include at least one of a walking cycle, a stride length, a step length, a step width, a value obtained by dividing each of the stride length, the step length, and the step width by the height, and a value obtained by dividing the stride length by the step width. For example, the input gait feature may be only the walking speed. Environmental information may also be input into the seventh trained model.

[0081] The information to be input to the seventh trained model (input information according to MCI) is set in advance and stored in, for example, a storage unit. The estimation unit 33 extracts the preset information as input information from the bioinformation and environmental information acquired in step S12 and the gait feature amount acquired in step S19, and inputs the extracted input information to the seventh trained model.

[0082] As described above, the estimation unit 33 estimates the value (e.g., the value (or judgment result) described above, such as the hip joint angle) from at least the gait feature using a trained model (machine learning model) that has been trained in advance by machine learning according to the value to be estimated (or judgment result).

[0083] At least one of the hip joint angle and knee joint angle, FRT, time standing on one leg with eyes open, ratio of standing on one leg with eyes open / closed, TUG, and MCI are examples of evaluation items of physical function. The estimation unit 33 performs estimation for at least two or more evaluation items out of the five evaluation items. For example, the two or more evaluation items may include at least two of the ankle joint range of motion, FRT, standing on one leg with eyes open, standing on one leg with eyes closed, and TUG. The estimation unit 33 may estimate the estimated value as the score of the evaluation item, or may estimate a value corresponding to the estimated value (or the estimated result) as the score based on a table in which the estimated value (or the estimated result) and the score are associated with each other.

[0084] Next, the estimation unit 33 estimates the functional level of the user U's body based on the scores of the evaluation items of the physical function (S21). The estimation unit 33 estimates the functional level of each of the items of the subsystem that detail the balance ability, that is, biomechanical constraints, stability limit and verticality, posture change, sensory function, and walking stability. For example, the estimation unit 33 estimates the functional level for the biomechanical constraints based on at least one score of the hip joint angle and the knee joint angle. Also, the estimation unit 33 estimates the functional level for the stability limit and verticality based on, for example, the score of the FRT. Also, the estimation unit 33 estimates the functional level for the posture change based on, for example, the score of the time of standing on one leg with eyes open. Also, the estimation unit 33 may estimate the functional level for the predictive posture control based on, for example, the score of the time of standing on one leg with eyes open. Also, the estimation unit 33 estimates the functional level for the sensory function based on, for example, the score of the ratio of standing on one leg with eyes open / closed. Furthermore, the estimation unit 33 estimates a functional level for walking stability based on at least one score of the TUG and the MCI, for example. The functional level may be, for example, a numerical value or a stepwise level.

[0085] The estimation unit 33 estimates the functional level based on a table in which the scores and the functional levels are associated with each other, but the method of estimating the functional level is not limited to this.

[0086] The estimation unit 33 outputs the estimated physical functional level of the user U to the proposing unit .

[0087] Next, the proposing unit 34 determines an intervention method for the user U based on the physical functional level of the user U (S22).

[0088] Next, the output unit 35 outputs the estimated physical functional level and the determined intervention method to the terminal device 50, and the terminal device 50 acquires the functional level and the intervention method (S23). Note that, in step S23, it is sufficient that at least the intervention method is output.

[0089] Next, the terminal device 50 displays the physical function level and the intervention method on the display unit 51 (S24). This enables the physical function estimation system 1 to inform the user U of the intervention method according to the gait features of the user U via the terminal device 50.

[0090] It is advisable to periodically estimate the physical function level as described above. The two or more evaluation items may be the same for each user U, for example. This allows the change (progression) in the user U's physical function to be known. For example, an intervention method may be proposed according to an evaluation item with a large amount of change. In addition, the two or more evaluation items may be determined each time, for example, based on the biometric information of the user U (for example, age, etc.). This allows the status of the user U's health concerns, which are determined based on the biometric information of the user U, to be known by checking the scores of the evaluation items.

[0091] [3. Verification of correlation] In the following, verification of whether each evaluation item can be estimated based on gait feature amounts estimated based on a moving image will be described with reference to FIGS.

[0092] First, the estimation of hip joint angles and knee joint angles, which are examples of evaluation items, will be described with reference to Fig. 6 to Fig. 8. Fig. 6 is a diagram showing the correlation in the estimation of hip joint angles between motion capture and a video according to this embodiment. Fig. 6 and Fig. 7 explain the correlation between the value of the evaluation item based on the gait feature amount of a user U acquired by motion capture and the value of the evaluation item based on the gait feature amount of the user U acquired based on a three-dimensional skeletal model from a video.

[0093] In Figure 6, points for 108 people are plotted, with each point representing a hip joint angle based on the gait features of user U obtained by motion capture and a hip joint angle based on the gait features of user U obtained based on a 3D skeletal model from a video, and the correlation coefficient ("R" in the figure) is also shown.

[0094] Specifically, Fig. 6 is a plot of hip joint angles obtained by inputting gait features of user U acquired by motion capture into a trained model that has been trained to output hip joint angles using gait features as input, and hip joint angles obtained by inputting gait features of user U acquired based on a video into the trained model. The correlation coefficient R is 0.713, which means there is a correlation.

[0095] This means that, when the hip joint angle obtained by inputting the gait feature of the user U acquired by motion capture is taken as the correct hip joint angle, the correct hip joint angle can be calculated based on the hip joint angle obtained by inputting the gait feature of the user U acquired based on a video. In other words, the hip joint angle of the user U can be estimated from a video capturing an image of the user U walking.

[0096] FIG. 7 is a diagram showing the correlation in estimating the knee joint angle between motion capture and a video according to the present embodiment.

[0097] In Figure 7, points for 108 people are plotted, with each point representing the knee joint angle based on the gait features of user U obtained by motion capture and the knee joint angle based on the gait features of user U obtained based on a 3D skeletal model from a video, and the correlation coefficient ("R" in the figure) is also shown.

[0098] Specifically, Fig. 7 is a plot of the knee joint angle obtained by inputting the gait features of user U acquired by motion capture into a trained model trained to output the knee joint angle using the gait features as input, and the knee joint angle obtained by inputting the gait features of user U acquired based on a video into the trained model. The correlation coefficient R is 0.577, which means there is a correlation.

[0099] This means that, when the knee joint angle obtained by inputting the gait features of the user U acquired by motion capture is taken as the correct knee joint angle, the correct knee joint angle can be calculated based on the knee joint angle obtained by inputting the gait features of the user U acquired based on a video. In other words, the knee joint angle of the user U can be estimated from a video capturing an image of the user U walking.

[0100] Fig. 8 is a diagram showing the accuracy rate when estimating a normal walking range of motion from a video according to the present embodiment. Fig. 8 shows the accuracy rate of the estimated values ​​of the hip joint angle and knee joint angle obtained by inputting the gait feature of the user U acquired based on the video, in the case where the hip joint angle and knee joint angle obtained by inputting the gait feature of the user U acquired by motion capture are regarded as the correct answer. 1σ (σ is standard deviation) indicates the accuracy rate when the difference between the correct answer and the estimated value is within 1σ, and 2σ indicates the accuracy rate when the difference between the correct answer and the estimated value is within 2σ.

[0101] As shown in FIG. 8, for both the hip joint angle and the knee joint angle, it is possible to estimate values ​​close to the correct answer with a probability of about 80% at 1σ and over 90% at 2σ.

[0102] As described above, since there is a correlation between the hip joint angle and knee joint angle estimated based on a video and the hip joint angle and knee joint angle estimated based on motion capture, it is considered that the hip joint angle and knee joint angle can be estimated based on gait features estimated based on a video.

[0103] Hereinafter, FRT and the like will be described with reference to Figs. 9 to 14. Here, Figs. 9 to 14 will describe verification of the correlation between the value of an evaluation item based on the gait feature of a user U acquired by motion capture and the actual measured value of the evaluation item, and the correlation between the value of an evaluation item based on the gait feature of the user U acquired based on a three-dimensional skeleton model from a video and the actual measured value of the evaluation item. The actual measured values ​​of the evaluation items are common values. Also, in Figs. 9 to 14, verification is performed using "SVR Linear", "SVR RBF", "XGboost" and "Random Forest" as the algorithms of the trained model used to estimate the evaluation items, but the algorithms used are not limited to these. Any existing algorithm may be used as the algorithm of the trained model. Also, [R] in Figs. 9 to 14 indicates a correlation coefficient, and "MSE" indicates a mean square error.

[0104] Next, the estimation of FRT, which is an example of an evaluation item, will be described with reference to Fig. 9. Fig. 9 is a diagram showing the correlation in the estimation of FRT between motion capture and video according to the present embodiment. Fig. 9 is calculated based on data of 106 people who performed FRT.

[0105] In addition, for both the motion capture and video, the trained model shows the results when the following input information is input: gender, height, weight, age, walking cycle, walking speed, stride length, step length, step width, the values ​​obtained by dividing stride length, step length, and step width by height, and the value obtained by dividing stride length by step width.

[0106] As shown in Figure 9, in motion capture, there is a correlation when the algorithms are "SVR Linear", "XGboost" and "Random Forest", and the correlation coefficient is especially highest when it is "SVR Linear". In addition, in video, there is a correlation when the algorithms are "SVR Linear" and "XGboost", and the correlation coefficient is especially highest when it is "SVR Linear".

[0107] For this reason, when estimating FRT, it is recommended to use the "SVR Linear" algorithm for the trained model (e.g., the third trained model). Note that the MSE values ​​are similar for motion capture and video.

[0108] Next, estimation of one-legged standing time with eyes open, which is an example of an evaluation item, will be described with reference to Fig. 10. Fig. 10 is a diagram showing the correlation in estimating one-legged standing time with eyes open between motion capture and video according to this embodiment. Fig. 10 is calculated based on data from 108 people who performed one-legged standing with eyes open. Fig. 10 also shows the result of estimating one-legged standing time with eyes open, which was logarithmized using log2 after adding a component of variation during walking (feature amount of variation) to the input information.

[0109] For both the motion capture and the video, the trained model is input with the following input information: gender, height, weight, age, walking cycle, walking speed, stride length, step length, step width, values ​​obtained by dividing each of the stride length, step length, and step width by height, and values ​​obtained by dividing the stride length by step width, as well as feature values ​​of the variations in each of the walking cycle, walking speed, stride length, step length, step width, values ​​obtained by dividing each of the stride length, step length, and step width by height, and values ​​obtained by dividing the stride length by step width. The feature values ​​of the variations are input with the aim of improving the correlation coefficient.

[0110] As shown in Figure 10, for motion capture, the correlation coefficient is highest when the algorithm is "SVR Linear." For video, the correlation coefficient is highest when the algorithm is "XGboost."

[0111] For the eyes-open one-legged standing time, the correlation coefficient was less than 0.5, but the correlation coefficient was similar between the motion capture and video samples. In other words, the video samples could be used to estimate the time with the same degree of accuracy as the motion capture samples.

[0112] For this reason, when estimating the time spent standing on one leg with eyes open, the algorithm of the trained model (for example, the fourth trained model) should be "SVR Linear" when gait features based on motion capture are input, and "XGboost" when gait features based on video are input. Note that the MSE values ​​are similar for motion capture and video.

[0113] Next, the estimation of the proportion of one-legged standing with eyes open / closed, which is an example of an evaluation item, will be described with reference to Fig. 11. Fig. 11 is a diagram showing the correlation in the estimation of the proportion of one-legged standing with eyes open / closed between the motion capture and video according to this embodiment. Fig. 11 is calculated based on data from 108 people who performed one-legged standing with eyes open and closed. Fig. 11 also shows the result of estimating the one-legged standing time logarithmized by log2 (eyes open / eyes closed) by adding a component of variation during walking (feature amount of variation) to the input information.

[0114] For both the motion capture and the video, the trained model is input with the following input information: gender, height, weight, age, walking cycle, walking speed, stride length, step length, step width, values ​​obtained by dividing each of the stride length, step length, and step width by height, and values ​​obtained by dividing the stride length by step width, as well as feature values ​​of the variations in each of the walking cycle, walking speed, stride length, step length, step width, values ​​obtained by dividing each of the stride length, step length, and step width by height, and values ​​obtained by dividing the stride length by step width. The feature values ​​of the variations are input with the aim of improving the correlation coefficient.

[0115] As shown in Figure 11, the correlation coefficient is highest when the algorithm is "Random Forest" for both motion capture and video. For the proportion of standing on one leg with eyes open / closed, the correlation coefficient is less than 0.5, but the correlation coefficient is similar for motion capture and video. In other words, estimation can be performed from video with the same level of accuracy as motion capture.

[0116] For this reason, when estimating the ratio of open / closed eyes on one leg, it is recommended to use the "Random Forest" algorithm for the trained model (for example, the fifth trained model). Note that the MSE values ​​are similar for motion capture and video.

[0117] Next, the estimation of TUG, which is an example of an evaluation item, will be described with reference to Fig. 12. Fig. 12 is a diagram showing the correlation in the estimation of TUG between motion capture and video according to the present embodiment. Fig. 12 is calculated based on data of 108 people who performed TUG.

[0118] In addition, for both the motion capture and video, the trained model shows the results when the following input information is input: gender, height, weight, age, walking cycle, walking speed, stride length, step length, step width, the values ​​obtained by dividing stride length, step length, and step width by height, and the value obtained by dividing stride length by step width.

[0119] As shown in Figure 12, in motion capture, there is a correlation when the algorithms are "SVR Linear" and "Random Forest." In addition, in videos, there is a correlation when the algorithms are "XGboost" and "Random Forest."

[0120] There is a correlation between motion capture and video when the algorithm is "Random Forest." In other words, a common algorithm can be used for motion capture and video.

[0121] For this reason, when estimating TUG, it is recommended to use the "Random Forest" algorithm for the trained model (e.g., the sixth trained model). Note that the MSE values ​​are similar for motion capture and video.

[0122] Next, estimation of a healthy person or MCI, which is an example of an evaluation item, will be described with reference to Figs. 13 and 14. Fig. 13 is a diagram showing an evaluation result (AUC: Area Under the Curve) in estimation of a healthy person or MCI (10 gait features) using motion capture and video according to this embodiment. Fig. 13 is calculated based on data from 81 people who underwent the MMSE (Mini-Mental State Examination) test. AUC indicates an evaluation index for two-class classification. If the AUC is 0.65 or more, the estimation method is determined to be a good method.

[0123] Note that for both the motion capture and the video, the trained model shows the results when the following input information is input: gender, height, weight, age, walking cycle, walking speed, stride length, step length, step width, values ​​obtained by dividing each of the stride length, step length, and step width by height, and value obtained by dividing the stride length by step width. The trained model performs binary classification of the input information as being either healthy or MCI. In other words, the trained model outputs whether the user U is healthy or has MCI.

[0124] As shown in Figure 13, for motion capture, the AUC is 0.65 or higher when the algorithm is "Random Forest." For video, there is no algorithm with an AUC of 0.65 or higher.

[0125] Therefore, when inputting the above 10 gait features to estimate whether a person is healthy or MCI, the input information should include gait features estimated based on motion capture, and the algorithm of the trained model (e.g., the seventh trained model) should be "Random Forest."

[0126] Here, the inventors discovered that the walking speed of people with MCI tends to be slower than that of healthy people, and further verified the estimation of healthy people or MCI when the gait feature is only walking speed. The verification results will be described below with reference to Fig. 14. Fig. 14 is a diagram showing the evaluation results (AUC) in the estimation of healthy people or MCI (gait feature: only walking speed) of motion capture and video according to the present embodiment.

[0127] As shown in Figure 14, in motion capture, the AUC is 0.65 or higher when the algorithms are "SVR RBF" and "XGboost". In addition, in video, the AUC is 0.65 or higher when the algorithm is "XGboost". In addition, by reducing the gait feature values ​​input to the trained model to only walking speed, the AUC is improved for both motion capture and video.

[0128] In addition, for both motion capture and video, the AUC is 0.65 or higher when the algorithm is "XGboost." In other words, a common algorithm can be used for both motion capture and video.

[0129] For this reason, when estimating whether a person is healthy or has MCI, the gait feature value input to the trained model (e.g., the seventh trained model) should be only the walking speed. When estimating whether a person is healthy or has MCI, the trained model should use the "XGboost" algorithm. Note that the MSE values ​​are the same for motion capture and video.

[0130] As described above, it is possible to estimate the results of the evaluation items from the gait features.

[0131] [4. Effects, etc.] As described above, the physical function estimation system 1 according to one embodiment of the present invention includes an analysis unit 32 (an example of a first estimation unit) that estimates gait features of a user U from a video captured of the user U walking, and an estimation unit 33 (an example of a second estimation unit) that estimates evaluation results for each of two or more evaluation items for evaluating physical function based on the gait features.

[0132] This makes it possible to obtain evaluation results (e.g., scores) for two or more evaluation items based on the gait features. For example, if it is found that there is an abnormality in a physical function, it is possible to estimate which evaluation item has the abnormality by checking the evaluation results for each of the two or more evaluation items. Thus, when there is a problem with a physical function, the physical function estimation system 1 can estimate the cause. Being able to estimate the cause is expected to enable more appropriate intervention for the user U.

[0133] Furthermore, the analysis unit 32 may estimate a skeleton of the user U from the user U appearing in the video, and estimate gait feature amounts based on the estimated skeleton.

[0134] This makes it possible to obtain evaluation results for two or more evaluation items based on a video of the user U walking. In other words, it is possible to easily obtain evaluation results without preparing dedicated equipment or the like.

[0135] In addition, the user U is wearing a marker, and the video is a video capturing the user U walking while wearing the marker. The analysis unit 32 may estimate the gait feature amount based on the marker shown in the video.

[0136] This makes it possible to estimate the cause of a problem in physical function, if any, even when gait features acquired based on motion capture are used.

[0137] The two or more evaluation items may include a first item related to a balance system, and the gait feature amount may include at least one of a walking cycle, a walking speed, a stride length, a step length, and a step width. The estimation unit 33 may estimate the first item based on the at least one of the items.

[0138] This makes it possible to estimate whether or not the cause of a problem with physical function is in the balance system. By including items related to the balance system in the evaluation items, it is expected that more appropriate interventions can be performed for the user U regarding the balance system.

[0139] The two or more evaluation items may include a second item related to flexibility, and the gait feature amount may include at least one of a hip joint angle and a knee joint angle. The estimation unit 33 may estimate the second item based on the at least one of the hip joint angle and the knee joint angle.

[0140] This makes it possible to estimate whether or not the cause of a problem with physical function is in the flexibility system. By including items related to the flexibility system in the evaluation items, it is expected that more appropriate interventions for the flexibility system can be performed for user U.

[0141] The two or more evaluation items may include a third item related to the muscle power system, and the gait feature amount may include walking speed. The estimation unit 33 may estimate the third item based on at least the walking speed.

[0142] This makes it possible to estimate whether the cause of a problem with physical function is in the muscle strength system. By including items related to the muscle strength system in the evaluation items, it is expected that more appropriate interventions can be performed for the user U regarding the muscle strength system.

[0143] Furthermore, the estimation unit 33 estimates the scores for each of the two or more evaluation items as the evaluation results.

[0144] This allows the evaluation results to be quantified, making it easier to identify the causes simply by looking at the score.

[0145] Furthermore, the estimation unit 33 may further estimate the evaluation results of each of two or more evaluation items based on the biometric feature amount of the user U.

[0146] This makes it possible to obtain an evaluation result that is estimated also according to the biometric feature amount of the user U. For example, it is expected that the accuracy of the evaluation result will be improved.

[0147] Furthermore, the estimation unit 33 may further estimate the evaluation results of each of two or more evaluation items based on an environmental feature amount indicating the walking environment of the user U.

[0148] This makes it possible to obtain an evaluation result estimated according to the environmental feature amount of the user U. For example, it is expected that the accuracy of the evaluation result will be improved.

[0149] In addition, the two or more evaluation items include at least two of ankle joint range of motion, FRT, standing on one leg with eyes open, standing on one leg with eyes closed, and TUG.

[0150] This makes it possible to estimate the evaluation results of each evaluation item of the balance system. If balance ability is impaired, the cause can be estimated in detail.

[0151] A physical function estimation method according to an embodiment of the present invention estimates gait features of a user U from a video of the user U walking (S19), and estimates evaluation results of two or more evaluation items for evaluating physical function based on the gait features (S20). A program according to an embodiment of the present invention is a program for causing a computer to execute the above-described physical function estimation method.

[0152] This provides the same effects as those of the above-described physical function estimation system 1.

[0153] (Other embodiments) Although the physical function estimation system according to one or more aspects has been described based on the embodiment, the present invention is not limited to this embodiment. As long as it does not deviate from the spirit of the present invention, various modifications conceived by a person skilled in the art to this embodiment and forms constructed by combining components in different embodiments may also be included in the present invention.

[0154] For example, in the above-mentioned embodiments, an example of estimating balance ability as an example of a physical function based on gait features has been described, but the physical function is not limited to balance ability. The physical function may include, for example, muscle strength, flexibility, and the like. For example, the control device may estimate lower limb muscle strength information of a user shown in a video based on the knee bending angle of the leg that steps forward while the user is walking. The control device may also estimate lower limb muscle strength information of a person shown in a video based on a change in walking speed when the user stops walking. In this way, physical functions other than balance ability may also be estimated from gait features.

[0155] For example, the physical function evaluation item may include an evaluation item related to the flexibility system (an example of a second item), and the gait feature amount may include at least one of a hip joint angle and a knee joint angle, and the estimation unit may estimate the evaluation item related to the flexibility system based on the at least one of the hip joint angle and the knee joint angle.

[0156] Also, for example, the evaluation items of physical function may include evaluation items related to the muscle system (an example of a third item), the gait features may include at least walking speed, and the estimation unit may estimate the evaluation items related to the muscle system based on at least the walking speed.

[0157] Furthermore, the order in which each step (process) in the flowchart described in the above embodiment is executed is merely an example for specifically explaining the present invention, and an order other than the above may be used. The order of multiple processes may be changed, and multiple processes may be executed in parallel. Furthermore, a process executed by a specific processing unit may be executed by another processing unit. Furthermore, some of the steps may be executed simultaneously (in parallel) with other steps, and some of the steps may not be executed.

[0158] In addition, the division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as one functional block, one functional block may be divided into multiple blocks, or some functions may be transferred to another functional block. Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in a time-sharing manner by a single piece of hardware or software.

[0159] In the above embodiment, the physical function estimation system is realized by a plurality of devices, but it may be realized as a single device. When the physical function estimation system is realized by a plurality of devices, each component of the physical function estimation system may be allocated to the plurality of devices in any manner.

[0160] In the above embodiment, the control device is realized by a single device, but may be realized by multiple devices. When the control device is realized by multiple devices, each component of the control device may be distributed among the multiple devices in any manner. The communication method between the multiple devices is not particularly limited, and may be wireless communication, wired communication, or a combination of wireless communication and wired communication.

[0161] In the above embodiments, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0162] Moreover, each of the components described in the above embodiments may be realized as software, or may be realized as an LSI, which is typically an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip to include some or all of them. Here, LSI is used, but it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration. Furthermore, the method of integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. After LSI manufacture, a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connection or setting of the circuit cells inside the LSI may be used. Furthermore, if an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology or a different derived technology, it is natural that the integration of the components may be performed using that technology.

[0163] A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple processing units on a single chip, and specifically, is a computer system that includes a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. Computer programs are stored in the ROM. The system LSI achieves its functions by the microprocessor operating in accordance with the computer program.

[0164] In addition, the general or specific aspects of the present invention may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. Also, the present invention may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. For example, the present invention may be realized as a program for causing a computer to execute the physical function estimation method of the above embodiment, or as a computer-readable non-transitory recording medium in which such a program is stored. For example, such a program may be recorded on a recording medium and distributed or circulated. For example, the distributed program may be installed in a device having another processor, and the program may be executed by the processor, thereby making it possible to cause the device to perform each of the above processes. [Explanation of symbols]

[0165] 1. Physical function estimation system 32 Analysis part (1st estimation part) 33 Estimation Department (Second Estimation Department) U User

Claims

1. a first estimation unit that estimates a gait feature of a user from a video captured of the user walking; a second estimation unit that estimates an evaluation result of each of two or more evaluation items for evaluating a physical function, the evaluation items including at least one of a Functional Reach Test (FRT), an open-eyes one-legged standing, and a closed-eyes one-legged standing, based on the gait feature amount; the first estimation unit identifies a three-dimensional skeletal model of the user shown in the video by using an algorithm used in a machine learning model, and estimates the gait feature amount by using a position change of each skeleton point of the identified three-dimensional skeletal model; The second estimation unit acquires an output of a trained model obtained by inputting the estimated gait feature amount into the trained model as a score of the evaluation result for each of the two or more evaluation items. Physical function estimation system.

2. The first estimation unit estimates a skeleton of the user from the user shown in the video, and estimates the gait feature amount based on the estimated skeleton. The physical function estimation system according to claim 1 .

3. The user is wearing a marker; the video is a video capturing an image of the user walking while wearing the marker, The first estimation unit estimates the gait feature amount based on the markers shown in the video. The physical function estimation system according to claim 1 .

4. The two or more evaluation items include a first item related to a balance system, The gait feature amount includes at least one of a walking cycle, a walking speed, a stride length, a step width, and a step width; The second estimation unit estimates the first item based on the at least one of the items. The body function estimation system according to any one of claims 1 to 3.

5. The two or more evaluation items include a second item related to a flexible system, the gait feature amount includes at least one of a hip joint angle and a knee joint angle; The second estimation unit estimates the second item based on the at least one of the items. The body function estimation system according to any one of claims 1 to 4.

6. The two or more evaluation items include a third item related to the muscle strength system, the gait feature amount includes a walking speed; The second estimation unit estimates the third item based on at least the walking speed. The body function estimation system according to any one of claims 1 to 5.

7. The second estimation unit estimates a score for each of the two or more evaluation items as the evaluation result. The body function estimation system according to any one of claims 1 to 6.

8. The second estimation unit further estimates an evaluation result for each of the two or more evaluation items based on a biometric feature of the user. The body function estimation system according to any one of claims 1 to 7.

9. The second estimation unit further estimates an evaluation result for each of the two or more evaluation items based on an environmental feature amount indicating a walking environment of the user. The body function estimation system according to any one of claims 1 to 8.

10. The two or more evaluation items include at least two of the following: ankle joint range of motion, FRT (Functional Reach Test), open-eyes one-legged standing, closed-eyes one-legged standing, and TUG (Timed Up and Go). The body function estimation system according to any one of claims 1 to 9.

11. A physical function estimation method executed by a physical function estimation system including a first estimation unit that estimates gait features of a user from a video of the user walking, and a second estimation unit that estimates evaluation results of two or more evaluation items for evaluating physical function, the evaluation items including at least one of a Functional Reach Test (FRT), standing on one leg with eyes open, and standing on one leg with eyes closed, based on the gait features, the first estimation unit and the second estimation unit are processors; The first estimation unit estimates the gait feature amount of the user from the video capturing an image of the user walking; the second estimation unit estimates the evaluation results of each of the two or more evaluation items for evaluating a physical function, the evaluation items including at least one of the FRT, the open-eyes one-legged standing, and the closed-eyes one-legged standing, based on the gait feature amount; In estimating the gait feature amount, the first estimation unit identifies a three-dimensional skeletal model of the user shown in the video by using an algorithm used in a machine learning model, and estimates the gait feature amount by using a position change of each skeleton point of the identified three-dimensional skeletal model; In estimating the evaluation result, the second estimation unit inputs the estimated gait feature amount into a trained model and acquires an output of the trained model as a score of the evaluation result for each of the two or more evaluation items. Physical function estimation method.

12. A program for causing a computer to execute the body function estimation method according to claim 11.

Citation Information

Patent Citations

  • Gait recognition method for multi-gait feature combined collaborative dictionary

    CN108921062A

  • Health measuring apparatus

    JP2008104667A

  • Walking movement analyzer

    JP2008173250A

  • Motion analysis and evaluation device, motion analysis and evaluation method, and program

    JP2016140591A

  • Physical fitness measurement method, activity support method, program, and physical fitness measurement system

    JP2019204451A

Cited By

  • Machine learning method, computer-readable recording medium, and information processing device

    WO2026204271A1