A gait analysis system that evaluates video gait analysis results by support phase, determines the analysis quality, and automatically generates a natural language report.

Abstract

This system provides a gait analysis system that easily analyzes gait patterns from walking videos taken with smartphones, etc., and automatically generates a natural language report that can be understood without requiring specialized knowledge. [Solution] The gait analysis system of this invention consists of a camera device that films a person walking, a gait analysis device that analyzes the walking state using the video from the camera device, and an output device that automatically generates and outputs a natural language report based on the analysis results of the gait analysis device. Skeletal points are estimated from the video, and the left and right support phases are extracted to calculate gait indicators such as trunk sway, joint inclination, knee sway, and step width. Multiple viewpoints, including frontal and lateral views, are analyzed separately, and the video quality is evaluated using the visibility of skeletal points, the number of frames, and the continuity of joint coordinates. The obtained indicators are compiled as structured data, and the characteristics of each left and right support phase and walking tendencies are expressed in text using artificial intelligence and output, thereby enabling simple and reliable gait evaluation.

Description

Technical Field

[0003] , , , , , , , , , , , , ,

[0004] , ,

[0001] The present invention relates to a walking analysis system that analyzes a walking state using a video of a person's walking and presents the analysis result as an easy-to-understand sentence to the user.

Background Art

[0002] Conventionally, the mainstream method for walking analysis is to use expensive three-dimensional motion analysis devices or multiple sensors, making it difficult to use in small-scale facilities or home environments. In recent years, technologies for estimating joint positions and the like from videos are known, but there is a problem that the obtained numerical information requires specialized knowledge and is difficult for general users and on-site staff to understand. In addition, in many existing systems, analysis is performed without considering video quality, resulting in problems such as unreliable results being output due to insufficient frame numbers, unstable detection of skeleton points, and jumps in joint coordinates. Furthermore, the evaluation is centered on the overall average, and features such as differences between the left and right support phases and trunk compensation are not sufficiently visualized.

Summary of the Invention

Problems to be Solved by the Invention

[0003] An object of the present invention is to provide a walking analysis system that can easily analyze a walking state from a walking video taken with a smartphone or the like, automatically determine the video quality, and automatically generate a natural language report that can be understood without requiring specialized knowledge using feature amounts for each left and right support phase.

Means for Solving the Problems

[0004] The walking analysis system according to the present invention includes the following. 1. A camera device that photographs a person's walking 2. A walking analysis device having the following configuration · An acquisition unit that acquires video data of a subject's walking. • An analysis unit that estimates the skeletal points of a person from the aforementioned video data and generates time-series data relating to the head, shoulders, pelvis, knees, and feet. A support period determination unit that extracts the left support period and the right support period from the aforementioned time-series data. • An index calculation unit that calculates a walking index that includes at least the following for each of the aforementioned left-right support phases. • Lateral sway of the trunk or head • Shoulder or pelvic tilt • Knee joint varus or valgus or degree of instability • Variation in stride length or foot position A viewpoint-based analysis unit that distinguishes and processes videos taken from the front and videos taken from the side, calculating left-right asymmetry in the front view and trunk flexion, knee flexion, and ground contact tendency in the side view. • A summary generation unit that compiles the calculated walking index as structured data. A report generation unit that uses artificial intelligence to process the aforementioned structured data into natural language and automatically generates a text report that includes characteristics for each left-right support period, overall trends, and usage precautions. A quality determination unit that evaluates the analysis quality of the aforementioned video data using at least one of the visibility information of skeletal points, the number of frames available for analysis, and the temporal continuity of joint coordinates. Based on the evaluation results of the quality determination unit, the control unit controls whether to output walking indicators, whether to display them as reference, or whether to hide them by body part. 3. An output device that displays the aforementioned text report and analysis results on the user terminal or outputs them as document files. Furthermore, the report generation unit does not perform a medical diagnosis, but generates text as information about walking tendencies and other reference information. [Effects of the Invention]

[0005] According to this invention, Gait analysis becomes possible using only standard video recording. Analysis of the left and right support phases allows us to understand gait asymmetry and trunk compensation. Converting numerical indicators into natural language makes them understandable even to users without specialized knowledge. By automatically determining video quality, the output of unreliable analysis results can be suppressed. By assessing the quality of each body part, it becomes possible to display or hide partially unreliable results as reference information. This makes it possible to provide practical gait assessments at low cost in small medical institutions, nursing care facilities, and home settings. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 shows an example of a gait analysis system configuration that acquires gait videos and performs skeletal estimation, support phase determination, gait index calculation, analysis quality assessment, and text report generation. [Modes for carrying out the invention]

[0007] The gait analysis system according to this invention has a configuration in which a video of walking captured by a camera device such as a smartphone is transmitted to an analysis server, and the analysis server installed in the gait analysis device performs skeletal estimation, support phase determination, gait index calculation, and analysis quality determination. The output device outputs the analysis results either as a screen display on the user's terminal or as a document in PDF format. [Examples]

[0008] As an example, videos of the subject walking, filmed from the front and side using a smartphone camera, were used. The video was then transmitted to an analysis server installed in the gait analysis device, and after estimating the skeletal points, the left and right support phases were extracted to calculate trunk sway, knee sway, and step width. Furthermore, the analysis quality was evaluated based on the visibility of skeletal points and the number of effective frames, and the analysis results were output as a written report using an output device. [Industrial applicability]

[0009] This invention can be used for gait assessment in medical institutions, nursing care facilities, rehabilitation facilities, and home environments, and has industrial applicability. [Explanation of Symbols]

[0010] 1. Video acquisition unit 2 Skeleton Estimation Unit 3 Support period determination section 4 Indicator calculation section 4a Analysis quality judgment section 5. Viewpoint-Based Analysis Section 6 Summary generator 7 Sentence generation section

Claims

[Claim 1] Claim 1 A gait analysis system that analyzes the gait state using video data of a person walking as input, comprising: a skeletal estimation means for estimating the skeletal points of the person from the video data; an analysis means for generating time-series data of the head, shoulders, pelvis, knees, and feet based on the skeletal points; a support period determination means for extracting the left support period and the right support period from the time-series data, respectively; an index calculation means for calculating gait indices for each left and right support period, including the amount of lateral sway of the trunk or head, the amount of tilt of the shoulders or pelvis, the amount of varus or valgus or sway of the knee joint, and the variability of the step width or foot position; and video footage taken from the front and video footage taken from the side. A gait analysis system comprising: a viewpoint-based analysis means that distinguishes and processes the following, calculating left-right asymmetry in the frontal direction and trunk flexion, knee flexion, and ground contact tendency in the lateral direction; a summary generation means that compiles the calculated gait indices as structured data; a report generation means that takes the structured data as input and performs natural language generation processing using artificial intelligence to automatically generate a text report including characteristics and gait tendencies for each left-right support phase; and an output means that displays the text report on a user terminal or outputs it as a document file, wherein the text report does not provide medical diagnosis but is presented as gait tendencies and reference information. Claim 2 A gait analysis system according to claim 1, characterized in that it includes a quality determination means that evaluates the analysis quality of the video data using at least one of the visibility information of skeletal points, the number of effective frames, and the temporal continuity of joint coordinates, and controls whether or not to output a gait index or to display its reliability based on the evaluation result. Claim 3 A gait analysis system according to claim 2, characterized in that the quality determination means determines the analysis quality for each body part and hides or displays the analysis results for each body part.

Images