Gait measurement device, gait measurement method, and recording medium

Abstract

In order to generate highly accurate walking data, this gait measurement device comprises: a video acquisition unit that acquires video data; an action recognition unit that extracts spatial coordinates of a target portion constituting the skeleton of a person appearing in the video data; a walking data generation unit that, using the extracted spatial coordinates of the target portion, generates walking data corresponding to the movement of the feet of the person; and an output unit that outputs the generated walking data.

Description

Gait measurement device, gait measurement method, and recording medium

[0001] The present disclosure relates to a gait measurement device, a gait measurement method, and a recording medium.

[0002] With growing interest in healthcare, services that provide information according to gait patterns are attracting attention. Patent Document 1 discloses a gait analysis device that measures walking movements. The gait analysis device of Patent Document 1 measures the walking movements of a subject using motion capture. Using walking movement data measured by such a method, highly accurate walking data can be calculated. If a large amount of highly accurate walking data can be collected, it becomes possible to estimate the health condition of the subject.

[0003] JP 2009-125270 A

[0004] There is little publicly available data on walking movements measured using a method such as that described in Patent Document 1. Therefore, it has been difficult to collect a large amount of walking movement data to the extent that it is possible to estimate the health condition of a target person. In order to estimate the health condition of a target person, it is necessary to collect a large amount of highly accurate walking data.

[0005] An object of the present disclosure is to provide a gait measurement device, a gait measurement method, and a recording medium that are capable of generating highly accurate walking data.

[0006] A gait measurement device according to one aspect of the present disclosure includes a video acquisition unit that acquires video data, a motion recognition unit that extracts spatial coordinates of target parts that constitute the skeleton of a person appearing in the video data, a gait data generation unit that generates gait data corresponding to foot movement using the spatial coordinates of the extracted target parts, and an output unit that outputs the generated gait data.

[0007] A gait measurement method according to one aspect of the present disclosure acquires video data, extracts spatial coordinates of target parts constituting the skeleton of a person depicted in the video data, generates gait data corresponding to foot movement using the spatial coordinates of the extracted target parts, and outputs the generated gait data.

[0008] A program according to one aspect of the present disclosure causes a computer to perform the following processes: acquiring video data; extracting spatial coordinates of target parts that constitute the skeleton of a person depicted in the video data; generating walking data corresponding to foot movement using the spatial coordinates of the extracted target parts; and outputting the generated walking data.

[0009] According to the present disclosure, it is possible to provide a gait measurement device, a gait measurement method, and a recording medium that are capable of generating highly accurate walking data.

[0010] 1 is a block diagram showing an example of the configuration of a gait measurement device according to the present disclosure. FIG. 1 is a conceptual diagram showing an example of video data input to a gait measurement device according to the present disclosure. FIG. 2 is a conceptual diagram showing an example of skeletal data superimposed on video data input to a gait measurement device according to the present disclosure. FIG. 3 is a conceptual diagram for explaining an example of skeletal data extracted by a gait measurement device according to the present disclosure. FIG. 4 is a conceptual diagram for explaining a coordinate system set for each frame constituting video data input to a gait measurement device according to the present disclosure. FIG. 5 is a conceptual diagram for explaining an example of spatial positions and spatial angles used by a gait measurement device according to the present disclosure. FIG. 6 is a conceptual diagram for explaining an example of generation of gait data by a gait measurement device according to the present disclosure. FIG. 7 is a conceptual diagram for explaining a gait cycle. FIG. 8 is a conceptual diagram showing an example arrangement of measurement devices that measure gait data. FIG. 9 is a conceptual diagram for explaining an example of measurement of gait data using a measurement device mounted on a shoe. FIG. 10 is a conceptual diagram for explaining an example of measurement of gait data using a gait measurement device according to the present disclosure. FIG. 11 is a graph showing a comparative example of pseudo gait data measured using a gait measurement device according to the present disclosure and actual gait data measured using a measurement device. FIG. 1 is a conceptual diagram showing an example of a conversion model for converting pseudo gait data measured using a gait measurement device according to the present disclosure into measured gait data measured using a measurement device. FIG. 1 is a conceptual diagram for describing an example of sparse modeling. FIG. 1 is a conceptual diagram for describing an example of generating a sparse coefficient vector using measured gait data measured using a measurement device implemented in a shoe. FIG. 1 is a conceptual diagram for describing an example of generating a sparse coefficient vector using pseudo gait data calculated using a gait measurement device. FIG. 1 is a conceptual diagram for describing an example of converting pseudo gait data into measured gait data using a sparse modeling technique. FIG. 1 is a conceptual diagram for describing an example in which a sparse coefficient vector of gait data measured using a measurement device implemented in a shoe is restored. FIG. 2 is a flowchart for describing an example of the operation of a gait measurement device according to the present disclosure. FIG. 2 is a block diagram for describing an example of the configuration of a gait measurement device according to the present disclosure.Fig. 1 is a flowchart for describing an example of the operation of a gait measurement device according to the present disclosure. Fig. 2 is a conceptual diagram for describing an application example according to the present disclosure. Fig. 3 is a block diagram for describing an example of the configuration of a gait measurement device according to the present disclosure. Fig. 4 is a flowchart for describing an example of the operation of a gait measurement device according to the present disclosure. Fig. 5 is a block diagram for describing an example of a hardware configuration for executing control and processing according to the present disclosure.

[0011] Hereinafter, embodiments for carrying out the present disclosure will be described with reference to the drawings. In this disclosure, drawings used in describing each embodiment relate to one or more embodiments. Furthermore, elements included in each drawing may apply to one or more embodiments. The embodiments described below are limited in a manner that is technically preferable for carrying out the present disclosure, but this does not limit the scope of the disclosure. In all drawings used in describing the following embodiments, similar parts are designated by the same reference numerals unless otherwise specified. In the following embodiments, repeated description of similar configurations and operations may be omitted. The direction of arrows in the drawings indicates an example of the flow of signals, data, etc., and does not limit the direction of signals, data, etc.

[0012] First Embodiment First, a gait measurement device according to the first embodiment will be described with reference to the drawings. The gait measurement device of this embodiment acquires video data in which a person is captured. The gait measurement device of this embodiment recognizes the person included in the video data. Based on the recognized movement of the person, the gait measurement device of this embodiment generates data related to the person's gait (gait data). The gait data includes acceleration, angular velocity, and angle (posture angle) according to foot movement.

[0013] 1 is a block diagram showing an example of the configuration of a gait measurement device according to the present disclosure. Gait measurement device 10 includes an image acquisition unit 11, an action recognition unit 12, a walking data generation unit 13, and an output unit 17. Gait measurement device 10 also includes an action recognition model 120.

[0014] The video acquisition unit 11 acquires video data that shows at least one person. The video data is composed of multiple frames captured in chronological order. There are no particular limitations on the time intervals at which each of the multiple frames is captured. Furthermore, there are no particular limitations on the format of the video data. The video acquisition unit 11 outputs the acquired video data to the action recognition unit 12.

[0015] 2 is a conceptual diagram showing an example of video data showing people. The video data 111 is composed of multiple frames captured in chronological order. In the video data 111, people appearing in the frames are shown as white frames, but in actual frames, people are not displayed as white frames. In the example of FIG. 2, multiple people appear in the video data 111.

[0016] The action recognition unit 12 acquires video data from the video acquisition unit 11. The action recognition unit 12 uses the action recognition model 120 to extract skeletal data of a person appearing in each of the multiple frames that make up the acquired video data.

[0017] The action recognition model 120 is a model that outputs skeletal data of a person appearing in video data in response to input of the video data. For example, the action recognition model 120 is a model that extracts skeletal data using the technology disclosed in Non-Patent Document 1 (Non-Patent Document 1: Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, “OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp. 172-186, 1 Jan. 2021, doi: 10.1109 / TPAMI.2019.2929257.).

[0018] The action recognition model 120 extracts a person from each of multiple frames constituting the input video data. The action recognition model 120 extracts a representative part (target part) of the person. Hereinafter, the representative part of the person is referred to as the target part. For example, the action recognition model 120 extracts joints and extremities as target parts. The action recognition model 120 outputs the generated skeletal data. The skeletal data includes position coordinates of the target part of each person extracted in each frame. The action recognition model 120 outputs the generated skeletal data. The action recognition model 120 may generate skeletal data in which the extracted target parts are connected by connecting lines. For example, the skeletal data is displayed superimposed on the person appearing in the video data. Note that the processing by the action recognition model 120 described here is merely an example and does not limit the processing by the action recognition model 120. The action recognition model 120 may perform any internal processing as long as it outputs skeletal data of the person appearing in the video data in response to input video data.

[0019] 3 is a conceptual diagram showing an example of skeletal data being superimposed on a person appearing in video data. In the example of FIG. 3, skeletal data is superimposed on the person appearing in video data 112. When skeletal data is superimposed on the person, the movement of the person is visualized. For example, by using the technology of Non-Patent Document 1, it is possible to generate a video in which skeletal data is superimposed on a person.

[0020] For example, the action recognition model 120 may be a machine learning model trained using a machine learning technique. For example, the action recognition model 120 is a model trained using a dataset in which image data is used as an explanatory variable and skeletal data is used as a target variable (label) as training data. For example, the action recognition model 120 may be a model trained using a dataset in which video data composed of a plurality of image data (frames) is used as an explanatory variable and skeletal data is used as a target variable (label) as training data. For example, the action recognition model 120 is a training model trained using a convolutional neural network (CNN) technique. For example, the action recognition model 120 is a model trained using a principal component analysis (PCA) technique. For example, the action recognition model 120 is a training model trained using a variational autoencoder (VAE). For example, the action recognition model 120 is a training model trained using a conditional generative adversarial network (GAN) technique. The above techniques are merely examples and do not limit the training techniques for the action recognition model 120.

[0021] FIG. 4 is a conceptual diagram showing an example of skeletal data extracted from video data. FIG. 4 shows skeletal data 115 of a person extracted from each frame. In the skeletal data 115 extracted from each frame, the person's skeleton is expressed in a format in which circles representing target parts are connected by connecting lines. Target parts include joint parts and extremity parts. Joint parts include shoulders, elbows, wrists, neck, chest, waist, hips, knees, and ankles. Extremity parts include the head, fingertips, and toes. Target parts that are related to each other are connected by connecting lines. For example, the head and neck, wrists and fingertips, heels and toes, etc. are connected by connecting lines.

[0022] The action recognition unit 12 calculates the position coordinates of a specific target part using skeletal data extracted from the video data. In this embodiment, the action recognition unit 12 calculates the position coordinates of the toes and heels in each frame. The action recognition unit 12 calculates the position coordinates of the arch of the foot using the calculated position coordinates of the toes and heels. The action recognition unit 12 converts the position coordinates of the target part from the two-dimensional coordinate system in each frame to a three-dimensional position (spatial position) in a three-dimensional coordinate system (spatial coordinate system). The position coordinates of the target part are expressed as three-dimensional spatial coordinates in the spatial coordinate system of the frame from which the skeletal data was extracted.

[0023] FIG. 5 is a conceptual diagram illustrating the relationship between a local coordinate system set at the arch of the foot and a world coordinate system set relative to the ground in each frame. FIG. 5 shows a coordinate system set at the right foot. In the world coordinate system (X-axis, Y-axis, Z-axis), when a user is standing upright facing the direction of travel, the horizontal direction is set as the X-axis direction, the direction of travel as the Y-axis direction, and the direction of gravity as the Z-axis direction. The example in FIG. 5 conceptually illustrates the relationship between the local coordinate system (X-axis, Y-axis, Z-axis) and the world coordinate system (X-axis, Y-axis, Z-axis). The example in FIG. 5 does not accurately illustrate the relationship between the local coordinate system and the world coordinate system, which changes depending on the user's walking.

[0024] Furthermore, the action recognition unit 12 calculates the angle formed by two connecting lines connected to a specific target part using the calculated position coordinates of the target part. In this embodiment, the action recognition unit 12 calculates the angle formed by two connecting lines connected to the heel in each frame. The angle formed by the two connecting lines connected to the heel corresponds to the angle formed between the foot and the shin. The action recognition unit 12 converts the calculated three-dimensional angle (spatial angle) formed between the foot and the shin into a three-dimensional angle (postural angle) formed by the sole of the foot with respect to the ground in each frame. The process of converting the spatial angle into a posture angle may be performed by the walking data generation unit 13.

[0025] FIG. 6 is a conceptual diagram for explaining an example of a spatial position and a spatial angle. The spatial position corresponds to the three-dimensional position coordinates of the target part J. In the example of FIG. 6, the position of the heel is shown as the target part J. The spatial angle corresponds to the three-dimensional angle θ (Euler angle) formed by two connecting lines connected to the target part J. In the example of FIG. 6, the three-dimensional angle θ formed by the connecting line indicating the foot and the connecting line indicating the shin is shown, with the target part J indicating the heel as the center.

[0026] The action recognition unit 12 calculates, for each frame, spatial position coordinates indicating the three-dimensional position (spatial position) of the target part in each frame and a posture angle corresponding to the three-dimensional angle formed by the sole of the foot with respect to the ground in each frame. The action recognition unit 12 outputs a data set of the spatial position coordinates and posture angle calculated for each frame constituting the video data to the walking data generation unit 13. The action recognition unit 12 outputs a data set calculated for multiple temporally consecutive frames.

[0027] The walking data generation unit 13 acquires a data set calculated for multiple temporally consecutive frames. The walking data generation unit 13 calculates three-dimensional acceleration (spatial acceleration) in accordance with changes in spatial position coordinates between frames. The walking data generation unit 13 sets the person's forward direction (forward) in each frame as a moving axis. The walking data generation unit 13 sets the person's up-down direction in each frame as a vertical axis. The walking data generation unit 13 also sets the person's left-right direction in each frame as a horizontal axis. The moving axis, vertical axis, and horizontal axis are mutually orthogonal to each other. The walking data generation unit 13 calculates moving acceleration along the moving axis, vertical acceleration along the vertical axis, and lateral acceleration along the lateral axis. The walking data generation unit 13 may calculate three-dimensional velocity (spatial velocity) in accordance with changes in spatial position coordinates between frames.

[0028] The walking data generation unit 13 also calculates three-dimensional angular velocities (spatial angular velocities) in response to changes in posture angle between frames. The walking data generation unit 13 calculates angular velocities around the forward axis, the vertical axis, and the lateral axis. The walking data generation unit 13 may also calculate three-dimensional angular accelerations (spatial angular accelerations) in response to changes in posture angle between frames.

[0029] FIG. 7 is a conceptual diagram illustrating an example of generating gait data using video data. For each person appearing in each of the multiple frames constituting the video data, spatial coordinates of the target portion are extracted. The gait data generation unit 13 performs quadratic differentiation on the spatial coordinates extracted from the multiple frames constituting the video data to derive acceleration. The gait data generation unit 13 also calculates a rotation matrix of the spatial coordinates extracted from the multiple frames constituting the video data to derive a posture angle. The gait data generation unit 13 also calculates a first derivative of the derived posture angle to derive angular velocity. The posture angle, angular velocity, and acceleration are derived with respect to the three axes of the forward direction, vertical direction, and lateral direction. A data set of the posture angle, angular velocity, and acceleration derived with respect to the three axes of the forward direction, vertical direction, and lateral direction is generated for each frame.

[0030] The walking data generation unit 13 integrates the acceleration, angular velocity, and attitude angle calculated for each frame across multiple frames to generate time-series data for each. The walking data generation unit 13 may generate time-series data for velocity and angular acceleration. The time-series data generated by the walking data generation unit 13 is called walking data. For example, the walking data is expressed as a walking waveform in a two-dimensional coordinate space with time or walking period on the horizontal axis and each value of acceleration, angular velocity, and attitude angle on the vertical axis. The walking data generation unit 13 outputs the generated walking data to the output unit 17.

[0031] The output unit 17 acquires walking data from the walking data generation unit 13. The output unit 17 outputs the acquired walking data. For example, the output unit 17 may be configured to output the walking data to a terminal device or a server in which software that uses the walking data is implemented. For example, the output unit 17 may be configured to output the walking data to an external system that uses the walking data. There are no particular limitations on the use of the walking data.

[0032] [Gait Cycle] Next, a gait cycle showing the periodicity of a person's walking will be described with reference to the drawings. FIG. 8 is a conceptual diagram for explaining a step gait cycle based on the right foot. A step gait cycle based on the left foot is similar to that of the right foot. The horizontal axis of FIG. 8 indicates one gait cycle of the right foot, starting from the point when the heel of the right foot hits the ground and ending from the point when the heel of the right foot hits the ground again. The horizontal axis of FIG. 8 is normalized with the step gait cycle set to 100%. Normalizing one gait cycle to 100% is called first normalization. One gait cycle of one foot is broadly divided into a stance phase, in which at least a portion of the sole of the foot is in contact with the ground, and a swing phase, in which the sole of the foot is off the ground. The stance phase is the period in which at least a portion of the sole of the foot is in contact with the ground. The stance phase is further divided into an early stance phase T1, a mid-stance phase T2, a final stance phase T3, and an early swing phase T4. The swing phase is the period during which the sole of the foot is off the ground. The swing phase is further divided into an early swing phase T5, a mid-swing phase T6, and a final swing phase T7. The horizontal axis in FIG. 8 is normalized so that the stance phase is 60% and the swing phase is 40%. Normalizing gait waveform data so that the stance phase is 60% and the swing phase is 40% is called second normalization. Note that the periods shown in FIG. 8 are merely examples and do not limit the periods that make up a gait cycle or the names of those periods.

[0033] As shown in FIG. 8 , multiple events occur during walking. Multiple events that occur during walking are also called walking events. P1 represents heel strike (HS). Heel strike is an event in which the heel of the right foot touches the ground. P2 represents opposite toe off (OTO). Opposite toe off is an event in which the toe of the left foot leaves the ground while the sole of the right foot is in contact with the ground. P3 represents heel rise (HR). Heel rise is an event in which the heel of the right foot lifts while the sole of the right foot is in contact with the ground. P4 represents opposite heel strike (OHS). Opposite heel strike is an event in which the heel of the left foot touches the ground. P5 represents toe off (TO). Toe-off is an event in which the toe of the right foot leaves the ground while the sole of the left foot is in contact with the ground. P6 represents foot adjacent (FA). Foot crossing is an event in which the left and right feet cross while the sole of the left foot is in contact with the ground. P7 represents tibia vertical (TV). Tibia vertical is an event in which the tibia of the right foot becomes approximately perpendicular to the ground while the sole of the left foot is in contact with the ground. P8 represents heel strike. P8 corresponds to the end point of the gait cycle that begins with P1 and the start point of the next gait cycle. Note that the gait events shown in FIG. 8 are merely examples and do not limit the events that occur during walking or the names of these events.

[0034] The timing of heel strike is the timing of the minimum peak immediately after the maximum peak that appears in the time series data of forward acceleration (Y-direction acceleration). The maximum peak that marks the heel strike timing corresponds to the maximum peak of the walking waveform data for one step cycle. The section between consecutive heel strikes corresponds to one step cycle. The timing of toe lift is the timing of the rise of the maximum peak that appears after the stance phase period in which no fluctuations appear in the time series data of forward acceleration (Y-direction acceleration). The timing midway between the timing of the minimum roll angle and the timing of the maximum roll angle corresponds to the mid-stance phase.

[0035] [Gait Data] Next, the characteristics of the gait data generated by the gait measurement device 10 of this embodiment will be described with reference to the drawings. Below, gait data generated using sensor data measured by a measurement device placed inside the footwear will be compared with gait data generated by the gait measurement device 10 of this embodiment. Gait data generated using sensor data measured by a measurement device placed inside the footwear will be referred to as actual gait data. Gait data generated by the gait measurement device 10 of this embodiment will be referred to as pseudo gait data.

[0036] FIG. 9 is a conceptual diagram showing an example in which measurement devices for measuring walking data are placed inside shoes on both feet. In the example of FIG. 9 , the measurement device 110 is placed at a position corresponding to the back of the arch of the foot. The measurement device 110 includes sensors for measuring spatial acceleration and spatial angular velocity. For example, the measurement device 110 is placed in an insole inserted into the shoe 100. For example, the measurement device 110 may be placed on the bottom of the shoe 100. For example, the measurement device 110 may be embedded in the body of the shoe 100. The measurement device 110 may be detachable from the shoe 100 or may not be detachable from the shoe 100. The measurement device 110 may be placed at a position other than the back of the arch of the foot as long as it can measure sensor data related to foot movement. The measurement device 110 may also be placed in socks worn by the user or in an accessory such as an anklet worn by the user. The measurement device 110 may also be attached directly to the foot or embedded in the foot. The measurement device 110 may be placed inside one of the shoes 100 as long as it can measure sensor data that can generate walking data.

[0037] In the example of FIG. 9 , a local coordinate system is set with the measuring device 110 as the reference, and includes an x-axis in the left-right direction, a y-axis in the front-back direction, and a z-axis in the up-down direction. In this embodiment, the x-axis is positive to the left, the y-axis is positive to the rear, and the z-axis is positive to the up. FIG. 9 shows an example in which the same coordinate system is set for the left foot and the right foot. For example, when measuring devices 110 manufactured to the same specifications are placed in left and right shoes 100, the up-down orientations (Z-axis directions) of the measuring devices 110 placed in the left and right shoes 100 are the same. In this case, the three axes of the local coordinate system set for the sensor data derived from the left foot and the three axes of the local coordinate system set for the sensor data derived from the right foot are the same for the left and right.

[0038] FIG. 10 is a conceptual diagram illustrating an example of measuring gait data using a measurement device mounted on a shoe. When a measurement device mounted on a shoe is used, gait data (measured gait data S1) is measured according to actual musculoskeletal movements and actual foot movements. The process of converting gait T reflected in actual walking into measured gait data S1 using the measurement device is called conversion process Q11. In conversion process Q11, because musculoskeletal and foot movements are included in actual walking movements, noise can be added to the measured gait data S1 measured by the measurement device. For example, adding random noise to the measured gait data S1 can increase the amount of gait data.

[0039] FIG. 11 is a conceptual diagram illustrating an example of gait data measurement using the gait measurement device 10. The gait measurement device 10 uses the action recognition model 120 to extract musculoskeletal movements (skeletal data) from video data. The gait measurement device 10 extracts foot movements from the skeletal data. The gait measurement device 10 extracts spatial coordinates of positions including target parts included in the skeletal data as foot movements. The gait measurement device 10 executes a gait data calculation process using the extracted spatial coordinates indicating the foot movements to generate gait data (pseudo gait data S2). The process of converting gait T included in the video data into pseudo gait data S2 using the measurement device is referred to as conversion process Q12. In conversion process Q12, noise can be added to the skeletal data extracted using the action recognition model 120, the spatial coordinates of positions including target parts extracted from the skeletal data, and the pseudo gait data S2 generated by the gait data calculation process. For example, the amount of gait data can be increased by adding random noise to these data and coordinates. Compared to the measured walking data S1, the pseudo walking data S2 has more stages in which noise can be added, and therefore, more diverse walking data can be generated.

[0040] FIG. 12 is a graph showing a comparison between measured gait data S1 and pseudo gait data S2. FIG. 12 shows time series data (gait waveform) of forward acceleration for one walking cycle. FIG. 12 shows time series data (gait waveform) of vertical acceleration for one walking cycle. In FIG. 12, measured gait data S1 is shown by a solid line, and pseudo gait data S2 is shown by a dashed line. The pseudo gait data S2 shows a similar change trend to the measured gait data S1, but is less accurate than the measured gait data S1. A physical condition that can be estimated based on a rough change trend in the gait waveform can be estimated using the pseudo gait data S2. However, a physical condition that cannot be estimated without distinguishing fine changes is difficult to estimate using the pseudo gait data S2. Therefore, in order to estimate a physical condition with high accuracy, it is necessary to improve the accuracy of the pseudo gait data S2.

[0041] 13 is a conceptual diagram showing an example of a conversion model for converting pseudo walking data into measured walking data. If the pseudo walking data S2 can be converted into measured walking data S1, highly accurate walking data can be generated. The process of converting the conversion process Q11 shown in FIG. 10 into the conversion process Q12 shown in FIG. 11 is referred to as arithmetic process f. The arithmetic process f corresponds to the inverse conversion of the arithmetic process f. -1 The conversion model 131 can convert the pseudo walking data S2 into the measured walking data S1 by performing the calculation process f -1 By executing the above, the pseudo walking data S2 is converted into the actual walking data S1.

[0042] For example, a data set is generated for an actual gait T, including measured gait data S1 measured using the measurement device 110 and pseudo gait data S2 generated from video data. For example, the measured gait data S1 and the pseudo gait data S2 are expanded by adding random noise. The data set can be expanded by associating the expanded measured gait data S1 and the pseudo gait data S2. High-precision measured gait data S1 can be expanded by training the expanded data set on a machine learning model that outputs measured gait data S1 in response to input of pseudo gait data S2. For example, by using a machine learning technique such as deep learning, it is possible to train a machine learning model to output high-precision measured gait data S1.

[0043] [Sparse Modeling] Next, an example of data augmentation using the sparse modeling technique will be described with reference to the drawings. In sparse modeling, an originally non-sparse signal is decomposed to extract sparse information.

[0044] FIG. 14 is a conceptual diagram for explaining an example of sparse modeling. A time-domain signal contains many non-zero points. A sparse signal can be obtained by converting the time-domain signal into a frequency-domain signal through a Fast Fourier Transform (FFT). An arbitrary time-domain vector x contained in n-dimensional Euclidean space can be n is an arbitrary frequency domain vector X nIf there is a linear mapping that transforms the frequency domain vector X into the time domain vector x, then a dictionary matrix A that satisfies the following equation 1 can be assumed (n is a natural number): AX = x (1) The above equation 1 indicates that the frequency domain vector X can be transformed into the time domain vector x using the dictionary matrix A.

[0045] For example, the dictionary matrix A can be constructed using a transformation method such as an inverse Fourier transform (IFFT), a discrete cosine transform, or a wavelet transform. However, even when using the above transformation methods, there are cases in which a sparse signal cannot be obtained. To address such cases, a sparse coding method can be used to learn measured data and construct the dictionary matrix A. Gait data includes overlapping data within a gait cycle. Therefore, the sparse coding method can also be applied to generating gait data.

[0046] For example, the dictionary matrix A can be derived using the k-SVD (Singular Value Decomposition) method. In a linear space based on the column vectors of the dictionary matrix A, the signal is sparse. However, since the size of the dictionary matrix A is set in advance, the number of basis vectors is not optimal. If the dictionary matrix A is decomposed using SVD and an orthonormal basis within the range of the dictionary matrix A is found, the number of basis vectors can be optimized. In this way, the original signal can be restored using an inverse Fourier transform (IFFT).

[0047] FIG. 15 is a conceptual diagram illustrating an example of generating a sparse coefficient vector using gait data measured using a shoe-mounted measurement device. When a shoe-mounted measurement device is used, gait data (measured gait data S1) is measured according to actual musculoskeletal movements and actual foot movements. The dictionary matrix A converts the vector of the time-domain measured gait data S1 into a frequency-domain sparse coefficient vector V1. The process of converting the gait T expressed in actual walking into the sparse coefficient vector V1 is called the conversion process Q21. In the conversion process Q21, because musculoskeletal and foot movements are included in actual walking movements, noise can be added to the measured gait data S1 measured by the measurement device. For example, the sparse coefficient vector V1 can be increased by adding random noise to the measured gait data S1.

[0048] FIG. 16 is a conceptual diagram illustrating an example of generating a sparse coefficient vector using gait data calculated using the gait measurement device 10. The gait measurement device 10 uses the action recognition model 120 to extract musculoskeletal movements (skeletal data) from video data. The gait measurement device 10 extracts foot movements from the skeletal data. The gait measurement device 10 extracts, as foot movements, spatial coordinates of positions including target parts included in the skeletal data. The gait measurement device 10 executes a gait data calculation process using the extracted spatial coordinates indicating the foot movements to generate gait data (pseudo gait data S2). The dictionary matrix B converts the vector of the time-domain pseudo gait data S2 into a sparse coefficient vector V2 in the frequency domain. The process of converting the gait T included in the video data into the sparse coefficient vector V2 is called a conversion process Q22. In the conversion process Q22, noise can be added to the skeleton data extracted using the action recognition model 120, the spatial coordinates of the position including the target part extracted from the skeleton data, and the pseudo gait data S2 generated by the gait data calculation process. For example, the sparse coefficient vector V2 can be increased by adding random noise to these data and coordinates. Compared to the sparse coefficient vector V1, the sparse coefficient vector V2 has more stages at which noise can be added, making it possible to generate a variety of sparse coefficient vectors.

[0049] FIG. 17 is a conceptual diagram showing an example of converting pseudo gait data S2 into measured gait data S1 using a sparse modeling technique. In the example of FIG. 17, a conversion model 132 is used that converts a sparse coefficient vector V2 into a sparse coefficient vector V1. For example, the conversion model 132 is constructed by training a machine learning model configured to output a sparse coefficient vector V1 in response to input of a sparse coefficient vector V2, using a data set of the sparse coefficient vector V1 and the sparse coefficient vector V2. For example, the conversion model 132 is constructed using a machine learning technique such as deep learning.

[0050] In the example of FIG. 17 , the pseudo gait data S2 is converted into a sparse coefficient vector V2 using a dictionary matrix B. The conversion model 132 outputs a sparse coefficient vector V1 in response to the input of the sparse coefficient vector V2. The sparse coefficient vector V1 is converted into a dictionary matrix A that performs an inverse conversion of the dictionary matrix A. -1 The data is converted into actual walking data S1 by the above method.

[0051] FIG. 18 is a conceptual diagram illustrating an example in which a sparse coefficient vector is reconstructed from gait data measured using a shoe-mounted measurement device. In the example of FIG. 18 , the measured gait data (solid line) and the reconstructed gait data (dash-dotted line) are shown superimposed. In the example of FIG. 18 , the measured gait data was converted into a sparse coefficient vector with a compression rate of 92 percent using a dictionary matrix generated using the measured values ​​of the gait data. The sparse coefficient vector was reconstructed using a dictionary matrix that performs the inverse transformation used in the transformation. The correlation coefficient R between the measured gait data (solid line) and the reconstructed gait data (dash-dotted line) was 0.9905. Thus, by using a sparse modeling technique, the measured data can be reconstructed with high accuracy using a compressed sparse coefficient vector.

[0052] (Operation) Next, the operation of gait measurement device 10 according to this embodiment will be described with reference to the drawings. FIG. 19 is a flowchart for explaining an example of the operation of a gait measurement device according to the present disclosure. In the explanation of the processing of the flowchart of FIG. 19, the components of gait measurement device 10 will be described as the actors performing the operations. The actor performing the processing according to the flowchart of FIG. 19 may be gait measurement device 10.

[0053] In FIG. 19, first, the video acquisition unit 11 acquires video data (step S11).

[0054] Next, the movement recognition unit 12 converts the walking movement of the person included in the video data into a skeleton model (step S12).

[0055] Next, the action recognition unit 12 extracts the spatial coordinates of the target part in the transformed skeletal model (step S13).

[0056] Next, the walking data generating unit 13 generates walking data using the spatial coordinates of the extracted target portion (step S14).

[0057] Next, the output unit 17 outputs the generated walking data (step S15).

[0058] As described above, the gait measurement device of this embodiment includes a video acquisition unit, a movement recognition unit, a walking data generation unit, and an output unit. The video acquisition unit acquires video data. The movement recognition unit extracts spatial coordinates of target portions constituting the skeleton of a person depicted in the video data. In response to input of the video data, the movement recognition unit converts the video data into skeletal data including target portions of the person depicted in the video data using a movement recognition model that outputs skeletal data of at least one person depicted in the video data. For example, the movement recognition model is a machine learning model constructed using a machine learning technique. The walking data generation unit generates walking data corresponding to the foot movement using the spatial coordinates of the extracted target portions. The output unit outputs the generated walking data.

[0059] The gait measurement device of this embodiment uses a motion recognition model to convert video data into skeletal data that includes target parts of a person. The gait measurement device of this embodiment uses spatial coordinates of the target parts included in the skeletal data to generate gait data corresponding to foot movements. A large amount of video data containing people can be collected. Therefore, according to this embodiment, highly accurate gait data can be generated using a large amount of collected video data.

[0060] In one aspect of the present embodiment, the action recognition unit calculates, as walking data, spatial acceleration, spatial angular velocity, and spatial angle indicating foot movement, using skeletal data of the person extracted from the video data. According to this aspect, the spatial acceleration, spatial angular velocity, and spatial angle indicating foot movement can be calculated according to the person's movement extracted from the video data.

[0061] Second Embodiment Next, a gait measurement device according to a second embodiment will be described with reference to the drawings. The gait measurement device of this embodiment differs from the gait measurement device of the first embodiment in that it calculates gait parameters using generated walking data. In the following, a description of the same components as those of the first embodiment will be omitted.

[0062] 20 is a block diagram showing an example of the configuration of a gait measurement device according to the present disclosure. Gait measurement device 20 includes an image acquisition unit 21, an action recognition unit 22, a walking data generation unit 23, a parameter calculation unit 25, and an output unit 27. Gait measurement device 20 also includes an action recognition model 220.

[0063] The video acquisition unit 21 has the same configuration as the video acquisition unit 11 of the first embodiment. The video acquisition unit 21 acquires video data that shows at least one person. The video data is composed of multiple frames captured in chronological order. The video acquisition unit 21 outputs the acquired video data to the action recognition unit 22.

[0064] The action recognition unit 22 has the same configuration as the action recognition unit 12 of the first embodiment. The action recognition unit 22 acquires video data from the video acquisition unit 21. The action recognition unit 22 extracts skeletal data of a person appearing in each of a plurality of frames constituting the acquired video data using an action recognition model 220. The action recognition model 220 is a model that outputs skeletal data of a person appearing in video data in response to input of the video data.

[0065] The action recognition unit 22 calculates the position coordinates of a specific target part using skeletal data extracted from the video data. In this embodiment, the action recognition unit 22 calculates the position coordinates of the toes and heels in each frame. The action recognition unit 22 calculates the position coordinates of the arch of the foot using the calculated position coordinates of the toes and heels. The action recognition unit 22 converts the position coordinates of the target part from the two-dimensional coordinate system in each frame to a three-dimensional position (spatial position) in a three-dimensional coordinate system (spatial coordinate system). The position coordinates of the target part are expressed as three-dimensional spatial coordinates in the spatial coordinate system of the frame from which the skeletal data was extracted.

[0066] Furthermore, the action recognition unit 22 calculates the angle formed by two connecting lines connected to a specific target part using the calculated position coordinates of the target part. In this embodiment, the action recognition unit 22 calculates the angle formed by two connecting lines connected to the heel in each frame. The angle formed by the two connecting lines connected to the heel corresponds to the angle formed between the foot and the shin. The action recognition unit 22 converts the calculated three-dimensional angle (spatial angle) formed between the foot and the shin into a three-dimensional angle (postural angle) formed by the sole of the foot with respect to the ground in each frame. The process of converting the spatial angle into a posture angle may be performed by the walking data generation unit 23.

[0067] The action recognition unit 22 calculates, for each frame, spatial position coordinates indicating the three-dimensional position (spatial position) of the target part in each frame and a posture angle corresponding to the three-dimensional angle formed by the sole of the foot with respect to the ground in each frame. The action recognition unit 22 outputs a data set of the spatial position coordinates and posture angle calculated for each frame constituting the video data to the walking data generation unit 23. The action recognition unit 22 outputs a data set calculated for multiple temporally consecutive frames.

[0068] The walking data generation unit 23 has the same configuration as the walking data generation unit 13 of the first embodiment. The walking data generation unit 23 acquires a data set calculated for multiple temporally consecutive frames. The walking data generation unit 23 calculates three-dimensional acceleration (spatial acceleration) in accordance with changes in spatial position coordinates between frames. The walking data generation unit 23 calculates forward acceleration along a forward axis, vertical acceleration along a vertical axis, and lateral acceleration along a lateral axis. The walking data generation unit 23 may also calculate three-dimensional velocity (spatial velocity) in accordance with changes in spatial position coordinates between frames.

[0069] The walking data generation unit 23 also calculates three-dimensional angular velocities (spatial angular velocities) in response to changes in posture angle between frames. The walking data generation unit 23 calculates angular velocities around the forward axis, the vertical axis, and the lateral axis. The walking data generation unit 23 may also calculate three-dimensional angular acceleration (spatial angular acceleration) in response to changes in posture angle between frames.

[0070] The walking data generation unit 23 integrates the acceleration, angular velocity, and posture angle calculated for each frame across multiple frames to generate time-series data for each. The walking data generation unit 23 may generate time-series data for velocity and angular acceleration. The time-series data generated by the walking data generation unit 23 is called walking data. For example, the walking data is expressed as a walking waveform in a two-dimensional coordinate space with time or walking period on the horizontal axis and each value of acceleration, angular velocity, and posture angle on the vertical axis. The walking data generation unit 23 outputs the generated walking data to the parameter calculation unit 25.

[0071] The parameter calculation unit 25 acquires gait data from the gait data generation unit 23. The parameter calculation unit 25 extracts time series data for one walking cycle from the time series data of acceleration in three axial directions and angular velocity around three axes included in the gait data. The time series data for one walking cycle is also called gait waveform data. The parameter calculation unit 25 extracts gait waveform data based on the timing of walking events detected from the time series data of the gait data. For example, the parameter calculation unit 25 extracts gait waveform data that starts at the timing of a heel strike and ends at the timing of the next heel strike.

[0072] The parameter calculation unit 25 normalizes (first normalization) the time of the extracted walking waveform data for one step cycle to a walking cycle of 0 to 100% (percent). The timing of 1%, 10%, etc. included in the 0 to 100% walking cycle is also called a walking phase. Furthermore, the parameter calculation unit 25 normalizes (second normalization) the first-normalized walking waveform data for one step cycle so that the stance phase is 60% and the swing phase is 40%. By performing second normalization on the walking waveform data, it is possible to reduce the deviation in the walking phase, which reveals gait characteristics.

[0073] For example, the parameter calculation unit 25 extracts gait waveform data for one step gait cycle using the traveling acceleration. The parameter calculation unit 25 extracts gait waveform data for one step gait cycle for accelerations / angular velocities / angles other than the traveling acceleration in accordance with the gait cycle of the traveling acceleration. The parameter calculation unit 25 extracts gait waveform data for accelerations in three axial directions, gait waveform data for angular velocities around three axes, and gait waveform data for angles around three axes (posture angles). The parameter calculation unit 25 normalizes the extracted gait waveform data for one step gait cycle.

[0074] The parameter calculation unit 25 may extract gait waveform data for one step gait cycle using acceleration / angular velocity other than forward acceleration. For example, the parameter calculation unit 25 may detect heel strike and toe lift from time series data of vertical acceleration. The timing of heel strike is the timing of a steep minimum peak that appears in the time series data of vertical acceleration. At the timing of the steep minimum peak, the value of vertical acceleration becomes approximately 0. The minimum peak that marks the timing of heel strike corresponds to the minimum peak of the gait waveform data for one step gait cycle. The interval between consecutive heel strikes is a gait cycle. The timing of toe lift is the timing of an inflection point in the time series data of vertical acceleration, where the vertical acceleration gradually increases after passing through a period of small fluctuation following the maximum peak immediately after heel strike. Alternatively, the parameter calculation unit 25 may extract gait waveform data for one step gait cycle using both forward acceleration and vertical acceleration. Furthermore, the parameter calculation unit 25 may extract walking waveform data for one walking cycle using accelerations other than the forward acceleration and vertical acceleration, angular velocities, angles, and the like.

[0075] The parameter calculation unit 25 calculates gait parameters using the normalized walking waveform data. For example, the gait parameters are used to estimate a physical state. There are no particular limitations on the gait parameters calculated by the parameter calculation unit 25. For example, the parameter calculation unit 25 calculates gait parameters related to distance, height, angle, speed, time, CPEI (Center of Pressure Exclusion Index), frailty level, etc. Representative gait parameters are listed below. Specific methods for calculating the following gait parameters will be omitted.

[0076] For example, the parameter calculation unit 25 calculates indices related to distance and height as gait parameters. For example, the parameter calculation unit 25 calculates a stride length, a turning distance, a foot lift height, FTC (Foot Clearance), and MTC (Minimum Toe Clearance). The stride length indicates the distance between the front foot and the rear foot while walking. The turning distance indicates the maximum distance that the foot is separated outward in the direction of travel during the swing phase. The foot lift height indicates the maximum distance between the measurement device 110 and the ground during the swing phase. The FTC indicates the maximum distance between the heel and the ground during the swing phase. The MTC indicates the minimum distance between the toe and the ground during the swing phase.

[0077] For example, the parameter calculation unit 25 calculates angle-related indicators as gait parameters. For example, the parameter calculation unit 25 calculates the contact angle, the takeoff angle, the toe direction, the heel-strike roll angle, the toe-off roll angle, the swing leg peak angular velocity, and the hallux angle. The contact angle indicates the maximum value of the angle between the sole of the foot and the ground at heel-strike. The takeoff angle indicates the angle between the sole of the foot and the ground during the swing phase. The toe direction indicates the average value of the orientation of the toe relative to the direction of forward motion during the swing phase. The heel-strike roll angle is the angle between the ankle and the ground at heel-strike, as viewed from a rearward perspective. The toe-off roll angle is the angle between the ankle and the ground at push-off, as viewed from a rearward perspective. The swing leg peak angular velocity is the angular velocity in the ankle dorsiflexion direction during the period from immediately after push-off until the toe comes closest to the ground. The hallux angle indicates the angle at which the big toe is tilted toward the index toe. Specifically, the hallux angle is the angle between the center line of the first metatarsal and the center line of the first proximal phalanx.

[0078] For example, the parameter calculation unit 25 calculates speed-related indices as gait parameters. For example, the parameter calculation unit 25 calculates walking speed, cadence, and maximum swing speed. Walking speed indicates the walking speed. Cadence indicates the number of steps per minute. Maximum swing speed indicates the speed at which the leg is swung out during the swing phase.

[0079] For example, the parameter calculation unit 25 calculates time-related indicators as gait parameters. For example, the parameter calculation unit 25 calculates stance time, load-bearing time, sole contact time, push-off time, swing time, and DST (Double Support Time). Stance time indicates the time during which the foot is in contact with the ground during walking. Stance time is the sum of load-bearing time, sole contact time, and push-off time. Load-bearing time is the time during the stance phase from when the heel contacts the ground to when the toe contacts the ground. Sole contact time is the time during the stance phase when the entire sole of the foot is in contact with the ground and is horizontal to the ground. Push-off time is the time during the stance phase from when the sole is in contact with the ground to when the toe pushes off the ground. Swing time indicates the time during which the foot is off the ground during walking. DST is divided into DST1 and DST2. DST1 indicates the time during which the foot carrying the measurement device 110 is in front of the other foot during a period when both feet are in contact with the ground at the same time. DST2 indicates the time during which the foot carrying the measurement device 110 is behind the other foot during a period when both feet are in contact with the ground at the same time.

[0080] For example, the parameter calculation unit 25 calculates a center of pressure exclusion index (CPEI) as a gait parameter. The CPEI indicates an estimated rate of expansion of the center of foot pressure applied to the ground during the stance phase.

[0081] For example, the parameter calculation unit 25 calculates a frailty level as a gait parameter. The frailty level is an estimated value of a frailty state according to a walking state. For example, the parameter calculation unit 25 estimates an index indicating a determination result regarding frailty as the frailty level. If there is no possibility of frailty, the parameter calculation unit 25 estimates an index indicating that the subject is not frail. If there is a possibility of frailty, the parameter calculation unit 25 estimates an index indicating that the subject is likely to be frail. Furthermore, if there is a high possibility of frailty, the parameter calculation unit 25 estimates an index indicating that there is a high possibility of frailty.

[0082] The parameter calculation unit 25 may extract feature quantities related to gait parameters from the walking waveform data. For example, the parameter calculation unit 25 extracts feature quantities for each walking phase cluster according to preset conditions. A walking phase cluster is a cluster that integrates temporally consecutive walking phases. A walking phase cluster includes at least one walking phase. A walking phase cluster may also include a single walking phase. The parameter calculation unit 25 may extract physical ability feature quantities used to estimate physical ability. For example, the physical ability feature quantities are used to estimate at least one of physical abilities such as grip strength (total muscle strength of the entire body), dynamic balance, lower limb muscle strength, mobility ability, and static balance.

[0083] The parameter calculation unit 25 outputs the calculated gait parameters to the output unit 27. The gait parameters calculated by the parameter calculation unit 25 may be selected depending on the application.

[0084] The output unit 27 acquires gait parameters from the parameter calculation unit 25. The output unit 27 outputs the acquired gait parameters. For example, the output unit 27 may be configured to output the gait parameters to a terminal device or server on which software that uses the gait parameters is implemented. For example, the output unit 27 may be configured to output the gait parameters to an external system or the like that uses the gait parameters. There are no particular limitations on the use of the gait parameters.

[0085] (Operation) Next, the operation of the gait measurement device 20 according to the present embodiment will be described with reference to the drawings. FIG. 21 is a flowchart for explaining an example of the operation of the gait measurement device according to the present disclosure. In the explanation of the processing of the flowchart of FIG. 21 , the components of the gait measurement device 20 will be described as the subject of the operations. The subject of the operations of the processing according to the flowchart of FIG. 21 may be the gait measurement device 20.

[0086] In FIG. 21, first, the video acquisition unit 21 acquires video data (step S21).

[0087] Next, the movement recognition unit 22 converts the walking movement of the person included in the video data into a skeleton model (step S22).

[0088] Next, the action recognition unit 22 extracts the spatial coordinates of the target part in the transformed skeletal model (step S23).

[0089] Next, the walking data generating unit 23 generates walking data using the spatial coordinates of the extracted target portion (step S24).

[0090] Next, the parameter calculation unit 25 calculates gait parameters using the generated walking data (step S25).

[0091] Next, the output unit 27 outputs the generated gait parameters (step S26).

[0092] As described above, the gait measurement device includes a video acquisition unit, a movement recognition unit, a walking data generation unit, a parameter calculation unit, and an output unit. The video acquisition unit acquires video data. The movement recognition unit extracts spatial coordinates of target portions constituting the skeleton of a person depicted in the video data. The movement recognition unit converts the input video data into skeletal data including target portions of the person depicted in the video data using a movement recognition model that outputs skeletal data of at least one person depicted in the video data. For example, the movement recognition model is a machine learning model constructed using a machine learning technique. The walking data generation unit generates gait data corresponding to foot movement using the spatial coordinates of the extracted target portions. The parameter calculation unit calculates gait parameters using the generated gait data. For example, the parameter calculation unit calculates gait parameters used to estimate a body state. The output unit outputs the calculated gait parameters.

[0093] The gait measurement device of this embodiment uses a motion recognition model to convert video data into skeletal data including target parts of a person. The gait measurement device of this embodiment uses spatial coordinates of the target parts included in the skeletal data to generate gait data corresponding to foot movements. The gait measurement device of this embodiment calculates gait parameters using the generated gait data. A large amount of video data containing people can be collected. Therefore, according to this embodiment, highly accurate gait parameters can be generated using a large amount of collected video data.

[0094] Third Embodiment Next, a gait measurement device according to a third embodiment will be described with reference to the drawings. The gait measurement device of this embodiment differs from the gait measurement devices of the first and second embodiments in that it estimates a physical state using calculated walking data. In the following, a description of the same components as those of the first and second embodiments will be omitted.

[0095] 22 is a block diagram showing an example of the configuration of a gait measurement device according to the present disclosure. Gait measurement device 30 includes an image acquisition unit 31, an action recognition unit 32, a walking data generation unit 33, a parameter calculation unit 35, an estimation unit 36, and an output unit 37. Gait measurement device 30 also includes an action recognition model 320.

[0096] The video acquisition unit 31 has the same configuration as the video acquisition unit 11 of the first embodiment. The video acquisition unit 31 acquires video data that shows at least one person. The video data is composed of multiple frames captured in chronological order. The video acquisition unit 31 outputs the acquired video data to the action recognition unit 32.

[0097] The action recognition unit 32 has the same configuration as the action recognition unit 12 of the first embodiment. The action recognition unit 32 acquires video data from the video acquisition unit 31. The action recognition unit 32 extracts skeletal data of a person appearing in each of a plurality of frames constituting the acquired video data using an action recognition model 320. The action recognition model 320 is a model that outputs skeletal data of a person appearing in video data in response to input of the video data.

[0098] The action recognition unit 32 calculates the position coordinates of a specific target part using skeletal data extracted from the video data. In this embodiment, the action recognition unit 32 calculates the position coordinates of the toes and heels in each frame. The action recognition unit 32 calculates the position coordinates of the arch of the foot using the calculated position coordinates of the toes and heels. The action recognition unit 32 converts the position coordinates of the target part from the two-dimensional coordinate system in each frame to a three-dimensional position (spatial position) in a three-dimensional coordinate system (spatial coordinate system). The position coordinates of the target part are expressed as three-dimensional spatial coordinates in the spatial coordinate system of the frame from which the skeletal data was extracted.

[0099] Furthermore, the action recognition unit 32 uses the calculated position coordinates of the target part to calculate the angle formed by two connecting lines connected to the specific target part. In this embodiment, the action recognition unit 32 calculates the angle formed by two connecting lines connected to the heel in each frame. The angle formed by the two connecting lines connected to the heel corresponds to the angle formed between the foot and the shin. The action recognition unit 32 converts the calculated three-dimensional angle (spatial angle) formed between the foot and the shin into a three-dimensional angle (postural angle) formed by the sole of the foot with respect to the ground in each frame. The process of converting the spatial angle into a posture angle may be performed by the walking data generation unit 33.

[0100] The action recognition unit 32 calculates, for each frame, spatial position coordinates indicating the three-dimensional position (spatial position) of the target part in each frame and a posture angle corresponding to the three-dimensional angle formed by the sole of the foot with respect to the ground in each frame. The action recognition unit 32 outputs a data set of the spatial position coordinates and posture angle calculated for each frame constituting the video data to the walking data generation unit 33. The action recognition unit 32 outputs a data set calculated for multiple temporally consecutive frames.

[0101] The walking data generation unit 33 has the same configuration as the walking data generation unit 13 of the first embodiment. The walking data generation unit 33 acquires a data set calculated for multiple temporally consecutive frames. The walking data generation unit 33 calculates three-dimensional acceleration (spatial acceleration) in accordance with changes in spatial position coordinates between frames. The walking data generation unit 33 calculates forward acceleration along a forward axis, vertical acceleration along a vertical axis, and lateral acceleration along a lateral axis. The walking data generation unit 33 may also calculate three-dimensional velocity (spatial velocity) in accordance with changes in spatial position coordinates between frames.

[0102] Furthermore, the walking data generation unit 33 calculates three-dimensional angular velocity (spatial angular velocity) in accordance with changes in posture angle between frames. The walking data generation unit 33 calculates angular velocity around the forward axis, angular velocity around the vertical axis, and angular velocity around the lateral axis. The walking data generation unit 33 may also calculate three-dimensional angular acceleration (spatial angular acceleration) in accordance with changes in posture angle between frames.

[0103] The walking data generation unit 33 integrates the acceleration, angular velocity, and posture angle calculated for each frame across multiple frames to generate time-series data for each. The walking data generation unit 33 may generate time-series data for velocity and angular acceleration. The time-series data generated by the walking data generation unit 33 is called walking data. For example, the walking data is expressed as a walking waveform in a two-dimensional coordinate space with time or walking period on the horizontal axis and each value of acceleration, angular velocity, and posture angle on the vertical axis. The walking data generation unit 33 outputs the generated walking data to the parameter calculation unit 35.

[0104] The parameter calculation unit 35 has a configuration similar to that of the parameter calculation unit 25 of the second embodiment. The parameter calculation unit 35 acquires gait data from the gait data generation unit 33. The parameter calculation unit 35 extracts time series data for one walking cycle from the time series data of acceleration in three axial directions and angular velocity around three axes included in the gait data. The time series data for one walking cycle is also referred to as gait waveform data. The parameter calculation unit 35 extracts gait waveform data based on the timing of walking events detected from the time series data of the gait data. For example, the parameter calculation unit 35 extracts gait waveform data that starts at the timing of a heel strike and ends at the timing of the next heel strike.

[0105] The parameter calculation unit 35 normalizes (first normalization) the time of the extracted walking waveform data for one step cycle to a walking cycle of 0 to 100% (percent). The timing of 1%, 10%, etc. included in the 0 to 100% walking cycle is also called a walking phase. Furthermore, the parameter calculation unit 35 normalizes (second normalization) the first-normalized walking waveform data for one step cycle so that the stance phase is 60% and the swing phase is 40%.

[0106] The parameter calculation unit 35 calculates gait parameters using the normalized gait waveform data. For example, the gait parameters are used to estimate a physical state. There are no particular limitations on the gait parameters calculated by the parameter calculation unit 35. For example, the parameter calculation unit 35 calculates gait parameters related to distance, height, angle, speed, time, CPEI (Center of Pressure Exclusion Index), frailty level, etc. The parameter calculation unit 35 may extract feature values ​​used to calculate and estimate the gait parameters from the gait waveform data.

[0107] The parameter calculation unit 35 outputs gait parameters including gait parameters and feature amounts to the estimation unit 36. The gait parameters calculated by the parameter calculation unit 35 may be selected according to the physical condition of the subject to be estimated. There are no particular limitations on the physical condition of the subject to be estimated. For example, the physical condition of the subject to be estimated may include physical ability, health status, disease risk, etc.

[0108] The estimation unit 36 ​​acquires gait parameters from the parameter calculation unit 35. The estimation unit 36 ​​estimates the user's physical state using the acquired gait parameters. The estimation unit 36 ​​estimates the physical state using an estimation model 350 that has learned the physical state. The estimation model 350 is a machine learning model trained by machine learning. For example, the estimation model 350 outputs information about the physical state to be estimated in response to input of gait parameters. For example, the information about the physical state is information indicating whether the user is in the physical state to be estimated. For example, the information about the physical state is an index indicating the degree, accuracy, probability, etc. of the likelihood of the user becoming in the physical state to be estimated. For example, the information about the physical state is an index indicating the degree, accuracy, probability, etc. of the likelihood of the user becoming in the physical state to be estimated.

[0109] For example, estimation model 350 is a physical ability estimation model that outputs an index (physical ability score) related to physical ability in response to input of gait parameters. For example, the physical abilities to be estimated include grip strength (total muscle strength), dynamic balance, lower limb muscle strength, mobility, static balance, etc. The physical ability estimation model outputs an index (score) of the physical ability to be estimated in response to input of gait parameters that represent the characteristics of the physical ability to be estimated.

[0110] For example, the estimation model 350 is a disease risk estimation model that estimates disease risk using attribute data, gait parameters, and physical ability scores. The disease risk indicates the risk of contracting a specific disease. For example, specific diseases include gout, diabetes, hypertension, nephrolithiasis, liver cirrhosis, arteriosclerosis, thromboembolism, dyslipidemia, hypercholesterolemia, hyperlipidemia, etc. For example, specific diseases include lower back pain, sleep apnea syndrome, insomnia, depression, osteoarthritis of the knee, Parkinson's syndrome, etc. The disease risk estimation model outputs a disease risk index that indicates the risk of contracting the specific disease to be estimated in response to input of gait parameters that represent the characteristics of the specific disease to be estimated.

[0111] For example, estimation model 350 is registered in gait measurement device 30 when the product is shipped from a factory. Estimation model 350 may also be registered in gait measurement device 30 when the gait measurement device 30 is calibrated. For example, estimation model 350 may be stored in a storage device (not shown) such as an external server. In this case, it is sufficient for gait measurement device 30 to be able to access estimation model 350 via an interface (not shown) connected to the Internet network.

[0112] For example, the estimation model 350 is a model trained on a data set using training data in which gait parameters of multiple subjects are used as explanatory variables and scores related to physical conditions are used as objective variables. The estimation model 350 may also be a model trained on a data set in which attributes and gait waveform data of multiple subjects are used as explanatory variables and scores related to physical conditions are used as objective variables. For example, the estimation model 350 may be a model trained on training data in which gait waveform data of accelerations in three axial directions, angular velocities around three axes, and angles around three axes (posture angles) are used as explanatory variables.

[0113] For example, the estimation model 350 may be generated by learning using a linear regression algorithm. For example, the estimation model 350 may be generated by learning using a support vector machine (SVM) algorithm. For example, the estimation model 350 may be generated by learning using a Gaussian process regression (GPR) algorithm. For example, the estimation model 350 may be generated by learning using a random forest (RF) algorithm. For example, the estimation model 350 may be generated by unsupervised learning that classifies the user's physical state according to input gait parameters. There are no particular limitations on the algorithm used to train the estimation model 350. For example, the estimation model 350 may be a machine learning model such as an incomplete heterogeneous variational autoencoder or a random forest. An incomplete heterogeneous variational autoencoder can estimate the user's physical state even if there are some deficiencies in attribute data, gait parameters, physical ability scores, etc.

[0114] For example, the estimation unit 36 ​​may be configured to generate recommendation information according to the physical condition. For example, the estimation unit 36 ​​may generate recommendation information by applying information according to the physical condition to a predetermined document format. For example, the estimation unit 36 ​​may generate recommendation information using a large-scale language model. The user can take appropriate action by making a decision according to the recommendation information.

[0115] The output unit 37 acquires information about the physical state estimated by the estimation unit 36. The output unit 37 outputs the acquired information about the physical state. For example, the output unit 37 may be configured to output the acquired information to a terminal device or a server in which software that uses the information about the physical state is implemented. For example, the output unit 37 may be configured to output the acquired information to an external system or the like that uses the information about the physical state. There are no particular limitations on the use of the information about the physical state.

[0116] (Operation) Next, the operation of the gait measurement device 30 according to this embodiment will be described with reference to the drawings. FIG. 23 is a flowchart for explaining an example of the operation of the gait measurement device according to the present disclosure. In the explanation of the processing of the flowchart of FIG. 23, the components of the gait measurement device 30 will be described as the subject of the operations. The subject of the operations of the processing according to the flowchart of FIG. 23 may be the gait measurement device 30.

[0117] In FIG. 23, first, the video acquisition unit 31 acquires video data (step S31).

[0118] Next, the movement recognition unit 32 converts the walking movement of the person included in the video data into a skeleton model (step S32).

[0119] Next, the action recognition unit 32 extracts the spatial coordinates of the target part in the transformed skeletal model (step S33).

[0120] Next, the walking data generating unit 33 generates walking data using the spatial coordinates of the extracted target portion (step S34).

[0121] Next, the parameter calculation unit 35 calculates gait parameters using the generated walking data (step S35).

[0122] Next, the estimation unit 36 ​​estimates the body state using the estimated gait parameters (step S36).

[0123] Next, the output unit 37 outputs information relating to the estimated physical condition (step S37).

[0124] (Application Example) Next, an application example according to the present embodiment will be described with reference to the drawings. In the following application example, information about the user's physical condition estimated by the gait measurement device 30 is displayed on the screen of a mobile terminal used by the user.

[0125] FIG. 24 is a conceptual diagram showing an example of displaying the estimation results by gait measurement device 30 on the screen of mobile device 370 used by the user. FIG. 24 shows an example in which information corresponding to the physical condition estimated by gait measurement device 30 is displayed on the screen of mobile device 370. In the example of FIG. 24 , information related to the estimated physical condition, such as "Dynamic balance is deteriorating," is displayed on the screen of mobile device 370. In addition, in the example of FIG. 24 , recommendation information corresponding to the estimated physical condition, such as "Training M is recommended. Please watch the video below," is displayed on the screen of mobile device 370. The user who has confirmed the information displayed on the screen of mobile device 370 can practice training that will lead to improvement of their physical condition by exercising by referring to the video of Training M in accordance with the recommendation information.

[0126] As described above, the gait measurement device includes a video acquisition unit, a movement recognition unit, a walking data generation unit, a parameter calculation unit, an estimation unit, and an output unit. The video acquisition unit acquires video data. The movement recognition unit extracts spatial coordinates of target portions constituting the skeleton of a person depicted in the video data. In response to input of the video data, the movement recognition unit converts the video data into skeletal data including target portions of the person depicted in the video data using a movement recognition model that outputs skeletal data of at least one person depicted in the video data. For example, the movement recognition model is a machine learning model constructed using a machine learning technique. The walking data generation unit generates gait data corresponding to foot movement using the spatial coordinates of the extracted target portions. The parameter calculation unit calculates gait parameters used to estimate a physical state. The estimation unit estimates a physical state using the gait parameters. The output unit outputs information related to the estimated physical state.

[0127] The gait measurement device of this embodiment uses a motion recognition model to convert video data into skeletal data including target parts of a person. The gait measurement device of this embodiment uses spatial coordinates of the target parts included in the skeletal data to generate gait data corresponding to foot movements. The gait measurement device of this embodiment calculates gait parameters using the generated gait data. Furthermore, the gait measurement device of this embodiment estimates the user's physical state using the estimated gait parameters. A large amount of video data containing people can be collected. Therefore, according to this embodiment, a physical state can be estimated using highly accurate gait parameters generated using a large amount of collected video data.

[0128] Fourth Embodiment Next, a gait measurement device according to a fourth embodiment will be described with reference to the drawings. The gait measurement device according to this embodiment has a simplified configuration of the gait measurement devices according to the first to third embodiments.

[0129] (Configuration) Fig. 1 is a block diagram showing an example of the configuration of a gait measurement device according to the present disclosure. Gait measurement device 40 includes an image acquisition unit 41, an action recognition unit 42, a walking data generation unit 43, and an output unit 47. Image acquisition unit 41 acquires image data. Action recognition unit 42 extracts spatial coordinates of target parts constituting the skeleton of a person shown in the image data. Walking data generation unit 43 generates walking data corresponding to foot movements using the spatial coordinates of the extracted target parts. Output unit 47 outputs the generated walking data.

[0130] (Operation) Next, the operation of gait measurement device 40 will be described with reference to the drawings. FIG. 26 is a flowchart illustrating an example of the operation of gait measurement device 40. In describing the processing according to the flowchart of FIG. 26, the components of gait measurement device 40 will be described as the actors performing the operations. The actor performing the processing according to the flowchart of FIG. 26 may be gait measurement device 40.

[0131] In FIG. 26, first, the video acquisition unit 41 acquires video data (step S41).

[0132] Next, the action recognition unit 42 extracts the spatial coordinates of target parts that make up the skeleton of the person shown in the video data (step S42).

[0133] Next, the walking data generating unit 43 generates walking data according to the movement of the feet using the spatial coordinates of the extracted target portion (step S43).

[0134] Next, the output unit 47 outputs the generated walking data (step S44).

[0135] The gait measurement device of this embodiment generates gait data corresponding to foot movements using spatial coordinates of target parts constituting the skeleton of a person appearing in video data. A large amount of video data showing people can be collected. Therefore, according to this embodiment, highly accurate gait data can be generated using a large amount of collected video data.

[0136] (Hardware) Next, a hardware configuration for executing the control and processing in the present disclosure will be described with reference to the drawings. Here, an information processing device 90 (computer) in Fig. 27 is given as an example of such a hardware configuration. The information processing device 90 in Fig. 27 is an example configuration for executing the control and processing in the present disclosure and does not limit the scope of the present disclosure.

[0137] 27 , an information processing device 90 includes a processor 91, a memory 92, an auxiliary storage device 93, an input / output interface 95, and a communication interface 96. In FIG. 27 , interface is abbreviated as I / F (Interface). The processor 91, memory 92, auxiliary storage device 93, input / output interface 95, and communication interface 96 are connected to each other via a bus 98 so as to be able to communicate data with each other. The processor 91, memory 92, auxiliary storage device 93, and input / output interface 95 are also connected to a network such as the Internet or an intranet via the communication interface 96.

[0138] The processor 91 loads a program (instructions) stored in an auxiliary storage device 93 or the like into the memory 92. For example, the program is a software program for executing the control and processing in the present disclosure. The processor 91 executes the program loaded into the memory 92. The processor 91 executes the program to execute the control and processing in the present disclosure.

[0139] The memory 92 is a storage device having an area in which a program is loaded. The processor 91 loads a program stored in an auxiliary storage device 93 or the like into the memory 92. The memory 92 is realized by a volatile memory such as a dynamic random access memory (DRAM). Alternatively, a non-volatile memory such as a magnetoresistive random access memory (MRAM) may be used as the memory 92.

[0140] The auxiliary storage device 93 stores various data such as programs. For example, the auxiliary storage device 93 is realized by a local disk such as a hard disk or flash memory. Note that it is also possible to configure the system so that various data is stored in the memory 92, thereby omitting the auxiliary storage device 93.

[0141] The input / output interface 95 is an interface for connecting the information processing device 90 to peripheral devices based on standards and specifications. The communication interface 96 is an interface for connecting to external systems and devices via a network such as the Internet or an intranet based on standards and specifications. The input / output interface 95 and the communication interface 96 may be a common interface for connecting to external devices.

[0142] Input devices such as a keyboard, mouse, and touch panel may be connected to the information processing device 90 as needed. These input devices are used to input information and settings. When a touch panel is used as the input device, a screen having the function of the touch panel serves as the interface. The processor 91 and the input devices are connected via an input / output interface 95.

[0143] The information processing device 90 may be equipped with a display device for displaying information. When the display device is equipped, the information processing device 90 is equipped with a display control device (not shown) for controlling the display of the display device. The information processing device 90 and the display device are connected via an input / output interface 95.

[0144] The information processing device 90 may be equipped with a drive device. The drive device acts as an intermediary between the processor 91 and a recording medium (program recording medium) to read data and programs stored on the recording medium and to write processing results of the information processing device 90 to the recording medium. The information processing device 90 and the drive device are connected via an input / output interface 95.

[0145] The above is an example of a hardware configuration for enabling the control and processing of the present disclosure. The hardware configuration of Fig. 27 is an example of a hardware configuration for executing the control and processing of the present disclosure and does not limit the scope of the present disclosure. A program that causes a computer to execute the control and processing of the present disclosure is also included in the scope of the present disclosure.

[0146] A program recording medium on which a program for executing the processing of this embodiment is recorded is also included within the scope of the present invention. For example, the program recording medium is a computer-readable, non-transitory recording medium. The recording medium can be, for example, an optical recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). The recording medium may also be a semiconductor recording medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card. The recording medium may also be a magnetic recording medium such as a flexible disk, or other recording medium.

[0147] The components in the present disclosure may be combined in any manner. The components in the present disclosure may be realized by software. The components in the present disclosure may be realized by circuits.

[0148] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0149] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. In the supplementary notes below, dependent claims in each category may also be made dependent on other categories. The statements included in the supplementary notes below have significance as grounds for amendment. (Supplementary Note 1) A gait measurement device comprising: a video acquisition unit that acquires video data; a motion recognition unit that extracts spatial coordinates of target parts constituting the skeleton of a person captured in the video data; a gait data generation unit that generates gait data corresponding to foot movement using the spatial coordinates of the extracted target parts; and an output unit that outputs the generated gait data. (Supplementary Note 2) The gait measurement device according to Supplementary Note 1, wherein the motion recognition unit converts, in response to input of the video data, the video data into skeletal data including the target parts of the people captured in the video data using a motion recognition model that outputs skeletal data of at least one person captured in the video data. (Supplementary Note 3) The gait measurement device according to Supplementary Note 2, wherein the motion recognition model is a machine learning model constructed using a machine learning technique. (Supplementary Note 4) The gait measurement device according to Supplementary Note 2, wherein the action recognition unit calculates, as the gait data, spatial acceleration, spatial angular velocity, and spatial angle indicating foot movement using the skeletal data of the person extracted from the video data. (Supplementary Note 5) The gait measurement device according to any one of Supplements 1 to 4, comprising a parameter calculation unit that calculates gait parameters using the gait data, and the output unit outputs the calculated gait parameters. (Supplementary Note 6) The gait measurement device according to Supplementary Note 5, wherein the parameter calculation unit calculates the gait parameters used to estimate a physical state. (Supplementary Note 7) The gait measurement device according to Supplementary Note 6, comprising an estimation unit that estimates a physical state using the gait parameters, and the output unit outputs information related to the estimated physical state. (Supplementary Note 8) The gait measurement device according to Supplementary Note 6, wherein the output unit is optimized according to the physical state of a user, and displays information prompting the user to make a decision on a screen of a mobile terminal used by the user.(Supplementary Note 9) A gait measurement method in which a computer acquires video data, extracts spatial coordinates of target parts constituting the skeleton of a person captured in the video data, generates gait data corresponding to foot movement using the extracted spatial coordinates of the target parts, and outputs the generated gait data. (Supplementary Note 10) The gait measurement method according to Supplementary Note 9, in which a motion recognition model that outputs skeletal data of at least one person captured in the video data in response to input of the video data is used to convert the skeletal data into skeletal data including the target parts of the people captured in the video data. (Supplementary Note 11) The gait measurement method according to Supplementary Note 10, in which the motion recognition model is a machine learning model constructed using a machine learning technique. (Supplementary Note 12) The gait measurement method according to Supplementary Note 10, in which the skeletal data of the person extracted from the video data is used to calculate, as the gait data, spatial acceleration, spatial angular velocity, and spatial angle indicating foot movement. (Supplementary Note 13) A gait measurement method according to any one of Supplementary Notes 9 to 12, which calculates gait parameters used to estimate a physical state using the gait data, and outputs the calculated gait parameters. (Supplementary Note 14) A gait measurement method according to Supplementary Note 13, which estimates a physical state using the gait parameters, and outputs information related to the estimated physical state. (Supplementary Note 15) A computer-readable, non-transitory recording medium having recorded thereon a program causing a computer to execute the following processes: acquiring video data; extracting spatial coordinates of target parts constituting the skeleton of a person depicted in the video data; generating gait data corresponding to foot movement using the extracted spatial coordinates of the target parts; and outputting the generated gait data. (Supplementary Note 16) A computer-readable, non-transitory recording medium having recorded thereon the program according to Supplementary Note 15, which converts the video data into skeletal data including the target parts of the people depicted in the video data using an action recognition model that outputs skeletal data of at least one person depicted in the video data in response to input of the video data. (Supplementary Note 17) A computer-readable non-transitory recording medium having recorded thereon the program according to Supplementary Note 16, wherein the action recognition model is a machine learning model constructed using a machine learning technique.(Supplementary Note 18) A computer-readable, non-transitory recording medium having recorded thereon the program according to Supplementary Note 16, which calculates, as the gait data, spatial accelerations, spatial angular velocities, and spatial angles that indicate foot movements, using the skeletal data of the person extracted from the video data. (Supplementary Note 19) A computer-readable, non-transitory recording medium having recorded thereon the program according to any one of Supplements 15 to 18, which calculates gait parameters used for estimating a physical state using the gait data, and outputs the calculated gait parameters. (Supplementary Note 20) A computer-readable, non-transitory recording medium having recorded thereon the program according to Supplementary Note 19, which estimates a physical state using the gait parameters, and outputs information about the estimated physical state. Furthermore, some or all of the configurations described in Supplementary Notes 2 to 8 that are subordinate to Supplementary Note 1 above may also be subordinate to Supplementary Note 9 and Supplementary Note 20 in the same subordinate relationship as Supplementary Notes 2 to 8. Furthermore, not limited to Appendix 1, Appendix 9, and Appendix 15, but within the scope of each of the above-mentioned embodiments, some or all of the configurations described as appendices may be subordinated to various hardware, software, various recording means for recording software, or systems.

[0150] 10, 20, 30, 40 Gait measurement device 11, 21, 31, 41 Image acquisition unit 12, 22, 32, 42 Action recognition unit 13, 23, 33, 43 Walking data generation unit 17, 27, 37, 47 Output unit 25, 35 Parameter calculation unit 110 Measurement device 120, 220, 320 Action recognition model 350 Estimation model

Claims

1. A gait measurement device comprising: a video acquisition unit that acquires video data; a motion recognition unit that extracts spatial coordinates of a target part that constitutes the skeleton of a person shown in the video data; a gait data generation unit that generates gait data corresponding to the movement of the feet using the extracted spatial coordinates of the target part; and an output unit that outputs the generated gait data.

2. The gait measurement device according to claim 1, wherein the motion recognition unit uses a motion recognition model that outputs skeleton data of at least one person shown in the video data in response to the input of the video data, and converts the video data into the skeleton data including the target part of the person shown in the video data.

3. The gait measurement device according to claim 2, wherein the motion recognition model is a machine learning model constructed using a machine learning method.

4. The gait measurement device according to claim 2, wherein the motion recognition unit calculates, as the gait data, a spatial acceleration, a spatial angular velocity, and a spatial angle indicating the movement of the feet using the skeleton data of the person extracted from the video data.

5. The gait measurement device according to any one of claims 1 to 4, further comprising a parameter calculation unit that calculates gait parameters using the gait data, and the output unit outputs the calculated gait parameters.

6. The gait measurement device according to claim 5, wherein the parameter calculation unit calculates the gait parameters used for estimating the physical state.

7. The gait measurement device according to claim 6, further comprising an estimation unit that estimates the physical state using the gait parameters, and the output unit outputs information regarding the estimated physical state.

8. The gait measurement device according to claim 6, wherein the output unit displays, on a screen of a mobile terminal used by the user, information that is optimized according to the physical state of the user and prompts the user to make a decision.

9. A gait measurement method in which a computer acquires video data, extracts spatial coordinates of a target part that constitutes the skeleton of a person shown in the video data, generates gait data corresponding to the movement of the feet using the extracted spatial coordinates of the target part, and outputs the generated gait data.

10. The gait measurement method according to claim 9, wherein, in response to the input of the video data, the video data is converted into the skeletal data including the target part of the person shown in the video data by using an action recognition model that outputs the skeletal data of at least one person shown in the video data.

11. The gait measurement method according to claim 10, wherein the action recognition model is a machine learning model constructed by using a machine learning method.

12. The gait measurement method according to claim 10, wherein using the skeletal data of the person extracted from the video data, the spatial acceleration, spatial angular velocity, and spatial angle indicating the movement of the foot are calculated as the walking data.

13. The gait measurement method according to any one of claims 9 to 12, wherein using the walking data, gait parameters used for estimating the physical state are calculated, and the calculated gait parameters are output.

14. The gait measurement method according to claim 13, wherein the physical state is estimated using the gait parameters, and information regarding the estimated physical state is output.

15. A computer-readable non-transitory recording medium recording a program for causing a computer to execute a process of acquiring video data, a process of extracting the spatial coordinates of a target part constituting the skeleton of a person shown in the video data, a process of generating walking data corresponding to the movement of the foot using the extracted spatial coordinates of the target part, and a process of outputting the generated walking data.

16. A computer-readable non-transitory recording medium recording the program according to claim 15, wherein, in response to the input of the video data, the video data is converted into the skeletal data including the target part of the person shown in the video data by using an action recognition model that outputs the skeletal data of at least one person shown in the video data.

17. A computer-readable non-transitory recording medium recording the program according to claim 16, wherein the action recognition model is a machine learning model constructed by using a machine learning method.

18. A computer-readable non-transitory recording medium recording the program according to claim 16, wherein using the skeletal data of the person extracted from the video data, the spatial acceleration, spatial angular velocity, and spatial angle indicating the movement of the foot are calculated as the walking data.

19. A computer-readable non-transitory recording medium recording the program according to any one of claims 15 to 18, which calculates gait parameters used for estimating a physical state using the walking data and outputs the calculated gait parameters.

20. A computer-readable non-transitory recording medium recording the program according to claim 19, which estimates a physical state using the gait parameters and outputs information regarding the estimated physical state.

Images