Harmony index estimation device, estimation system, harmony index estimation method, and program

Abstract

To provide a harmonization index estimation device, etc. capable of estimating a harmonization index on smoothness of a motion of the lumbar in the daily life accurately and easily.SOLUTION: A harmonization index estimation device includes: a communication unit for acquiring feature amount data including a feature amount used for estimating a harmonization index on smoothness of a motion of the lumbar, extracted from a walking waveform of a space acceleration and a space angular velocity included in sensor data on a motion of the foot of a subject; a storage unit for storing an estimation model for outputting an estimation value on the harmonization index in response to input of the feature amount included in the feature amount data; an estimation unit for inputting the feature amount included in the acquired feature amount data to the estimation model, and estimating the harmonization index of the subject according to the estimation value on the harmonization index output from the estimation model; and an output unit for outputting information according to the harmonization index of the subject.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a harmony index estimation device and the like that estimates a harmony index related to the smoothness of hip movement. [Background technology]

[0002] With growing interest in healthcare, services that provide information based on gait are gaining attention. For example, technology has been developed to analyze gait using sensor data measured by sensors mounted on footwear such as shoes. Time-series data from sensor data contains features associated with walking events related to physical conditions. By analyzing walking data that includes features associated with walking events, the subject's physical condition can be estimated. The harmonic ratio, which relates to the smoothness of hip movement during walking, is an index that indicates the sway and movement of the hips. If the harmonic ratio can be estimated with high accuracy by analyzing walking data, services that meet healthcare needs can be provided.

[0003] Patent Document 1 discloses a gait assessment device that evaluates a user's walking ability. The device in Patent Document 1 calculates multiple gait indices related to the walking state using multiple gait data acquired from a subject. The method in Patent Document 1 calculates the subject's gait score using gait data acquired by an acceleration sensor attached to the subject's waist. The method in Patent Document 1 calculates a harmonic ratio, which is one of the gait indices, from acceleration waveforms in the vertical, lateral, and front-to-back directions measured by the acceleration sensor attached to the subject's waist.

[0004] Patent Document 2 discloses a calculation device that calculates the step lengths of both the left and right feet using sensor data based on foot movements measured by sensors attached to the feet of a walker. The device in Patent Document 2 calculates the step lengths of both the left and right feet according to the walking event timings that appear in the walking waveform of the forward acceleration and the forward trajectory.

[0005] Patent Document 3 discloses a system for monitoring a person's rhythmic movement based on signals transmitted and received via a wireless multipath channel. Patent Document 3 also discloses that a person's gait is recognized as a rhythmic movement. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2020-151470 [Patent Document 2] International Publication No. 2022 / 038664 [Patent Document 3] Japanese Patent Publication No. 2020-144115 Summary of the Invention [Problem to be solved by the invention]

[0007] The method of Patent Document 1 uses acceleration measured by an acceleration sensor attached to the waist to measure one of the indices relating to the smoothness of waist movement. In daily life, a waist-attached sensor may restrict free movement. Furthermore, if the attached sensor is misaligned, the measurement accuracy decreases. Therefore, the method of Patent Document 1 cannot easily and accurately measure an index relating to the smoothness of waist movement in daily life.

[0008] The method of Patent Document 2 calculates the step lengths of both the left and right feet using sensor data based on foot movements. Patent Document 2 does not disclose estimating a harmony index using sensor data based on foot movements.

[0009] The method of Patent Document 3 recognizes rhythmic movements such as a person's gait based on signals transmitted and received via a wireless multipath channel. Therefore, the method of Patent Document 3 cannot recognize gait unless a wireless multipath channel is available.

[0010] An object of the present disclosure is to provide a harmony index estimation device and the like that can easily and accurately estimate a harmony index related to the smoothness of hip movement in daily life. [Means for solving the problem]

[0011] A harmony index estimation device according to one aspect of the present disclosure includes a communication unit that acquires feature data including features used to estimate a harmony index related to the smoothness of hip movement, the features being extracted from walking waveforms of spatial acceleration and spatial angular velocity included in sensor data related to the movement of the feet of a subject; a memory unit that stores an estimation model that outputs an estimated value related to the harmony index in response to input of the features included in the feature data; an estimation unit that inputs the features included in the acquired feature data into the estimation model and estimates the harmony index of the subject in response to the estimated value related to the harmony index output from the estimation model; and an output unit that outputs information related to the harmony index of the subject.

[0012] In a harmony index estimation method according to one aspect of the present disclosure, feature data including features used to estimate a harmony index related to the smoothness of hip movement is acquired, the feature data being extracted from the walking waveforms of spatial acceleration and spatial angular velocity included in sensor data related to the movement of the subject's feet; an estimation model that outputs an estimated value related to the harmony index in response to input of the features included in the feature data is stored; the features included in the acquired feature data are input into the estimation model; the harmony index of the subject is estimated in response to the estimated value related to the harmony index output from the estimation model; and information related to the subject's harmony index is output.

[0013] A program according to one aspect of the present disclosure causes a computer to perform the following processes: acquiring feature data including features used to estimate a harmony index related to the smoothness of hip movement, extracted from walking waveforms of spatial acceleration and spatial angular velocity included in sensor data related to the movement of the subject's feet; storing an estimation model that outputs an estimated value related to the harmony index in response to input of the features included in the feature data; inputting the features included in the acquired feature data into the estimation model and estimating the subject's harmony index in response to the estimated value related to the harmony index output from the estimation model; and outputting information related to the subject's harmony index. [Effects of the Invention]

[0014] According to the present disclosure, it is possible to provide a harmony index estimation device and the like that can easily and accurately estimate a harmony index related to the smoothness of hip movement in daily life. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram showing an example of the configuration of an estimation system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a measurement device included in the estimation system according to the first embodiment. [Figure 3] FIG. 2 is a conceptual diagram showing an example of the arrangement of a measurement device according to the first embodiment. [Figure 4] FIG. 2 is a conceptual diagram for explaining an example of the relationship between a local coordinate system and a world coordinate system set in the measurement apparatus according to the first embodiment. [Figure 5] FIG. 2 is a conceptual diagram for explaining a human body surface used in the explanation of the measurement device according to the first embodiment. [Figure 6] FIG. 1 is a conceptual diagram for explaining a walking cycle used in explaining the gait measurement device according to the first embodiment. [Figure 7] 4 is a graph for explaining an example of time-series data of sensor data measured by the measurement device according to the first embodiment. [Figure 8]FIG. 4 is a diagram for explaining an example of normalization of gait waveform data extracted from time-series data of sensor data measured by the measurement device according to the first embodiment. [Figure 9] 3 is a conceptual diagram for explaining a feature amount cluster from which a feature amount data generating unit of the measurement apparatus according to the first embodiment extracts feature amounts. FIG. [Figure 10] 1 is a block diagram showing an example of the configuration of a harmony index estimation device included in the estimation system according to the first embodiment. [Figure 11] FIG. 2 is a conceptual diagram illustrating learning of an estimation model used by a harmony index estimation device included in the estimation system according to the first embodiment. [Figure 12] FIG. 2 is a conceptual diagram illustrating estimation using an estimation model by a harmonization index estimation device included in the estimation system according to the first embodiment. [Figure 13] 5 is a flowchart for explaining an example of the operation of a measurement device included in the estimation system according to the first embodiment. [Figure 14] 5 is a flowchart illustrating an example of the operation of the harmonization index estimation device included in the estimation system according to the first embodiment. [Figure 15] FIG. 2 is a conceptual diagram for explaining an application example of the estimation system according to the first embodiment. [Figure 16] FIG. 2 is a conceptual diagram for explaining an application example of the estimation system according to the first embodiment. [Figure 17] FIG. 10 is a block diagram showing an example of the configuration of an estimation system according to a second embodiment. [Figure 18] FIG. 10 is a block diagram showing an example of the configuration of a harmonization index estimation device included in the estimation system according to the second embodiment. [Figure 19] FIG. 10 is a conceptual diagram illustrating learning of an estimation model used by a harmony index estimation device included in an estimation system according to a second embodiment. [Figure 20] 10 is a graph showing an example of time series data of vertical acceleration of the lower back. [Figure 21]10 is a graph showing an example of time-series data of acceleration of the lower back in the traveling direction. [Figure 22] 10 is a graph showing an example of time-series data of lateral acceleration of the waist; [Figure 23] FIG. 10 is a conceptual diagram illustrating estimation using an estimation model by a harmonization index estimation device included in an estimation system according to a second embodiment. [Figure 24] FIG. 10 is a conceptual diagram illustrating calculation of a harmonization index by a harmonization index estimation device included in an estimation system according to a second embodiment. [Figure 25] 10 is a table summarizing an example of input data used for estimating a harmony index related to the vertical direction by the harmony index estimation device included in the estimation system according to the first embodiment. [Figure 26] 10 is a table summarizing an example of input data used for estimating a harmony index related to a traveling direction by the harmony index estimation device included in the estimation system according to the first embodiment. [Figure 27] 10 is a table summarizing an example of input data used for estimating a harmony index related to the left-right direction by the harmony index estimation device included in the estimation system according to the first embodiment. [Figure 28] 10 is a flowchart illustrating an example of the operation of the harmonization index estimation device included in the estimation system according to the second embodiment. [Figure 29] 10 is a flowchart illustrating another example of the operation of the harmonization index estimation device included in the estimation system according to the second embodiment. [Figure 30] FIG. 10 is a block diagram showing an example of the configuration of a harmony index estimation device according to a third embodiment. [Figure 31] FIG. 2 is a block diagram illustrating an example of a hardware configuration for executing the processes of each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, the embodiments described below are limited in a manner that is technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following. In all drawings used to describe the following embodiments, the same reference numerals are used for similar parts unless otherwise specified. Furthermore, in the following embodiments, repeated explanations of similar configurations and operations may be omitted.

[0017] (First embodiment) First, an estimation system according to a first embodiment will be described with reference to the drawings. The estimation system of this embodiment uses a measurement device mounted on footwear to measure sensor data related to foot movement in response to a user's walking. The estimation system of this embodiment uses the measured sensor data to estimate a harmony index related to the smoothness of hip movement. The harmony index is also an index of harmony in walking.

[0018] In this embodiment, an example of estimating the Harmonic Ratio (hereinafter abbreviated as HR) of the lower back is given as the harmonic index. HR is an index that takes into account the fact that one walking cycle is made up of two cycles of acceleration change, including one cycle for one step of the right foot and one cycle for one step of the left foot. HR is calculated using frequency components obtained by Fourier transforming time-series data of the acceleration of the lower back in one walking cycle. The frequency components include even-numbered frequency components (even harmonics) that correspond to elements in the walking cycle, and odd-numbered frequency components (odd harmonics) that are elements that deviate from these. The even-numbered frequency components are also called even components. The odd-numbered frequency components are also called odd components.

[0019] The calculation method for HR differs depending on the direction of walking. HR in the vertical and forward direction is the ratio of the power sum of the even components to the power sum of the odd components. On the other hand, HR in the left-right direction is the ratio of the power sum of the odd components to the power sum of the even components, since one step (two steps) of each foot in one walking cycle constitutes one cycle.

[0020] Vertical HR (VT) and longitudinal HR (AP) are calculated using Equation 1 below.

number

[0021] The left-right HR(ML) is calculated using Equation 2 below.

number

[0022] The numerator / denominator of the above equation 1-2 is the power sum of the even or odd components in one walking cycle.

[0023] The left and right feet are connected to the pelvis via the lower legs and thighs. The hip and knee joints are located between the left and right feet and the pelvis, but the periodicity of the pelvis and lower back during walking is similar. Therefore, there are phases in which the movements of the left and right feet and the movements of the lower back are linked to each other. In this embodiment, a harmony index related to the smoothness of lower back movement is estimated using sensor data related to the movements of the feet.

[0024] (composition) FIG. 1 is a block diagram showing an example of the configuration of an estimation system 1 according to this embodiment. The estimation system 1 includes a measurement device 10 and a harmony index estimation device 13. In this embodiment, an example will be described in which the measurement device 10 and the harmony index estimation device 13 are configured as separate pieces of hardware. For example, the measurement device 10 is installed in footwear or the like of a subject (user) who is the subject of harmony index estimation. For example, the functions of the harmony index estimation device 13 are installed in a mobile terminal carried by the subject (user). Below, the configurations of the measurement device 10 and the harmony index estimation device 13 will be described separately.

[0025] [Measuring equipment] 2 is a block diagram showing an example of the configuration of the measurement device 10. The measurement device 10 has a sensor 11 and a feature data generation unit 12. In this embodiment, an example is given in which the sensor 11 and the feature data generation unit 12 are integrated. The sensor 11 and the feature data generation unit 12 may also be provided as separate devices.

[0026] 2, the sensor 11 has an acceleration sensor 111 and an angular velocity sensor 112. Fig. 2 shows an example in which the acceleration sensor 111 and the angular velocity sensor 112 are included in the sensor 11. The sensor 11 may include sensors other than the acceleration sensor 111 and the angular velocity sensor 112. A description of sensors other than the acceleration sensor 111 and the angular velocity sensor 112 that may be included in the sensor 11 will be omitted.

[0027] The acceleration sensor 111 is a sensor that measures acceleration in three axial directions (also called spatial acceleration). The acceleration sensor 111 measures acceleration (also called spatial acceleration) as a physical quantity related to foot movement. The acceleration sensor 111 outputs the measured acceleration to the feature data generation unit 12. For example, a piezoelectric, piezo-resistive, or capacitive sensor can be used as the acceleration sensor 111. There is no limitation on the measurement method of the sensor used as the acceleration sensor 111 as long as it can measure acceleration.

[0028] The angular velocity sensor 112 is a sensor that measures angular velocity (also called spatial angular velocity) around three axes. The angular velocity sensor 112 measures the angular velocity (also called spatial angular velocity) as a physical quantity related to foot movement. The angular velocity sensor 112 outputs the measured angular velocity to the feature data generation unit 12. For example, a vibration type or capacitance type sensor can be used as the angular velocity sensor 112. There is no limitation on the measurement method of the sensor used as the angular velocity sensor 112 as long as it can measure angular velocity.

[0029] The sensor 11 is realized by, for example, an inertial measurement unit that measures acceleration and angular velocity. An example of an inertial measurement unit is an IMU (Inertial Measurement Unit). The IMU includes an acceleration sensor 111 that measures acceleration in three axial directions and an angular velocity sensor 112 that measures angular velocity around three axes. The sensor 11 may be realized by an inertial measurement unit such as a VG (Vertical Gyro) or an AHRS (Attitude Heading). The sensor 11 may also be realized by a GPS / INS (Global Positioning System / Inertial Navigation System). The sensor 11 may be realized by a device other than an inertial measurement unit as long as it can measure physical quantities related to foot movement.

[0030] FIG. 3 is a conceptual diagram showing an example in which the measurement device 10 is placed inside the shoes 100 of both feet. In the example of FIG. 3, the measurement device 10 is placed at a position corresponding to the back of the arch of the foot. The measurement device 10 may be placed at a position other than the back of the arch of the foot, as long as it can measure sensor data related to foot movement. For example, the measurement device 10 may be placed in an insole inserted into the shoe 100. The measurement device 10 may also be placed on the bottom of the shoe 100. The measurement device 10 may also be embedded in the body of the shoe 100. The measurement device 10 may be detachable from the shoe 100 or may not be detachable from the shoe 100. The measurement device 10 may also be placed in socks worn by the user or in an accessory such as an anklet worn by the user. The measurement device 10 may also be attached directly to the foot or embedded in the foot. FIG. 3 shows an example in which the measurement device 10 is placed in the shoes 100 of both feet. The measurement device 10 may also be placed in the shoe 100 of one foot.

[0031] In the example of FIG. 3 , a local coordinate system is set with the measuring device 10 (sensor 11) as the reference, and includes an x-axis in the left-right direction, a y-axis in the forward direction, and a z-axis in the vertical direction. The x-axis is positive to the left, the y-axis is positive backward, and the z-axis is positive upward. The directions of the axes set for the sensor 11 may be the same for the left and right feet, or may be different for the left and right feet. For example, when sensors 11 manufactured with the same specifications are placed in left and right shoes 100, the up-down directions (Z-axis directions) of the sensors 11 placed in the left and right shoes 100 are the same. In this case, the three axes of the local coordinate system set for the sensor data derived from the left foot and the three axes of the local coordinate system set for the sensor data derived from the right foot are the same for the left and right feet. When sensors 11 manufactured with different specifications for the left and right feet are placed in the shoes 100, the up-down directions (Z-axis directions) of the sensors 11 placed in the left and right shoes 100 may be different.

[0032] FIG. 4 is a conceptual diagram illustrating a local coordinate system (x-axis, y-axis, z-axis) set in the measurement device 10 (sensor 11) installed on the backside of the arch of the foot, and a world coordinate system (x-axis, y-axis, z-axis) set relative to the ground. In the world coordinate system (x-axis, y-axis, z-axis), when a user is standing upright facing the direction of travel, the user's sideways direction is set as the x-axis direction (leftward is positive), the user's back direction is set as the y-axis direction (backward is positive), and the direction of gravity is set as the z-axis direction (vertically upward is positive). Note that the example in FIG. 4 conceptually illustrates the relationship between the local coordinate system (x-axis, y-axis, z-axis) and the world coordinate system (x-axis, y-axis, z-axis), and does not accurately illustrate the relationship between the local coordinate system and the world coordinate system, which changes depending on the user's walking.

[0033] FIG. 5 is a conceptual diagram illustrating planes (also referred to as human body planes) set for the human body. In this embodiment, a sagittal plane that divides the body into left and right, a coronal plane that divides the body into front and back, and a horizontal plane that divides the body horizontally are defined. As shown in FIG. 5, when the user is standing upright with the centerline of the feet pointing in the direction of travel, the world coordinate system and the local coordinate system coincide. In this embodiment, a rotation in the sagittal plane about the x-axis as the rotation axis is defined as roll, a rotation in the coronal plane about the y-axis as the rotation axis is defined as pitch, and a rotation in the horizontal plane about the z-axis as the rotation axis is defined as yaw. In addition, a rotation angle in the sagittal plane about the x-axis as the rotation axis is defined as roll angle, a rotation angle in the coronal plane about the y-axis as the rotation axis is defined as pitch angle, and a rotation angle in the horizontal plane about the z-axis as the rotation axis is defined as yaw angle. In the following, the x-axis, y-axis, and z-axis are referred to as three axes.

[0034] As shown in FIG. 2 , the feature data generation unit 12 (also referred to as a feature data generation device) includes an acquisition unit 121, a normalization unit 122, an extraction unit 123, a generation unit 125, and a transmission unit 127. For example, the feature data generation unit 12 is implemented by a microcomputer or microcontroller that performs overall control of the measurement device 10 and data processing. For example, the feature data generation unit 12 includes a central processing unit (CPU), random access memory (RAM), read-only memory (ROM), flash memory, etc. The feature data generation unit 12 controls the acceleration sensor 111 and the angular velocity sensor 112 to measure angular velocity and acceleration. For example, the feature data generation unit 12 may be implemented on a mobile device (not shown) carried by the subject (user). In this case, the sensor 11 may be provided with a communication function, and the sensor data transmitted from the sensor 11 may be received by the mobile device equipped with the feature data generation unit 12. When transmitting sensor data from the sensor 11, it is preferable that the sensor 11 has a stable walking detection function built in. In this way, the sensor 11 can be configured to transmit sensor data in response to detection of stable walking. For example, stable walking can be detected when acceleration in a specific direction exceeds a predetermined threshold.

[0035] The acquiring unit 121 acquires acceleration in three axial directions from the acceleration sensor 111. The acquiring unit 121 also acquires angular velocities around three axes from the angular velocity sensor 112. For example, the acquiring unit 121 performs analog-to-digital conversion (AD conversion) on the acquired physical quantities (analog data) such as angular velocities and accelerations. The physical quantities (analog data) measured by the acceleration sensor 111 and the angular velocity sensor 112 may be converted into digital data by the acceleration sensor 111 and the angular velocity sensor 112, respectively. The acquiring unit 121 outputs the converted digital data (also referred to as sensor data) to the normalizing unit 122. The acquiring unit 121 may be configured to store the sensor data in a storage unit (not shown). The sensor data includes at least acceleration data converted into digital data and angular velocity data converted into digital data. The acceleration data includes acceleration vectors in the three axial directions. The angular velocity data includes angular velocity vectors around three axes. The acceleration data and angular velocity data are associated with the time when the data was acquired. The acquiring unit 121 may also apply corrections to the acceleration data and angular velocity data, such as correction for mounting error, temperature correction, and linearity correction.

[0036] The normalization unit 122 acquires sensor data from the acquisition unit 121. The normalization unit 122 extracts time series data for one walking cycle (also referred to as walking waveform data) from the time series data of accelerations in three axial directions and angular velocities around three axes included in the sensor data. The normalization unit 122 normalizes the time of the extracted walking waveform data for one walking cycle to a walking cycle of 0 to 100% (percent) (also referred to as first normalization). Timings such as 1% and 10% included in the 0 to 100% walking cycle are also referred to as walking phases. The normalization unit 122 also normalizes the first normalized walking waveform data for one walking cycle so that the stance phase is 60% and the swing phase is 40% (also referred to as second normalization). The stance phase is a period when at least a part of the sole of the foot is in contact with the ground. The swing phase is a period when the sole of the foot is off the ground. If the walking waveform data is second-normalized, the influence of deviations in walking phase that may occur in each walking cycle can be reduced.

[0037] FIG. 6 is a conceptual diagram illustrating a step cycle based on the right foot. The step cycle based on the left foot is similar to that of the right foot. The horizontal axis of FIG. 6 represents one step cycle of the right foot, starting from the point when the heel of the right foot hits the ground and ending from the point when the heel of the right foot hits the ground again. The horizontal axis of FIG. 6 is first normalized so that the step cycle is 100%. The horizontal axis of FIG. 6 is also second normalized so that the stance phase is 60% and the swing phase is 40%. One step cycle of one leg is roughly divided into a stance phase in which at least a portion of the sole of the foot is in contact with the ground, and a swing phase in which the sole of the foot is off the ground. The stance phase is further divided into a load response period T1, a mid-stance period T2, a final stance period T3, and an early swing period T4. The swing phase is further divided into an early swing period T5, a mid-swing period T6, and a final swing period T7. Note that FIG. 6 is an example and does not limit the periods that make up a walking cycle or the names of those periods.

[0038] As shown in Figure 6, multiple events (also called walking events) occur during walking. E1 represents the event in which the heel of the right foot touches the ground (heel contact: HC). E2 represents the event in which the toe of the left foot leaves the ground (opposite toe off: OTO) while the sole of the right foot is in contact with the ground. E3 represents the event in which the heel of the right foot rises (heel rise: HR) while the sole of the right foot is in contact with the ground (opposite heel strike: OHS). E5 represents the event in which the toe of the right foot leaves the ground (toe off: TO) while the sole of the left foot is in contact with the ground (toe off: TO). E6 represents the event in which the left and right feet cross (foot crossing: FA) while the sole of the left foot is in contact with the ground (foot adjacent). E7 represents the event where the tibia of the right foot is nearly perpendicular to the ground (Tibia Vertical: TV) with the sole of the left foot touching the ground. E8 represents the event where the heel of the right foot touches the ground (Heel Contact: HC). E8 corresponds to the end point of the walking cycle that begins with E1 and also corresponds to the starting point of the next walking cycle. Note that Figure 6 is an example and does not limit the events that occur during walking or the names of those events.

[0039] FIG. 7 is a diagram illustrating an example of detecting heel strike HC and toe-off TO from time-series data (solid line) of forward acceleration (Y-direction acceleration). The timing of heel strike HC is the timing of the minimum peak immediately after the maximum peak that appears in the time-series data of forward acceleration (Y-direction acceleration). The maximum peak that marks the timing of heel strike HC corresponds to the maximum peak of the gait waveform data for one step cycle. The section between consecutive heel strikes HC is one step cycle. The timing of toe-off TO is the timing of the rise of the maximum peak that appears after the stance phase period in which no fluctuations appear in the time-series data of forward acceleration (Y-direction acceleration). FIG. 8 also shows time-series data (dashed line) of roll angle (angular velocity around the X-axis). The midpoint between the timing of the minimum roll angle and the timing of the maximum roll angle is the timing T of the transition from mid-stance phase T2 to end-stance phase T3. mThe timing of the transition from mid-stance T2 to end-stance T3 is m The parameters used to estimate the body state (also called gait parameters) are the timing T m can be calculated based on the above.

[0040] FIG. 8 is a diagram illustrating an example of normalization of gait waveform data. The normalization unit 122 detects heel strikes HC and toe lifts TO from time-series data of forward acceleration (Y-direction acceleration). The normalization unit 122 extracts the interval between successive heel strikes HC as gait waveform data for one step cycle. The normalization unit 122 converts the horizontal axis (time axis) of the gait waveform data for one step cycle into a gait cycle of 0 to 100% by first normalization. In FIG. 9, the gait waveform data after the first normalization is shown by the dashed line. In the gait waveform data (dashed line) after the first normalization, the timing of toe lifts TO is shifted from 60%.

[0041] In the example of FIG. 8, the normalization unit 122 normalizes the section from heel-strike HC, where the walking phase is 0%, to toe-off TO, which follows the heel-strike HC, to 0-60%. The normalization unit 122 also normalizes the section from toe-off TO to heel-strike HC, where the walking phase is 100%, to 60-100%. As a result, the gait waveform data for one step cycle is normalized into a section where the gait cycle is 0-60% (stance phase) and a section where the gait cycle is 60-100% (swing phase). In FIG. 9, the gait waveform data after the second normalization is shown by a solid line. In the gait waveform data after the second normalization (solid line), the timing of toe-off TO coincides with 60%. For example, when verifying the respective durations of the stance phase and swing phase and their ratios, the second normalization may be omitted.

[0042] 7 and 8 show an example in which gait waveform data for one step cycle is extracted and normalized based on the traveling acceleration (Y-direction acceleration). With respect to accelerations / angular velocities other than the traveling acceleration (Y-direction acceleration), the normalization unit 122 extracts and normalizes gait waveform data for one step cycle in accordance with the traveling acceleration (Y-direction acceleration) gait cycle. The normalization unit 122 may also generate time series data of angles around three axes by integrating time series data of angular velocities around three axes. In this case, the normalization unit 122 extracts and normalizes gait waveform data for one step cycle in accordance with the traveling acceleration (Y-direction acceleration) gait cycle, also with respect to angles around three axes.

[0043] The normalization unit 122 may extract / normalize the gait waveform data for one step gait cycle based on acceleration / angular velocity other than the forward acceleration (Y-direction acceleration) (not shown in the drawings). For example, the normalization unit 122 may detect heel strike HC and toe lift TO from time series data of vertical acceleration (Z-direction acceleration). The timing of heel strike HC is the timing of a steep minimum peak that appears in the time series data of vertical acceleration (Z-direction acceleration). At the timing of the steep minimum peak, the value of vertical acceleration (Z-direction acceleration) becomes almost zero. The minimum peak that marks the timing of heel strike HC corresponds to the minimum peak of the gait waveform data for one step gait cycle. The section between consecutive heel strikes HC is one step gait cycle. The timing of toe-off TO is the timing of an inflection point in the time-series data of vertical acceleration (Z-direction acceleration) where the vertical acceleration (Z-direction acceleration) gradually increases after passing through a section of small fluctuations following the maximum peak immediately after heel-contact HC. The normalization unit 122 may also extract / normalize the gait waveform data for one walking cycle based on both the forward acceleration (Y-direction acceleration) and the vertical acceleration (Z-direction acceleration). The normalization unit 122 may also extract / normalize the gait waveform data for one walking cycle based on acceleration, angular velocity, angle, etc. other than the forward acceleration (Y-direction acceleration) and the vertical acceleration (Z-direction acceleration).

[0044] The extraction unit 123 acquires walking waveform data for one step gait cycle normalized by the normalization unit 122. The extraction unit 123 extracts feature amounts used to estimate a harmony index from the walking waveform data for one step gait cycle. The extraction unit 123 extracts feature amounts (also referred to as cluster feature amounts) for each walking phase cluster from walking phase clusters that integrate temporally consecutive walking phases based on preset conditions. A walking phase cluster includes at least one walking phase. A walking phase cluster may also include a single walking phase. The walking waveform data and walking phases from which feature amounts used to estimate a harmony index are extracted will be described later.

[0045] 9 is a conceptual diagram for explaining extraction of feature quantities for estimating a harmony index from walking waveform data for one step cycle. For example, the extraction unit 123 extracts temporally consecutive walking phases i to i+m as a walking phase cluster C (i and m are natural numbers). In this embodiment, an example is given in which the walking phase cluster C used for estimating a harmony index is selected by correlation analysis using statistical parametric mapping. For example, the walking phase cluster C may be selected by Pearson's correlation analysis.

[0046] In the example of FIG. 9, walking phase cluster C includes m walking phases (components). That is, the number of walking phases (components) constituting walking phase cluster C (also referred to as the number of components) is m. FIG. 9 shows an example in which walking phases are integer values, but walking phases may be subdivided to the nearest decimal point. When walking phases are subdivided to the nearest decimal point, the number of components of walking phase cluster C corresponds to the number of data points in the section of the walking phase cluster. The extraction unit 123 extracts feature amounts from each of walking phases i to i+m. When walking phase cluster C is composed of a single walking phase j, the extraction unit 123 extracts feature amounts from that single walking phase j (j is a natural number).

[0047] The generation unit 125 applies a feature composition formula to the feature extracted from each of the walking phases constituting the walking phase cluster to generate a feature of the walking phase cluster (cluster feature). The feature composition formula is a calculation formula set in advance for generating a feature of the walking phase cluster. For example, the feature composition formula is a calculation formula related to arithmetic operations. For example, the cluster feature calculated using the feature composition formula is the integral average value, arithmetic mean value, slope, variance, etc. of the feature in each walking phase included in the walking phase cluster. For example, the generation unit 125 applies a calculation formula for calculating the slope or variance of the feature extracted from each walking phase constituting the walking phase cluster as the feature composition formula. For example, if a walking phase cluster is composed of a single walking phase, it is not possible to calculate the slope or variance, so a feature composition formula that calculates an integral average value, arithmetic mean value, etc. may be used. For example, if a walking phase cluster is composed of a single walking phase, the feature extracted from that walking phase may be set as the cluster feature.

[0048] Furthermore, the generation unit 125 calculates parameters related to gait (also referred to as gait parameters). The generation unit 125 calculates the gait parameters using feature amounts derived from the walking waveform data. The gait parameters include features used for estimating a physical state. The estimation system 1 may be configured so that the harmony index estimation device 13 calculates the gait parameters. Examples of gait parameters calculated by the generation unit 125 are listed below. The following gait parameters are merely examples and do not cover all parameters including gait features. In this embodiment, of the gait parameters listed below, those that have a high correlation in estimating the harmony index are selected. Details of the method for calculating the gait parameters will be omitted.

[0049] Examples of gait parameters include stride length, walking pitch, walking speed, contact angle, take-off angle, outward rotation distance (amount of rotation per minute), and toe direction (internal rotation / external rotation). Stride length is the distance between the toes of both feet when a step is taken with either the left or right foot and the toes land on the ground. Walking pitch is the number of steps taken within a predetermined time period and is used to calculate walking speed. Walking speed is the moving speed in a stride cycle. The walking speed may be an average value over multiple gait cycles. Contact angle is the angle (posture angle) of the sole of the foot with respect to the ground when the heel is in contact with the ground. Contact angle is the angle (posture angle) of the sole of the foot with respect to the ground when the toe is in contact with the ground. Outward rotation distance is the distance between the foot and a straight line indicating the path of movement of one foot in a stride cycle at the time when the foot is farthest from the path of movement of that foot. Toe direction is the angle between a straight line indicating the path of movement of one foot in a stride cycle and the center line of the foot when it is in a landed state.

[0050] Examples of gait parameters include roll angle, foot lift height, maximum angular velocity in plantar flexion, maximum angular velocity in dorsiflexion, maximum velocity, maximum foot information acceleration during swing, and cadence. For example, the roll angle at heel contact and toe-off is used as a gait parameter. Foot lift height corresponds to the vertical height of the foot. For example, the maximum angular velocity in plantar flexion and maximum angular velocity in dorsiflexion during the swing phase are used as gait parameters. For example, the maximum velocity during the swing phase is used as a gait parameter. The maximum foot information acceleration during swing is the maximum value of the vertical acceleration of the leg during swing, and is related to the rise of the hips in response to the coordination of the foot and hip movements. Cadence corresponds to the number of steps in 60 seconds.

[0051] Examples of gait parameters include stance time, swing time, DST (Double Support Time), load time, foot contact time, and push-off time. Stance time is the time corresponding to the duration of the stance phase. Swing time is the time corresponding to the duration of the swing phase. DST corresponds to the double support period during walking. DST includes DST1, which corresponds to the double support period after heel strike, and DST2, which corresponds to the double support period immediately before push-off. Load time is the time during which a load is applied to the sole of the foot. Load time is the time from heel strike to foot contact. Foot contact time is the time during which the main surface of the sole is in contact with the ground. Foot contact time is the time from foot contact to heel lift-off. Push-off time is the time from when a load is applied to the main surface of the sole to when the foot pushes off. Push-off time is the time during which the foot pushes off from the main surface of the sole.

[0052] The transmitting unit 127 outputs feature data including the cluster features and gait parameters generated by the generating unit 125. The cluster features and gait parameters are also referred to as first feature data. The transmitting unit 127 transmits the feature data to the harmony index estimation device 13. For example, the transmitting unit 127 transmits the feature data to the harmony index estimation device 13 via wireless communication. For example, the transmitting unit 127 is configured to transmit the feature data to the harmony index estimation device 13 via a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the transmitting unit 127 may conform to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark).

[0053] [Harmony index estimation device] 10 is a block diagram showing an example of the configuration of the harmony index estimation device 13. The harmony index estimation device 13 includes a communication unit 131, a calculation unit 133, a storage unit 135, an estimation unit 137, and an output unit 139.

[0054] The communication unit 131 acquires feature data from the measurement device 10. The communication unit 131 outputs the received feature data to the calculation unit 133. The communication unit 131 receives the feature data from the measurement device 10 via wireless communication. For example, the communication unit 131 is configured to receive the feature data from the measurement device 10 via a wireless communication function (not shown) that complies with standards such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the communication unit 131 may also be compliant with standards other than Bluetooth (registered trademark) or WiFi (registered trademark). The communication unit 131 may also receive the feature data from the measurement device 10 via a wired connection such as a cable.

[0055] The calculation unit 133 acquires special feature data. The calculation unit 133 calculates input data used to estimate the harmony index using the cluster features and gait parameters included in the acquired feature data. The calculation unit 133 calculates the average value / absolute value of the difference for the first feature values ​​(cluster features / gait parameters) for both feet used to estimate the harmony index. Hereinafter, the absolute value of the difference is also referred to as the difference. The average value and difference for the first feature values ​​for both feet calculated by the calculation unit 133 are also referred to as the second feature values. The second feature values ​​are used to estimate the harmony index. Instead of the average value and difference of the first feature values ​​included in the feature data generated by the measurement device 10, the first feature values ​​may be used directly to estimate the harmony index. In this case, the calculation unit 133 can be omitted.

[0056] The memory unit 135 stores an estimation model for estimating a harmony index using feature data extracted from the walking waveform data. The estimation model outputs an estimation result related to the harmony index in response to input data calculated by the calculation unit 133. The memory unit 135 stores estimation models trained on multiple subjects. When subject attributes are used for estimation, the memory unit 135 stores the subject attributes. For example, the subject attributes include the subject's gender, age, weight, height, etc. The harmony index estimation device 13 estimates harmony indices related to three directions: direction of travel, left / right walking, and vertical direction. The subject attributes differ depending on the direction of the harmony index to be estimated.

[0057] The estimation model is stored in the storage unit 135 at the time of shipping the product from the factory or at the time of calibration before the user uses the estimation system. For example, the estimation system 1 may be configured to use an estimation model stored in a storage device such as an external server. In this case, the estimation system 1 may be configured to use the estimation model via an interface (not shown) connected to the storage device.

[0058] The estimation unit 137 acquires input data used to estimate the harmonization index from the calculation unit 133. When the feature data generated by the measurement device 10 is used as is, the estimation unit 137 acquires the feature data as input data. When the attributes of the subject are used for estimation, the estimation unit 137 acquires the attributes of the subject from the storage unit 135.

[0059] The estimation unit 137 estimates the harmonic index using the acquired input data. In this embodiment, an example is given in which the HR of the lower back during a gait cycle is estimated. The HR of the lower back corresponds to the ratio of even frequency components to odd frequency components, or the ratio of odd frequency components to even frequency components, obtained by Fourier transforming the time-series data of the acceleration of the lower back during a gait cycle. The HR in the vertical direction and the direction of travel corresponds to the ratio of even frequency components to odd frequency components. The HR in the left-right direction corresponds to the ratio of odd frequency components to even frequency components.

[0060] The estimation unit 137 inputs input data to the estimation model stored in the storage unit 135. The estimation unit 137 outputs the estimation result of the harmonic index output from the estimation model. When using an estimation model stored in an external storage device constructed in the cloud, a server, or the like, the estimation unit 137 is configured to use the estimation model via an interface (not shown) connected to the storage device.

[0061] The harmony index is an index relating to the smoothness of hip movement and the harmony of walking. The estimation unit 137 estimates harmony indices for three directions: the left-right direction, the direction of travel, and the vertical direction. The harmony index for the direction of travel corresponds to the smoothness of hip movement in the front-to-back direction (in the sagittal plane) centered on the pelvis. The left-to-right harmony index corresponds to the smoothness of hip movement in the up-to-down direction (in the coronal plane) centered on the pelvis. The vertical harmony index corresponds to the smoothness of hip movement in the rotation of the body (in the horizontal plane) centered on the pelvis. Using the harmony index makes it possible to understand the smoothness of the subject's movement and the harmony of walking, which cannot be understood from foot movement alone.

[0062] FIG. 11 is a conceptual diagram for explaining an example of training of an estimation model used to estimate a harmony index. Explanatory variables and response variables related to multiple subjects are used to train the estimation model. The explanatory variables are second features generated based on the attributes of the subjects, cluster features generated according to the subjects' gait, and gait parameters. The explanatory variables may be first features including cluster features generated according to the subjects' gait and gait parameters. The response variables are actual measurements of the harmony index of spatial acceleration related to the waist. For example, the actual measurements of the harmony index are measured by an IMU attached to the waist. The harmony index related to the vertical direction HR z , Harmonic index HR for the direction of travel y , the harmony index HR in the left-right direction xis used as the response variable. For example, harmonic indices related to at least one of the vertical direction, the forward direction, and the left-right direction may be used as the response variable. For example, the vertical harmonic indices can be calculated by Fourier transforming the vertical acceleration and calculating the ratio between the power sum of the even components and the power sum of the odd components contained in those frequency components. For example, the forward harmonic indices can be calculated by Fourier transforming the forward acceleration and calculating the ratio between the power sum of the even components and the power sum of the odd components contained in those frequency components. Furthermore, the left-right harmonic indices can be calculated by Fourier transforming the left-right acceleration and calculating the ratio between the power sum of the odd components and the power sum of the even components contained in those frequency components. For example, the estimation model is a multiple regression model constructed using features selected by the leave-one-subject-out LASSO method.

[0063] For example, the estimation model is constructed by learning using a linear regression algorithm. For example, the estimation model is constructed by learning using a support vector machine (SVM) algorithm. For example, the estimation model is constructed by learning using a Gaussian process regression (GPR) algorithm. For example, the estimation model is constructed by learning using a random forest (RF) algorithm. The estimation model may also be constructed by unsupervised learning that classifies the subjects that generated the feature data according to the feature data. There are no particular limitations on the algorithm used to train the estimation model.

[0064] The estimation model may be constructed by learning using walking waveform data (sensor data) for one step cycle as explanatory variables. For example, the estimation model is constructed by supervised learning using walking waveform data of acceleration in three axes, angular velocity around three axes, and angles around three axes (posture angles) as explanatory variables and the harmonic index to be estimated as the objective variable.

[0065] FIG. 12 is a conceptual diagram showing an example of estimating a harmony index of a subject (user) using an estimation model. The estimation model is a model constructed in advance by the learning shown in FIG. 11. In the example of FIG. 12, second features generated based on the subject's attributes and first features (cluster features / gait parameters) generated according to the subject's gait are used as input data. In the case of an estimation model generated using first features including cluster features and gait parameters, the cluster features and gait parameters are used as input data. The estimation model outputs a harmony index in response to input data. In the example of FIG. 12, the harmony index HR z , Harmonic index HR for the direction of travel y , the harmony index HR in the left-right direction x is output as the estimation result. For example, the estimation model outputs the harmonic index HR z , Harmonic index HR for the direction of travel y , the harmony index HR in the left-right direction x may be configured so that at least one of the above is output as an estimation result.

[0066] The output unit 139 outputs the estimation result of the harmony index by the estimation unit 137. For example, the output unit 139 displays the estimation result of the harmony index on the screen of a mobile terminal of the subject (user). For example, the output unit 139 outputs the estimation result to an external system or the like that uses the estimation result. There are no particular limitations on the use of the information on the harmony index output from the harmony index estimation device 13.

[0067] For example, the harmonization index estimation device 13 is connected to an external system, such as a cloud or a server, via a mobile terminal (not shown) carried by the subject (user). The mobile terminal (not shown) is a portable communication device. For example, the mobile terminal is a mobile communication device with a communication function, such as a smartphone, a smart watch, or a mobile phone. For example, the harmonization index estimation device 13 is connected to the mobile terminal via a wired connection, such as a cable. For example, the harmonization index estimation device 13 is connected to the mobile terminal via wireless communication. For example, the harmonization index estimation device 13 is connected to the mobile terminal via a wireless communication function (not shown) conforming to standards such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the harmonization index estimation device 13 may conform to standards other than Bluetooth (registered trademark) or WiFi (registered trademark). The estimation result of the harmonization index may be used by an application installed on the mobile terminal. In this case, the mobile terminal executes processing using the estimation result using application software, etc., installed on the mobile terminal.

[0068] (operation) Next, the operation of the estimation system 1 will be described with reference to the drawings. Here, we will explain the measurement device 10 and the harmony index estimation device 13 included in the estimation system 1 individually. Regarding the measurement device 10, we will explain the operation of the feature data generation unit 12 included in the measurement device 10.

[0069] [Measuring equipment] Fig. 13 is a flowchart for explaining the operation of the feature amount data generating unit 12 included in the measurement device 10. In the explanation following the flowchart of Fig. 13, the feature amount data generating unit 12 will be described as the subject of the operation.

[0070] In FIG. 13, first, the feature amount data generator 12 acquires time-series data of sensor data relating to the foot movements of both feet (step S101).

[0071] Next, the feature amount data generating unit 12 extracts gait waveform data for one gait cycle from the time series data of the sensor data (step S102). The feature amount data generating unit 12 detects heel strikes and toe lifts from the time series data of the sensor data. The feature amount data generating unit 12 extracts time series data in the section between successive heel strikes as gait waveform data for one gait cycle.

[0072] Next, the feature amount data generating unit 12 normalizes the extracted walking waveform data for one step cycle (step S103). The feature amount data generating unit 12 normalizes the walking waveform data for one step cycle to a walking cycle of 0 to 100% (first normalization). Furthermore, the feature amount data generating unit 12 normalizes the ratio of the stance phase to the swing phase of the first normalized walking waveform data for one step cycle to 60:40 (second normalization).

[0073] Next, the feature data generation unit 12 extracts feature amounts from the normalized walking waveform from the walking phases used to estimate the harmony index (step S104). The feature data generation unit 12 extracts feature amounts used to estimate the harmony index.

[0074] Next, the feature data generation unit 12 generates first feature amounts using the extracted feature amounts (step S105). The feature data generation unit 12 generates the first feature amounts including cluster feature amounts and gait parameters according to the harmony index to be estimated.

[0075] Next, the feature amount data generator 12 integrates the first feature amounts for one walking cycle to generate feature amount data for one walking cycle (step S106).

[0076] Next, the feature amount data generating unit 12 outputs the generated feature amount data to the harmony index estimating device 13 (step S107).

[0077] [Harmony index estimation device] Fig. 14 is a flowchart for explaining the operation of the harmony index estimation device 13. In the explanation following the flowchart of Fig. 14, the harmony index estimation device 13 will be described as the subject of the operation.

[0078] In FIG. 14, first, the harmony index estimation device 13 acquires feature amount data used for estimating the harmony index from the measurement device 10 (step S131).

[0079] Next, the harmony index estimation device 13 calculates the average value of the first feature amounts included in the acquired feature amount data and the absolute value of the difference as the second feature amount (step S132).

[0080] Next, the harmony index estimation device 13 inputs the input data including the calculated second feature amount to an estimation model that estimates the harmony index (step S133).

[0081] Next, the harmony index estimation device 13 estimates the harmony index of the user according to the output (estimated value) from the estimation model (step S134).

[0082] Next, the harmony index estimation device 13 outputs information corresponding to the estimated harmony index (step S135). For example, the harmony index is output to a terminal device (not shown) carried by a user. For example, the information corresponding to the harmony index is output to a system that executes processing using the information.

[0083] (Application example) Next, application examples according to this embodiment will be described with reference to the drawings. In the application examples below, a function of a harmony index estimation device 13 installed on a mobile terminal carried by a user is shown to estimate information related to harmony indices using feature data measured by a measurement device 10 placed on a shoe.

[0084] 15 and 16 are conceptual diagrams showing an example of displaying the estimation result by the harmony index estimation device 13 on the screen of a mobile terminal 160 carried by a user walking while wearing shoes 100 on which the measuring device 10 is placed. In the example of Figs. 15 and 16, information according to the estimation result of the harmony index using feature amount data according to sensor data measured while the user was walking is displayed on the screen of the mobile terminal 160.

[0085] In the example of FIG. 15, the estimated results of the harmony index for three directions, namely the forward direction, the left-right direction, and the vertical direction, are displayed on the display unit of the mobile terminal 160. Also in the example of FIG. 15, recommendation information according to the estimated result of the harmony index, such as "You should train your core," is displayed on the display unit of the mobile terminal 160 according to the estimated value of the harmony index. Also in the example of FIG. 15, recommendation information according to the estimated result, such as "Training A is recommended. Please watch the video below," is displayed on the display unit of the mobile terminal 160 according to the estimated value of the harmony index. A user who has checked the information displayed on the display unit of the mobile terminal 160 can practice training to train their core by exercising by referring to the video of Training A according to the recommendation information.

[0086] In the example of FIG. 16, the estimated results of the harmony indices for three directions, namely the direction of travel, the left-right direction, and the vertical direction, are displayed on the display unit of the mobile terminal 160. Also in the example of FIG. 16, recommendation information such as "We recommend that you receive a medical examination at a hospital" is displayed on the display unit of the mobile terminal 160 according to the estimated values ​​of the harmony indices. For example, links to websites of hospitals that can provide medical examinations and telephone numbers may be displayed on the screen of the mobile terminal 160. A user who has checked the information displayed on the display unit of the mobile terminal 160 can receive a medical examination for a knee-related disease as appropriate by going to a hospital according to the recommendation information.

[0087] As described above, the estimation system of this embodiment includes a measurement device and a harmony index estimation device. The measurement device is attached to footwear of a subject for whom a harmony index, an index related to hip movement, is to be estimated. The measurement device includes a sensor and a feature data generation unit. The sensor measures spatial acceleration and spatial angular velocity. The sensor generates sensor data related to foot movement using the measured spatial acceleration and spatial angular velocity. The sensor outputs the generated sensor data. The feature data generation unit acquires time-series sensor data including gait features. The feature data generation unit extracts gait waveform data for one walking cycle from the time-series sensor data. The feature data generation unit normalizes the extracted gait waveform data. From the normalized gait waveform data, the feature data generation unit extracts feature values ​​used for estimating the harmony index from a walking phase cluster composed of at least one gait phase that is consecutive in time. The feature data generation unit generates feature data including the extracted feature values. The feature data generation unit outputs the generated feature data to the harmony index estimation device.

[0088] The harmonization index estimation device includes a communication unit, a storage unit, an estimation unit, and an output unit. The communication unit acquires feature data including first feature values ​​used to estimate a harmonization index related to the smoothness of hip movement, the first feature values ​​being extracted from gait waveforms of spatial acceleration and spatial angular velocity included in sensor data related to the movement of the feet of the subject. The first feature values ​​include at least one of gait parameters and cluster feature values ​​for each walking phase cluster. The storage unit stores an estimation model that outputs an estimated value related to the harmony index in response to an input of a first feature included in the feature data. The estimation unit inputs the first feature included in the acquired feature data to the estimation model and estimates the harmony index of the subject in response to the estimated value related to the harmony index output from the estimation model. The output unit outputs information related to the harmony index of the subject.

[0089] In this embodiment, a harmony index related to the smoothness of a subject's hip movement is estimated using a first feature extracted from sensor data related to the subject's leg movement. Therefore, according to this embodiment, a harmony index related to the smoothness of a subject's hip movement can be estimated easily and with high accuracy in daily life.

[0090] A harmonization index estimation device according to one aspect of the present embodiment includes a calculation unit (first calculation unit). The calculation unit calculates, as second feature quantities, average values ​​and differences of first feature quantities used to estimate the harmonization index, among the first feature quantities for both feet of the subject. The storage unit stores an estimation model that outputs an estimated value for the harmonization index in response to input of the second feature quantities. The estimation unit inputs the calculated second feature quantities to the estimation model, and estimates the harmonization index of the subject in response to the estimated value for the harmonization index output from the estimation model. According to this aspect, the harmonization index can be estimated with higher accuracy by using the second feature quantities generated using the first feature quantities for both feet.

[0091] In one aspect of this embodiment, the storage unit stores an estimation model that outputs an estimated value related to the harmony index in response to input of the subject's attributes and the second feature amount. The estimation unit inputs the subject's attributes and the second feature amount to the estimation model, and estimates the subject's harmony index in response to the estimated value related to the harmony index output from the estimation model. According to this aspect, the use of the subject's attributes enables the harmony index to be estimated with higher accuracy.

[0092] In one aspect of this embodiment, the storage unit stores an estimation model that outputs an estimated value for a harmony index in response to input of a first feature included in the feature data. The estimation model outputs at least one harmony index for three directions, i.e., the forward direction, left / right direction, and vertical direction, in a walking cycle as an estimated value for the harmony index. The estimation unit inputs the first feature included in the acquired feature data to the estimation model and estimates the subject's harmony index in response to the harmony index output from the estimation model. The estimation model outputs at least one harmony index for the three directions, i.e., the forward direction, left / right direction, and vertical direction. According to this aspect, the harmony index can be estimated with higher accuracy in response to the harmony index for the three directions, i.e., the forward direction, left / right direction, and vertical direction.

[0093] In one aspect of this embodiment, the harmonic index estimation device is implemented in a terminal device having a screen viewable by the subject. The harmonic index estimation device displays information about the harmonic index estimated in accordance with the subject's foot movements on the screen of the terminal device. According to this aspect, information about the harmonic index estimated for the subject can be accurately presented to the subject.

[0094] The harmonic index HR is an index of physical condition and health status. A person with a sufficiently large harmonic index HR is healthy. A person with a small harmonic index HR may be at risk of falling or may have progressive lower back pain. Therefore, the harmonic index HR can be used to determine the progression of the risk of falling or lower back pain. For example, a threshold value for the harmonic index HR is set in advance based on verification of multiple subjects regarding the progression of the symptom to be detected. In this way, the progression of the symptom can be estimated according to the estimated harmonic index HR.

[0095] (Second embodiment) Next, an estimation system according to a second embodiment will be described with reference to the drawings. The estimation system of this embodiment estimates odd and even components contained in frequency components obtained by Fourier transforming time-series data of spatial acceleration using different estimation models. The estimation system of this embodiment estimates a harmonic index using the estimated odd and even components.

[0096] (composition) FIG. 17 is a block diagram showing an example of the configuration of an estimation system 2 according to this embodiment. The estimation system 2 includes a measurement device 20 and a harmony index estimation device 23. In this embodiment, an example will be described in which the measurement device 20 and the harmony index estimation device 23 are configured as separate pieces of hardware. For example, the measurement device 20 is installed in footwear or the like of a subject (user) who is the subject of harmony index estimation. For example, the function of the harmony index estimation device 23 is installed in a mobile terminal carried by the subject (user). The measurement device 20 has the same configuration as the measurement device 10 of the first embodiment. In the following, a description of the measurement device 20 will be omitted, and the configuration of the harmony index estimation device 23 will be described.

[0097] [Harmony index estimation device] 18 is a block diagram showing an example of the configuration of the harmonization index estimation device 23. The harmonization index estimation device 23 includes a communication unit 231, a first calculation unit 233, a storage unit 235, an estimation unit 237, a second calculation unit 238, and an output unit 239.

[0098] The communication unit 231 has the same configuration as the communication unit 131 of the first embodiment. The communication unit 231 acquires feature amount data from the measurement device 20. The communication unit 231 outputs the received feature amount data to the first calculation unit 233.

[0099] The first calculation unit 233 is similar to the calculation unit 133 of the first embodiment. The first calculation unit 233 acquires special report quantity data. The first calculation unit 233 uses cluster features and gait parameters included in the acquired feature data to calculate input data used to estimate frequency components (odd components / even components) used to calculate the harmony index. The first calculation unit 233 calculates average values ​​for the first feature quantities of both feet used to estimate the harmony index. The first calculation unit 233 calculates absolute values ​​of differences for the first feature quantities of both feet used to estimate the harmony index. The first calculation unit 233 also calculates average values ​​for the gait parameters of both feet used to estimate the harmony index. The first calculation unit 233 calculates absolute values ​​of differences for the gait parameters of both feet used to estimate the harmony index. Hereinafter, the absolute values ​​of differences will also be referred to as differences. The average values ​​and differences of the first feature quantities / gait parameters of both feet calculated by the first calculation unit 233 are second feature quantities. The second feature amounts are used to estimate frequency components (odd components / even components). The first feature amounts and gait parameters may be used directly to estimate frequency components (odd components / even components) instead of the average values ​​or differences of the first feature amounts and gait parameters included in the feature amount data generated by measurement device 20. In this case, first calculation unit 233 may be omitted.

[0100] The storage unit 235 stores estimation models for estimating frequency components (odd components / even components) used in calculating the harmonic index. The storage unit 235 stores two estimation models for estimating odd components and even components included in the frequency components. In this embodiment, an example is given in which logarithmically transformed odd components / even components are estimated. The model for estimating the odd components included in the frequency components is called the first estimation model. The first estimation model outputs an estimation result related to the logarithmically transformed odd components in response to input data calculated by the first calculation unit 233. The model for estimating the even components included in the frequency components is called the second estimation model. The second estimation model outputs an estimation result related to the logarithmically transformed even components in response to input data calculated by the first calculation unit 233. The storage unit 235 stores estimation models trained on multiple subjects. If subject attributes are used for estimation, the storage unit 235 stores the subject attributes. For example, the subject's attributes include the subject's gender, age, weight, height, etc. The harmony index estimation device 23 estimates logarithmically transformed odd / even components used to calculate harmony indices for each of the three directions: vertical, forward walking, and left / right. The subject's attributes differ depending on the direction of the harmony index to be estimated.

[0101] The estimation model is stored in the storage unit 235 at the time of product shipment from the factory or at the time of calibration before the user uses the estimation system. For example, the estimation system 1 may be configured to use an estimation model stored in a storage device such as an external server. In this case, the estimation system 2 may be configured to use the estimation model via an interface (not shown) connected to the storage device.

[0102] The estimation unit 237 acquires input data used to estimate frequency components (odd components / even components) used in calculating the harmonic index from the first calculation unit 233. When the feature data generated by the measurement device 20 is used as is, the estimation unit 237 acquires the feature data as input data. When the attributes of the subject are used for estimation, the estimation unit 237 acquires the attributes of the subject from the storage unit 235.

[0103] The estimation unit 237 uses the acquired input data to estimate the frequency components (odd components / even components) used to calculate the harmony index. In this embodiment, an example is given in which logarithmically transformed odd and even components are estimated. The harmony index in the vertical direction and the traveling direction is the ratio between the odd and even components. The harmony index in the left-right direction is the ratio between the even and odd components.

[0104] The estimation unit 237 inputs input data for estimating logarithmically transformed odd components to a first estimation model stored in the storage unit 235. The estimation unit 237 also inputs input data for estimating logarithmically transformed even components to a second estimation model stored in the storage unit 235. The estimation unit 237 outputs the logarithmically transformed odd components output from the first estimation model and the logarithmically transformed even components output from the second estimation model to a second calculation unit 238. For example, the estimation models (first estimation model / second estimation model) are multiple regression models constructed using feature quantities selected by a leave-one-subject-out LASSO method.

[0105] FIG. 19 is a conceptual diagram illustrating an example of learning estimation models (first estimation model / second estimation model) used to estimate frequency components (odd components / even components) used in calculating a harmonic index. The first estimation model is used to estimate odd components. The second estimation model is used to estimate even components. In the example of FIG. 19, an example is given in which logarithmically transformed odd components / even components are estimated. The first estimation model and the second estimation model may be configured to estimate non-logarithmically transformed odd components / even components instead of logarithmically transformed odd components / even components.

[0106] The estimation models (first estimation model / second estimation model) are trained using explanatory variables and response variables related to multiple subjects. The explanatory variables include second features generated based on the subject's attributes, cluster features generated according to the subject's gait, and gait parameters. The explanatory variables may include first features including the cluster features generated according to the subject's gait and gait parameters.

[0107] The response variables are frequency components (odd components / even components) based on the actual measured values ​​of spatial acceleration related to the waist. For example, the actual measured values ​​of spatial acceleration related to the waist are measured by an IMU attached to the waist. In the example of FIG. 19, the frequency components (odd components / even components) related to the vertical direction, forward direction, and left / right direction are used as the response variables.

[0108] 20 to 22 are graphs showing examples of time series data of vertical acceleration, forward acceleration, and left-right acceleration of the waist. FIG. 20 is a graph showing an example of time series data of vertical acceleration of the waist. FIG. 21 is a graph showing an example of time series data of forward acceleration of the waist. FIG. 22 is a graph showing an example of time series data of left-right acceleration of the waist. The frequency components (odd components / even components) obtained by Fourier transforming each of the vertical acceleration, forward acceleration, and left-right acceleration of the waist for one stride are used as response variables.

[0109] In training the first estimation model, the odd components of the frequency components obtained by Fourier transforming the time series data of the vertical acceleration, forward acceleration, and left-right acceleration of the waist are used as the response variables. In training the second estimation model, the even components of the frequency components obtained by Fourier transforming the time series data of the vertical acceleration, forward acceleration, and left-right acceleration of the waist are used as the response variables. In the example of FIG. 19, the logarithmically transformed frequency components (odd components / even components) are used as the response variables. For example, the first estimation model and the second estimation model may be trained so that the power sum of the frequency components (odd components / even components) is estimated instead of the frequency components (odd components / even components).

[0110] For example, the estimation models (first estimation model / second estimation model) are constructed by learning using a linear regression algorithm. For example, the estimation models (first estimation model / second estimation model) are constructed by learning using a support vector machine (SVM) algorithm. For example, the estimation models (first estimation model / second estimation model) are constructed by learning using a Gaussian process regression (GPR) algorithm. For example, the estimation models (first estimation model / second estimation model) are constructed by learning using a random forest (RF) algorithm. The estimation models (first estimation model / second estimation model) may also be constructed by unsupervised learning that classifies the subjects that generated the feature data according to the feature data. There are no particular limitations on the algorithm used to train the estimation models (first estimation model / second estimation model).

[0111] The estimation models (first estimation model / second estimation model) may be constructed by learning using walking waveform data (sensor data) for one step cycle as explanatory variables. For example, the estimation models may be constructed by supervised learning using walking waveform data of acceleration in three axial directions, angular velocity around three axes, and angles around three axes (posture angles) as explanatory variables, and frequency components to be estimated (odd components / even components) as objective variables.

[0112] FIG. 23 is a conceptual diagram showing an example of estimating a harmony index of a subject (user) using estimation models (first estimation model / second estimation model). The estimation models (first estimation model / second estimation model) are models constructed in advance by the learning process shown in FIG. 19. In the example of FIG. 23, second feature values ​​generated based on the attributes of the subject, cluster feature values ​​generated according to the subject's gait, and gait parameters are used as input data. In the case of an estimation model generated using first feature values ​​including cluster feature values ​​and gait parameters, the cluster feature values ​​and gait parameters are used as input data. The estimation model outputs frequency components (odd components / even components) used to calculate the harmony index in response to input data. In the example of FIG. 23, the first estimation model outputs logarithmically transformed odd components as estimation results. In the example of FIG. 23, the second estimation model outputs logarithmically transformed even components as estimation results. When either the odd components or the even components are used, either the first estimation model or the second estimation model may be used.

[0113] The second calculation unit 238 acquires the odd and even components estimated by the estimation unit 237. The second calculation unit 238 calculates the power sum for one walking cycle for each of the estimated odd and even components. The second calculation unit 238 calculates a harmony index by calculating the ratio of the power sum of the odd components to the power sum of the even components in the vertical direction. The second calculation unit 238 calculates a harmony index by calculating the ratio of the power sum of the odd components to the power sum of the even components in the traveling direction. The second calculation unit 238 also calculates a harmony index by calculating the ratio of the power sum of the even components to the power sum of the odd components in the left-right direction.

[0114] FIG. 24 is a conceptual diagram showing an example of calculating the harmony index HR using logarithmically transformed frequency components (odd components / even components). The second calculation unit 238 calculates the logarithmically transformed harmony index HR by subtracting the logarithmically transformed odd components (power sum) from the logarithmically transformed even components (power sum) in the vertical and travel directions. The second calculation unit 238 calculates the harmony index (HR) by exponentially transforming the logarithmically transformed harmony index HR in the vertical and travel directions. z , H.R. y ) is calculated. In addition, the second calculation unit 238 calculates the logarithmically transformed harmony index HR by subtracting the logarithmically transformed even components (power sum) from the logarithmically transformed odd components (power sum) in the left-right direction. The second calculation unit 238 calculates the harmony index (HR x ) is calculated.

[0115] The output unit 239 outputs the calculation result (estimation result) of the harmony index by the second calculation unit 238. For example, the output unit 239 displays the estimation result of the harmony index on the screen of the portable terminal of the subject (user). For example, the output unit 239 outputs the estimation result to an external system or the like that uses the estimation result. There are no particular limitations on the use of the harmony index output from the harmony index estimation device 23.

[0116] For example, the harmonization index estimation device 23 is connected to an external system, such as a cloud or a server, via a mobile terminal (not shown) carried by the subject (user). The mobile terminal (not shown) is a portable communication device. For example, the mobile terminal is a mobile communication device with a communication function, such as a smartphone, a smart watch, or a mobile phone. For example, the harmonization index estimation device 23 is connected to the mobile terminal via a wired connection, such as a cable. For example, the harmonization index estimation device 23 is connected to the mobile terminal via wireless communication. For example, the harmonization index estimation device 23 is connected to the mobile terminal via a wireless communication function (not shown) conforming to standards such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the harmonization index estimation device 23 may conform to standards other than Bluetooth (registered trademark) or WiFi (registered trademark). The estimation result of the harmonization index may be used by an application installed on the mobile terminal. In this case, the mobile terminal executes processing using the estimation result using application software, etc., installed on the mobile terminal.

[0117] (Study example) Next, we will explain a learning example of the estimation model used to estimate the harmonic index by the harmonic index estimation device 23, along with verification results regarding the correlation between the frequency components (even components / odd components) used to estimate the harmonic index and feature data. Below, we present a verification example conducted on 45 subjects. In the following verification example, we verified the correlation between the measured and estimated values ​​of the harmonic index during walking. In this verification example, subjects wearing smart apparel and shoes equipped with a measurement device 20 walked two round trips along a 5-m straight path. An IMU measuring spatial acceleration and spatial angular velocity was installed on the waist of the smart apparel. The measured values ​​were derived using the measured values ​​of spatial acceleration and spatial angular velocity of the subject's waist. The predicted values ​​were estimated using sensor data measured by the measurement device 20 installed in the subject's shoes at the same time as the measured values. The correlation between the measured and estimated values ​​was evaluated using the intraclass correlation coefficient (ICC). The intraclass correlation coefficient ICC(2, k) was used to assess inter-rater reliability.

[0118] [Vertical direction] To estimate the frequency components (odd components / even components) for estimating the harmonic index in the vertical direction, the average value or difference of the first feature values ​​and gait parameters for both feet is used. To estimate the odd components, the difference of the gait parameters for both feet is used. To estimate the even components, the difference of the first feature values ​​and gait parameters for both feet is used. Two methods (first example / second example) are presented for the vertical direction.

[0119] <Example 1> In the first example, the first feature (cluster feature) and the average value / difference of the gait parameters are used for estimation. The estimation of the traveling direction and the left / right direction using the first example will be described.

[0120] A plurality of gait parameters are used to estimate the frequency components (odd components) for estimating the harmonic index in the vertical direction. For example, the differences between both feet for the dorsiflexion peak, the pronation-exit angle at contact, DST1, the load time of the maximum acceleration of the swing leg foot lift height, and the push-off time are used as second feature quantities.

[0121] A second feature quantity derived from the first feature quantity is used to estimate frequency components (even number components) for estimating a harmonic index related to the vertical direction. FIG. 25 is a table summarizing examples of second feature quantities derived from the first feature quantity, which are used to estimate frequency components (even number components) for estimating a harmonic index related to the vertical direction. Regarding the estimation of frequency components (even number components) related to the vertical direction, the traveling direction acceleration A y and vertical acceleration A z The average value of both feet for the acceleration in the forward direction A is used as the second feature. y Regarding the second feature F z1 is used for the estimation. Vertical acceleration A z Regarding the second feature F z2 is used for estimation.

[0122] Furthermore, multiple gait parameters are used to estimate frequency components (even-numbered components) for estimating harmonic indices in the vertical direction. For example, the average values ​​of both feet for the dorsiflexion peak and DST1 are used as second feature amounts.

[0123] The estimated value of the harmonic index for the vertical direction is calculated by calculating the ratio of the power sum of the even components to the power sum of the odd components. In this study, the intraclass correlation coefficient ICC(2, k) between the measured and estimated values ​​for the harmonic index for the vertical direction was 0.7804.

[0124] <Example 2> In the second example, the average value / difference of gait parameters is used for estimation. In the second example, the first feature (cluster feature) is not used for estimation. In the second example, frequency components (odd components / even components) based on the spatial acceleration of the waist are estimated using frequency components obtained by Fourier transforming the time-series data of the spatial acceleration of the feet. The frequency components based on the spatial acceleration of the feet (also called frequency features) are selected according to the number of vibrations per gait phase. A frequency component with n vibrations per gait phase is expressed as a frequency component of n vibrations (n ​​is a natural number). In the following example, the combination with the greatest correlation is selected by brute-force testing of combinations of components with 1 to 20 vibrations.

[0125] The frequency components (odd components) are estimated using the first, third, and fifth vibration frequency components based on the vertical acceleration of the foot. Additionally, multiple gait parameters are used in estimating the frequency components (odd components). For example, the differences between both feet for the dorsiflexion peak, the contact pronation / supination angle, DST1, the load time of the swing leg foot lift height maximum acceleration, and the push-off time are used in the estimation.

[0126] In estimating the frequency components (even components), the frequency components of the first, third, fourth, fifth, and sixth vibrations among the frequency components based on the vertical acceleration of the foot are used for estimation. In addition, in estimating the frequency components (odd components), multiple gait parameters are used for estimation. For example, the average values ​​of both feet for the dorsiflexion peak and DST1 are used for estimation.

[0127] For the second example, the intraclass correlation coefficient ICC(2, k) between the measured and estimated values ​​was 0.7756. In the second example, the correlation between the measured and estimated values ​​was equivalent to that of the first example, even without using the second feature derived from the cluster feature. Furthermore, according to the second example, the harmony index can be estimated with a smaller amount of data than in the first example. The method of the second example can also be applied to the forward direction and left-right direction, which will be described later.

[0128] <Direction of travel)> The average values ​​or differences of the first feature amounts and gait parameters for both feet are used to estimate the frequency components (odd components / even components) for estimating the harmonic index related to the traveling direction. The differences of the gait parameters for both feet are used to estimate the odd components. The differences of the first feature amounts and gait parameters for both feet are used to estimate the even components.

[0129] The second feature quantity derived from the first feature quantity is used to estimate the frequency components (odd components / even components) for estimating the harmony index related to the traveling direction. Fig. 26 is a table summarizing examples of the second feature quantity derived from the first feature quantity and used to estimate the frequency components (odd components / even components) for estimating the harmony index related to the traveling direction.

[0130] Regarding the estimation of the frequency components (odd components) related to the direction of travel, the vertical acceleration A z and angle E around the vertical axis z The average value of both feet for vertical acceleration A is used as the second feature. z Regarding the second feature F y1 is used for estimation. The angle around the vertical axis E z Regarding the second feature F y2 is used for estimation.

[0131] In addition, multiple gait parameters are used to estimate the frequency components (odd components) related to the direction of travel. For example, the differences between both feet in terms of plantar flexion angle, contact pronation / supination angle, DST2, maximum dorsiflexion / plantar flexion angular velocity during swing, and kick-off time are used as second feature quantities.

[0132] Regarding the estimation of the frequency components (even components) related to the traveling direction, the traveling direction acceleration A y and vertical acceleration A z The average value of both feet for the acceleration in the forward direction A is used as the second feature. y Regarding the second feature F y3 is used for the estimation. Vertical acceleration Az Regarding the second feature F y4 is used for estimation.

[0133] In addition, multiple gait parameters are used to estimate the frequency components (even-numbered components) related to the direction of travel. For example, the average values ​​of both feet for each of the following are used as second features: stride length, dorsiflexion peak, contact angle, DST1, and maximum swing leg velocity.

[0134] The estimated value of the harmonic index for the direction of travel is calculated by calculating the ratio of the power sum of the even components to the power sum of the odd components. In this study, the intraclass correlation coefficient ICC(2, k) between the measured and estimated values ​​for the harmonic index for the direction of travel was 0.6671.

[0135] <Left and right direction> The average values ​​or differences of the first feature amounts and gait parameters for both feet are used to estimate the frequency components (odd components / even components) for estimating the harmonic index in the left-right direction. The average values ​​of the first feature amounts and gait parameters for both feet are used to estimate the odd components. The differences of the gait parameters for both feet are used to estimate the even components.

[0136] To estimate the frequency components (odd components) for estimating the harmonic index in the left-right direction, multiple gait parameters are used. For example, the differences between both feet for plantar flexion peak, foot angle, DST1, maximum swing leg velocity, heel contact time, and push-off time are used as second feature quantities.

[0137] A second feature quantity derived from the first feature quantity is used to estimate frequency components (odd number components) for estimating a harmony index related to the left-right direction. FIG. 27 is a table summarizing examples of second feature quantities derived from the first feature quantity, which are used to estimate frequency components (odd number components) for estimating a harmony index related to the left-right direction. Regarding the estimation of frequency components (odd number components) related to the left-right direction, the angular velocity G about the traveling axis y and angle E around the axis of travel yThe average value of both feet regarding the angular velocity G about the axis of travel is used as the second feature. y Regarding the second feature F just before the heel strikes the ground, x1 is used for estimation. The angle E around the axis of travel y Regarding the second feature F just before the heel strikes the ground, x2 is used for estimation.

[0138] In addition, multiple gait parameters are used to estimate frequency components (even-numbered components) for estimating the harmonic index in the left-right direction. For example, the average values ​​of plantar flexion peak, foot angle, swing leg length, maximum swing leg velocity, heel contact time, and push-off time are used as second features.

[0139] The estimated value of the harmony index for the left-right direction is calculated by calculating the ratio of the power sum of the odd-numbered components to the power sum of the even-numbered components. In this study, the intraclass correlation coefficient ICC(2, k) between the measured and estimated values ​​for the harmony index for the left-right direction was 0.7488.

[0140] (operation) Next, the operation of the estimation system 2 will be described with reference to the drawings. Here, we will describe the operation of the harmony index estimation device 23 included in the estimation system 2. As for the measurement device 20, its operation is similar to that of the measurement device 10 of the first embodiment, and therefore its description will be omitted.

[0141] [Harmony index estimation device] Fig. 28 is a flowchart for explaining the operation of the harmonic index estimation device 23. In the explanation following the flowchart of Fig. 28, the explanation will be made with the harmonic index estimation device 23 as the subject of the operation. The flowchart of Fig. 28 relates to a case (first example) given in estimating frequency components (odd components) for estimating a harmonic index.

[0142] In FIG. 28, first, the harmony index estimation device 23 acquires feature amount data used for estimating the harmony index from the measurement device 20 (step S231).

[0143] Next, the harmony index estimation device 23 calculates the average value of the first feature amounts included in the acquired feature amount data and the absolute value of the difference as the second feature amount (step S232).

[0144] Next, the harmony index estimation device 23 inputs the input data including the calculated second feature amount to the first estimation model / second estimation model for estimating the harmony index (step S233). The harmony index estimation device 23 inputs the input data used to estimate the odd component to the first estimation model. The harmony index estimation device 23 inputs the input data used to estimate the even component to the second estimation model.

[0145] Next, the harmony index estimation device 23 calculates harmony indices using the outputs (estimated values) from the first estimation model and the second estimation model (step S234). The first estimation model outputs an estimation result for odd components. The second estimation model outputs an estimation result for even components. The harmony index estimation device 23 calculates harmony indices using odd and even components for each of the vertical direction, the traveling direction, and the left-right direction.

[0146] Next, the harmony index estimation device 23 outputs information corresponding to the calculated harmony index (step S235). For example, the harmony index is output to a terminal device (not shown) carried by the user. For example, the information corresponding to the harmony index is output to a system that executes processing using the information.

[0147] <Example 2> Next, a second example of estimating frequency components (odd components) for estimating a harmonization index related to the vertical direction will be described using a flowchart. Fig. 29 is a flowchart for explaining the second example. In the explanation following the flowchart of Fig. 29, the harmonization index estimation device 23 will be the subject of the operation.

[0148] In FIG. 29, first, the harmony index estimation device 23 acquires feature amount data used for estimating the harmony index from the measurement device 20 (step S251).

[0149] Next, the harmony index estimation device 23 calculates the average value of the first feature amounts included in the acquired feature amount data and the absolute value of the difference as the second feature amount (step S252).

[0150] Next, the harmonic index estimation device 23 converts the acceleration time series data included in the feature amount data into a frequency domain signal by Fourier transforming the data (step S253).

[0151] Next, the harmonic index estimation device 23 extracts frequency features to be used for estimation from the converted frequency domain signals (step S254). For example, the harmonic index estimation device 23 extracts frequency features to be used for estimation based on the number of vibrations per walking phase.

[0152] Next, the harmonic index estimation device 23 inputs input data including the second feature and the frequency feature to the first estimation model / second estimation model (step S255). The harmonic index estimation device 23 inputs the input data used to estimate the odd component to the first estimation model. The harmonic index estimation device 23 inputs the input data used to estimate the even component to the second estimation model.

[0153] Next, the harmony index estimation device 23 calculates harmony indices using the outputs (estimated values) from the first estimation model and the second estimation model (step S256). The first estimation model outputs an estimation result for odd components. The second estimation model outputs an estimation result for even components. The harmony index estimation device 23 calculates harmony indices using odd and even components for each of the vertical direction, the traveling direction, and the left-right direction.

[0154] Next, the harmony index estimation device 23 outputs information corresponding to the calculated harmony index (step S257). For example, the harmony index is output to a terminal device (not shown) carried by the user. For example, the information corresponding to the harmony index is output to a system that executes processing using the information.

[0155] As described above, the estimation system of this embodiment includes a measurement device and a harmony index estimation device. The measurement device is attached to footwear of a subject for whom a harmony index, an index related to hip movement, is to be estimated. The measurement device includes a sensor and a feature data generation unit. The sensor measures spatial acceleration and spatial angular velocity. The sensor generates sensor data related to foot movement using the measured spatial acceleration and spatial angular velocity. The sensor outputs the generated sensor data. The feature data generation unit acquires time-series sensor data including gait features. The feature data generation unit extracts gait waveform data for one walking cycle from the time-series sensor data. The feature data generation unit normalizes the extracted gait waveform data. From the normalized gait waveform data, the feature data generation unit extracts feature values ​​used for estimating the harmony index from a walking phase cluster composed of at least one gait phase that is consecutive in time. The feature data generation unit generates feature data including the extracted feature values. The feature data generation unit outputs the generated feature data to the harmony index estimation device.

[0156] The harmonic index estimation device includes a communication unit, a first calculation unit, a storage unit, an estimation unit, a second calculation unit, and an output unit. The communication unit acquires feature data including feature values ​​used to estimate a harmonic index related to the smoothness of hip movement, extracted from gait waveforms of spatial acceleration and spatial angular velocity included in sensor data related to the movement of the subject's feet. The first calculation unit calculates, as second feature values, average values ​​and differences of the first feature values ​​used to estimate the harmonic index, among the first feature values ​​for both feet of the subject. The storage unit stores estimation models including a first estimation model that outputs odd components in response to input of the second feature values, and a second estimation model that outputs even components in response to input of the second feature values. The even components are frequency components corresponding to elements in a gait cycle among frequency components obtained by Fourier transforming time series data of spatial acceleration of the hips. The odd components are frequency components not corresponding to elements in a gait cycle among frequency components obtained by Fourier transforming time series data of spatial acceleration of the hips. The estimation unit inputs the second feature amount to a first estimation model to estimate odd components. The estimation unit inputs the second feature amount to a second estimation model to estimate even components. The second calculation unit calculates the harmonic index of the subject using the power sum of the estimated odd and even components for one gait cycle. The output unit outputs information according to the harmony index of the subject.

[0157] In this embodiment, feature quantities extracted from sensor data related to the movement of the subject's legs are used to estimate frequency components (odd components / even components) obtained by Fourier transforming time-series data on spatial acceleration of the lower back. In this embodiment, the estimated frequency components (odd components / even components) are used to calculate a harmony index related to the smoothness of the subject's lower back movement. Therefore, according to this embodiment, it is possible to easily estimate a harmony index related to the smoothness of lower back movement in daily life with high accuracy.

[0158] (Third embodiment) Next, a harmonization index estimation device according to a third embodiment will be described with reference to the drawings. The harmonization index estimation device of this embodiment has a simplified configuration of the harmonization index estimation devices of the first and second embodiments.

[0159] 30 is a block diagram showing an example of the configuration of the harmony index estimation device 33 according to this embodiment. The harmony index estimation device 33 includes a communication unit 331, a storage unit 335, an estimation unit 337, and an output unit 339.

[0160] The communication unit 331 acquires feature data including features used to estimate a harmony index related to the smoothness of hip movement, extracted from walking waveforms of spatial acceleration and spatial angular velocity included in sensor data related to the movement of the subject's feet. The storage unit 335 stores an estimation model that outputs an estimated value related to the harmony index in response to input of the feature data. The estimation unit 337 inputs the feature data included in the acquired feature data to the estimation model, and estimates the subject's harmony index in response to the estimated value related to the harmony index output from the estimation model. The output unit 339 outputs information related to the subject's harmony index.

[0161] In this embodiment, a harmony index related to the smoothness of a subject's hip movement is estimated using feature amounts extracted from sensor data related to the subject's leg movements. Therefore, this embodiment makes it possible to easily estimate a harmony index related to the smoothness of hip movement in daily life with high accuracy.

[0162] (Hardware) Here, a hardware configuration for executing the processes according to each embodiment of the present disclosure will be described using an information processing device 90 (computer) in Fig. 31 as an example. Note that the information processing device 90 in Fig. 31 is an example configuration for executing the processes according to each embodiment, and does not limit the scope of the present disclosure.

[0163] As shown in Fig. 31, an information processing device 90 includes a processor 91, a main storage device 92, an auxiliary storage device 93, an input / output interface 95, and a communication interface 96. In Fig. 31, interface is abbreviated as I / F (Interface). The processor 91, the main storage device 92, the auxiliary storage device 93, the input / output interface 95, and the communication interface 96 are connected to each other via a bus 98 so as to be able to communicate data with each other. The processor 91, the main storage device 92, the auxiliary storage device 93, and the input / output interface 95 are also connected to a network such as the Internet or an intranet via the communication interface 96.

[0164] The processor 91 loads a program (instructions) stored in the auxiliary storage device 93 or the like into the main storage device 92. For example, the program is a software program for executing the processing of each embodiment. The processor 91 executes the program loaded into the main storage device 92. The processor 91 executes the program to perform the processing of each embodiment.

[0165] The main memory device 92 has an area in which a program is loaded. The processor 91 loads a program stored in the auxiliary memory device 93 or the like into the main memory device 92. The main memory device 92 is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory). Alternatively, a non-volatile memory such as an MRAM (Magneto-resistive Random Access Memory) may be configured / added to the main memory device 92.

[0166] The auxiliary storage device 93 stores various data such as programs. The auxiliary storage device 93 is realized by a local disk such as a hard disk or flash memory. Note that it is also possible to configure the main storage device 92 to store various data, thereby omitting the auxiliary storage device 93.

[0167] The input / output interface 95 is an interface for connecting the information processing device 90 to peripheral devices based on standards and specifications. The communication interface 96 is an interface for connecting to external systems and devices via a network such as the Internet or an intranet based on standards and specifications. The input / output interface 95 and the communication interface 96 may be a common interface for connecting to external devices.

[0168] Input devices such as a keyboard, mouse, and touch panel may be connected to the information processing device 90 as needed. These input devices are used to input information and settings. When a touch panel is used as the input device, a screen having the function of the touch panel serves as the interface. The processor 91 and the input devices are connected via an input / output interface 95.

[0169] The information processing device 90 may be equipped with a display device for displaying information. When a display device is equipped, the information processing device 90 is equipped with a display control device (not shown) for controlling the display of the display device. The information processing device 90 and the display device are connected via an input / output interface 95.

[0170] The information processing device 90 may be equipped with a drive device. The drive device acts as an intermediary between the processor 91 and a recording medium (program recording medium) to read data and programs stored on the recording medium and to write processing results of the information processing device 90 to the recording medium. The information processing device 90 and the drive device are connected via an input / output interface 95.

[0171] The above is an example of a hardware configuration for enabling the processing according to each embodiment of the present invention. The hardware configuration of Fig. 31 is an example of a hardware configuration for executing the processing according to each embodiment, and does not limit the scope of the present invention. A program for causing a computer to execute the processing according to each embodiment is also included in the scope of the present invention.

[0172] The scope of the present invention also includes a program recording medium on which the program according to each embodiment is recorded. The recording medium can be realized, for example, as an optical recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). The recording medium may also be realized as a semiconductor recording medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card. The recording medium may also be realized as a magnetic recording medium such as a flexible disk or other recording medium. When a program executed by a processor is recorded on a recording medium, the recording medium corresponds to a program recording medium.

[0173] The components of each embodiment may be combined in any manner, may be realized by software, or may be realized by a circuit.

[0174] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0175] 1, 2 Estimation system 10, 20 Measurement equipment 13, 23 Harmonic index estimator 111 Acceleration Sensor 112 Angular rate sensor 121 Acquisition Department 122 Normalization section 123 Extraction part 125 Generation part 127 Transmitter 131, 231, 331 Communications Department 133 Calculation Department 135, 235, 335 storage section 137, 237, 337 Estimation part 139, 239, 339 Output section 233 1st calculation section 238 2nd calculation section

Claims

1. a communication means for acquiring feature data including features used to estimate a harmonic index related to the smoothness of hip movement, the feature data being extracted from the walking waveforms of spatial acceleration and spatial angular velocity included in the sensor data related to the movement of the feet of the subject; a storage means for storing an estimation model that outputs an estimated value related to the harmony index in response to an input of a feature included in the feature data; an estimation means for inputting a feature included in the acquired feature data into the estimation model and estimating a harmony index of the subject according to an estimated value of the harmony index output from the estimation model; and an output means for outputting information according to the harmonic index of the subject.

2. The communication means is acquiring the feature data including a first feature including at least one of a gait parameter extracted from a walking waveform of spatial acceleration and spatial angular velocity included in the sensor data and a cluster feature for each walking phase cluster; The storage means storing the estimation model that outputs an estimated value related to the harmony index in response to input of the first feature amount included in the feature amount data; The estimation means 2. The harmonization index estimation device according to claim 1, wherein the first feature included in the acquired feature data is input to the estimation model, and the harmonization index of the subject is estimated according to an estimated value for the harmonization index output from the estimation model.

3. a first calculation means for calculating, as second feature quantities, an average value and a difference of the first feature quantities used to estimate the harmony index, among the first feature quantities for both feet of the subject; The storage means storing the estimation model that outputs an estimated value related to the harmony index in response to input of the second feature; The estimation means The harmonization index estimation device according to claim 2 , wherein the calculated second feature amount is input to the estimation model, and the harmonization index of the subject is estimated according to an estimated value for the harmonization index output from the estimation model.

4. The storage means storing the estimation model that outputs an estimated value related to the harmony index in response to input of the attribute of the subject and the second feature amount; The estimation means The harmony index estimation device according to claim 3 , wherein the attribute of the subject and the second feature amount are input to the estimation model, and the harmony index of the subject is estimated according to an estimated value for the harmony index output from the estimation model.

5. a second calculation means for calculating the harmonic index using even components corresponding to elements in a walking cycle and odd components not corresponding to elements in a walking cycle among frequency components obtained by Fourier transforming time series data of spatial acceleration of the lower back; The storage means storing the estimation models including a first estimation model that outputs the odd-numbered components in response to an input of the second feature amount and a second estimation model that outputs the even-numbered components in response to an input of the second feature amount; The estimation means inputting the second feature amount into the first estimation model to estimate the odd component; inputting the second feature amount into the second estimation model to estimate the even component; The second calculation means The harmonic index estimation device according to claim 3 , wherein the harmonic index of the subject is estimated using a power sum of the estimated odd-numbered components and the estimated even-numbered components for one walking cycle.

6. The storage means storing the estimation model that outputs, in response to input of the first feature amount included in the feature amount data, at least one of the harmony indices relating to three directions, i.e., a forward direction, a left-right direction, and a vertical direction, in a gait cycle as an estimated value of the harmony indices; The estimation means 3. The harmony index estimation device according to claim 2, wherein the first feature included in the acquired feature data is input to the estimation model, and the harmony index of the subject is estimated according to at least one of the harmony indexes relating to three directions, i.e., the traveling direction, the left-right direction, and the vertical direction, output from the estimation model.

7. The harmonic index estimation device according to any one of claims 1 to 6, a measurement device to be attached to the footwear of a subject who is a subject for whom a harmony index relating to smoothness of hip movement is to be estimated; The measuring device is a sensor that measures spatial acceleration and spatial angular velocity, generates sensor data related to foot movement using the measured spatial acceleration and spatial angular velocity, and outputs the generated sensor data; and a feature data generation means for acquiring time-series data of the sensor data including gait features, extracting gait waveform data for one walking cycle from the time-series data of the sensor data, normalizing the extracted gait waveform data, extracting, from the normalized gait waveform data, feature amounts used for estimating the harmonization index from a walking phase cluster formed by at least one temporally consecutive walking phase, generating feature data including the extracted feature amounts, and outputting the generated feature data to the harmonization index estimation device.

8. The harmonic index estimation device includes: implemented in a terminal device having a screen viewable by the subject, The estimation system according to claim 7 , wherein information about the harmony index estimated in accordance with the foot movement of the subject is displayed on a screen of the terminal device.

9. The computer Acquire feature data including features used to estimate a harmonic index related to the smoothness of hip movement, extracted from walking waveforms of spatial acceleration and spatial angular velocity included in sensor data related to the movement of the subject's feet; storing an estimation model that outputs an estimated value related to the harmony index in response to input of a feature included in the feature data; inputting the feature included in the acquired feature data into the estimation model, and estimating a harmony index of the subject according to the estimated value of the harmony index output from the estimation model; A harmony index estimation method that outputs information corresponding to the harmony index of the subject.

10. A process of acquiring feature data including features used to estimate a harmonic index related to the smoothness of hip movement, the feature data being extracted from the walking waveforms of spatial acceleration and spatial angular velocity included in the sensor data related to the movement of the subject's legs; a process of storing an estimation model that outputs an estimated value related to the harmony index in response to input of a feature included in the feature data; a process of inputting the feature included in the acquired feature data into the estimation model, and estimating the harmony index of the subject according to the estimated value of the harmony index output from the estimation model; and outputting information according to the subject's harmony index.

Images