Data generation device, learning system, data generation method, and program

Abstract

In order to generate a dataset used for learning even when a response variable is a continuous value, this data generation device is provided with: an acquisition unit for acquiring pair data constituted by a combination of measurement gait data relating to sensor data measured in accordance with the movement of a user's feet and a response variable corresponding to the measurement gait data; a measurement data processing unit for generating a measurement dataset vector by combining the response variable and a feature amount vector calculated using a feature amount extracted from measurement gait data, and generating a covariance matrix relating to a plurality of pair data; a pseudo data generation unit for generating pseudo gait data using the measurement gait data, and generating a pseudo dataset vector by combining a pseudo feature amount vector calculated using a pseudo feature amount extracted from the pseudo gait data, and a pseudo response variable generated using a covariance matrix relating to the pseudo feature amount vector; and an output unit for outputting the dataset.

Description

[Technical Field]

[0001] The present disclosure relates to a data generation device and the like that generates a data set used for training a model that estimates a physical state using gait data. [Background technology]

[0002] In response to growing interest in healthcare, services that provide information based on the characteristics of walking patterns (also called gait) are attracting attention. For example, technology is being developed to analyze gait based on sensor data measured by sensors mounted on footwear such as shoes. Characteristics of gait events related to health status appear in the time series data of sensor data.

[0003] Patent Document 1 discloses a learning device that generates a learning model using rehabilitation data collected within a predetermined period. The device in Patent Document 1 generates rehabilitation data including an index indicating the degree of recovery of a trainee and setting parameters as learning data. The device in Patent Document 1 performs machine learning using the learning data to generate a learning model that inputs the index and outputs recommended values ​​for the setting parameters.

[0004] Non-Patent Document 1 discloses an extension of inertial sensor-based walking data. In the method of Non-Patent Document 1, walking data is extended by adding random temporal fluctuations to the time series data of sensor data measured by an inertial sensor. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-007481 [Non-patent literature]

[0006] [Non-Patent Document 1] L. Tran and D. Choi, “Data Augmentation for Inertial Sensor-Based Gait Deep Neural Network”, IEEE Access, Vol.8, pp.12364-12378 (2020). Summary of the Invention [Problem to be solved by the invention]

[0007] In the method of Patent Document 1, in order to generate a learning model with sufficient accuracy, it is necessary to collect as much rehabilitation data as possible. However, in order to collect a sufficient amount of rehabilitation data, it is necessary to increase the trainee's walking training, and collecting the data requires a huge amount of time and effort.

[0008] The method of Non-Patent Document 1 expands the data set used for learning by expanding time-series data of sensor data corresponding to explanatory variables and associating response variables with the expanded explanatory variables. In other words, the method of Non-Patent Document 1 can save the time and effort required for data collection. For example, the method of Non-Patent Document 1 can expand the data set when the response variable is a fixed value, such as a personal authentication ID (Identifier). However, the method of Patent Document 1 cannot expand the data set when the response variable is a continuous numeric value because it cannot associate the response variable with the expanded explanatory variables.

[0009] An object of the present disclosure is to provide a data generation device and the like that can generate a dataset to be used for learning even when the response variable is a continuous numerical value. [Means for solving the problem]

[0010] A data generation device of one aspect of the present disclosure includes an acquisition unit that acquires paired data that combines measured gait data related to sensor data measured in accordance with movements of a user's feet and response variables corresponding to the measured gait data; a measurement data processing unit that generates measured data set vectors by combining the response variables with feature vectors calculated using features extracted from the measured gait data, and generates a covariance matrix related to the plurality of paired data; a pseudo data generation unit that generates pseudo gait data using the measured gait data, and generates pseudo data set vectors by combining pseudo feature vectors calculated using pseudo features extracted from the pseudo gait data with pseudo response variables generated using the covariance matrix related to the pseudo feature vectors; and an output unit that outputs a dataset including the measured data set vectors and the pseudo data set vectors.

[0011] A data generation method of one embodiment of the present disclosure acquires paired data that combines measured gait data related to sensor data measured in accordance with movement of a user's feet with response variables corresponding to the measured gait data, generates measured data set vectors by combining the response variables with feature vectors calculated using features extracted from the measured gait data, generates a covariance matrix related to a plurality of four paired data, generates pseudo gait data using the measured gait data, generates pseudo data set vectors by combining pseudo feature vectors calculated using pseudo features extracted from the pseudo gait data with pseudo response variables generated using the covariance matrix related to the pseudo feature vectors, and outputs a dataset including the measured data set vectors and the pseudo data set vectors.

[0012] A program according to one aspect of the present disclosure causes a computer to execute the following processes: acquiring paired data that combines measured gait data related to sensor data measured in accordance with movements of a user's feet with response variables corresponding to the measured gait data; generating a measured data set vector by combining a feature vector calculated using features extracted from the measured gait data with the response variables; generating a covariance matrix related to the plurality of paired data; generating pseudo gait data using the measured gait data; generating a pseudo data set vector by combining a pseudo feature vector calculated using pseudo features extracted from the pseudo gait data with a pseudo response variable generated using a covariance matrix related to the pseudo feature vector; and outputting a data set including the measured data set vector and the pseudo data set vector. [Effects of the Invention]

[0013] According to the present disclosure, it is possible to provide a data generation device or the like that can generate a data set to be used for learning even when the response variable is a continuous numerical value. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing an example of the configuration of a learning system according to a first embodiment. [Figure 2] FIG. 2 is a conceptual diagram for explaining an example of the arrangement of measurement devices that measure sensor data used by a data generation device included in the learning system according to the first embodiment. [Figure 3] FIG. 2 is a conceptual diagram for explaining an example of a coordinate system set in a measurement device that measures sensor data used by a data generation device included in a learning system according to the first embodiment. [Figure 4] FIG. 2 is a conceptual diagram for explaining an example of a gait event detected from time-series data of sensor data used by the data generating device included in the learning system according to the first embodiment. [Figure 5]FIG. 2 is a conceptual diagram for explaining an example of feature extraction by a data generation device included in the learning system according to the first embodiment. [Figure 6] FIG. 2 is a conceptual diagram for explaining an example of generation of a feature vector by a data generation device included in the learning system according to the first embodiment. [Figure 7] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 8] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 9] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 10] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 11] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 12] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 13] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 14] FIG. 2 is a conceptual diagram for explaining an example of data processing by a data generating device included in the learning system according to the first embodiment. [Figure 15] 1 is a conceptual diagram for explaining an example of the configuration of a learning device included in a learning system according to a first embodiment. [Figure 16] 4 is a flowchart illustrating an example of the operation of a data generating device included in the learning system according to the first embodiment. [Figure 17]10 is a flowchart for explaining an example of measurement data processing by a data generating device included in the learning system according to the first embodiment. [Figure 18] 10 is a flowchart illustrating an example of a pseudo data generation process performed by a data generation device included in the learning system according to the first embodiment. [Figure 19] 4 is a flowchart illustrating an example of the operation of a learning device included in the learning system according to the first embodiment. [Figure 20] 10 is a graph for explaining the evaluation of the accuracy of a response variable estimated by an estimation model generated using a dataset that has not been augmented by a data processing device provided in a learning system according to the first embodiment. [Figure 21] 10 is a graph for explaining an evaluation of the accuracy of a response variable estimated by an estimation model generated using a data set extended by a learning device provided in a learning system according to the first embodiment. [Figure 22] 10 is a graph for explaining an evaluation of the versatility of a response variable estimated by an estimation model generated using a dataset that has not been augmented by a data processing device provided in a learning system according to the first embodiment. [Figure 23] 10 is a graph for explaining an evaluation of the versatility of a response variable estimated by an estimation model generated using a dataset extended by a data processing device provided in a learning system according to the first embodiment. [Figure 24] FIG. 10 is a conceptual diagram for explaining application example 1-1 of the first embodiment. [Figure 25] FIG. 10 is a conceptual diagram for explaining application example 1-1 of the first embodiment. [Figure 26] FIG. 10 is a conceptual diagram for explaining application example 1-2 of the first embodiment. [Figure 27] FIG. 10 is a block diagram showing an example of the configuration of an estimation system according to a second embodiment. [Figure 28]FIG. 10 is a block diagram showing an example of the configuration of a measurement device included in an estimation system according to a second embodiment. [Figure 29] FIG. 10 is a conceptual diagram for explaining an example of the arrangement of measuring devices included in an estimation system according to a second embodiment. [Figure 30] FIG. 10 is a block diagram showing an example of the configuration of an estimation device included in an estimation system according to a second embodiment. [Figure 31] FIG. 10 is a conceptual diagram for explaining an example of estimation by an estimation device included in an estimation system according to a second embodiment. [Figure 32] 10 is a flowchart for explaining an example of the operation of a measurement device included in the estimation system according to the second embodiment. [Figure 33] 10 is a flowchart illustrating an example of a sensor data measurement process performed by a measurement device included in an estimation system according to a second embodiment. [Figure 34] 10 is a flowchart for explaining an example of gait parameter calculation processing by a measurement device provided in an estimation system according to a second embodiment. [Figure 35] 10 is a flowchart illustrating an example of an operation of an estimation device included in an estimation system according to a second embodiment. [Figure 36] FIG. 10 is a conceptual diagram for explaining an application example 2-1 of the second embodiment. [Figure 37] FIG. 10 is a conceptual diagram for explaining application example 2-2 of the second embodiment. [Figure 38] FIG. 10 is a conceptual diagram for explaining application example 2-2 of the second embodiment. [Figure 39] FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data generating device according to a third embodiment. [Figure 40] FIG. 2 is a block diagram showing an example of a hardware configuration for executing control and processing according to each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, the embodiments described below are limited in a manner that is technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following. In all drawings used to describe the following embodiments, the same reference numerals are used for similar parts unless otherwise specified. Furthermore, in the following embodiments, repeated explanations of similar configurations and operations may be omitted.

[0016] (First embodiment) First, a learning system according to a first embodiment will be described with reference to the drawings. The learning system of this embodiment generates a learning model that outputs response variables related to the gait events and physical state of a user in response to input of feature quantities generated based on physical quantities measured according to the user's walking.

[0017] (composition) FIG. 1 is a block diagram showing an example of the configuration of a learning system 10 according to this embodiment. The learning system 10 includes a data generating device 11 and a learning device 15. FIG. 1 illustrates the detailed configuration of the data generating device 11. The detailed configuration of the learning device 15 will be described later (FIG. 15). Below, the data generating device 11 and the learning device 15 will be described in order.

[0018] [Data Generator] The following describes the data generating device 11 of the learning system 10. The data generating device 11 includes an acquiring unit 111, a measurement data processing unit 112, a pseudo data generating unit 113, and an output unit 115.

[0019] The acquisition unit 111 acquires paired data according to the user's walking. The paired data is data that pairs gait data related to features (also called gait) included in the user's walking pattern with gait event indices (also called response variables) according to the gait data. In other words, the paired data is data that combines gait data with response variables corresponding to the gait data. For example, the acquisition unit 111 acquires paired data stored in a database constructed in a cloud or a server (not shown).

[0020] Gait data is data related to physical quantities (also referred to as sensor data) related to foot movement. For example, gait data is time-series data of physical quantities (also referred to as sensor data) related to foot movement. For example, gait data is values ​​of gait parameters related to events (also referred to as gait events) detected from time-series data of sensor data. Hereinafter, gait data based on measured sensor data is also referred to as measured gait data.

[0021] Gait event indices (response variables) are values ​​corresponding to gait data. For example, response variables are numerical values ​​related to gait parameters such as the user's walking speed, stride length, contact angle, takeoff angle, maximum foot lift height (sensor position), circumduction (traveling direction trajectory), and toe direction. For example, response variables are numerical values ​​related to the user's physical condition estimated based on gait parameters. For example, response variables are numerical values ​​indicating the degree of pronation / supination, hallux valgus, left-right symmetry, etc. of the user's foot. For example, response variables may be values ​​related to an identification number or attribute for identifying the user.

[0022] The time-series data of sensor data for one gait cycle (also called a gait waveform) is a collection of sensor data such as acceleration, angular velocity, and plantar angle. Hereinafter, each unit section of a gait cycle is called a "gait phase." For example, if the gait waveform for one gait cycle is divided equally between 0 and 100 percent (%), the gait phase is set in increments of 1%. There are no particular limitations on the division criteria for the gait waveform for one gait cycle. For example, the gait waveform for one gait cycle may be divided according to the measurement conditions of the sensor data and the gait parameters of the measurement target.

[0023] FIG. 2 is a conceptual diagram illustrating an example of the placement of a measurement device 110 that measures physical quantities related to foot movement. The measurement device 110 is placed inside the shoe 100. FIG. 2 shows an example in which the measurement device 110 is placed at a position corresponding to the back of the arch of the foot. For example, the measurement device 110 is placed in an insole inserted into the shoe 100. For example, the measurement device 110 is placed on the bottom surface of the shoe 100. For example, the measurement device 110 is embedded in the body of the shoe 100. The measurement device 110 may or may not be detachable from the shoe 100. The measurement device 110 may be placed at a position other than the back of the arch of the foot as long as it can acquire sensor data related to foot movement. The measurement device 110 may also be placed in socks worn by the user or in an accessory such as an anklet worn by the user. The measurement device 110 may also be attached directly to the foot or embedded in the foot. 2 shows an example in which the measuring device 110 is installed in the shoe 100 on the right foot. The measuring device 110 may also be installed in the shoe 100 on the left foot. Also, the measuring device 110 may be installed in the shoes 100 on both feet.

[0024] The measurement device 110 includes a sensor (not shown) that measures physical quantities related to the movement of the foot of a user wearing the footwear. The physical quantities related to the movement of the foot include acceleration in three axial directions (also called spatial acceleration) and angular velocity around three axes (also called spatial angular velocity). The physical quantities related to the movement of the foot also include velocity, angle, and position (trajectory) calculated by integrating the acceleration and angular velocity.

[0025] The acquisition unit 111 acquires paired data related to sensor data measured by the measurement device 110. There are no particular limitations on the source from which the paired data is acquired. For example, the acquisition unit 111 acquires paired data stored in a cloud, a server, or the like. For example, the acquisition unit 111 may acquire paired data stored in a mobile device carried by a user. Furthermore, the acquisition unit 111 may acquire measurement data from the measurement device 110 in real time.

[0026] FIG. 3 is a conceptual diagram illustrating a local coordinate system (x-axis, y-axis, z-axis) set in the measurement device 110 when the measurement device 110 is placed behind the arch of the foot. A local coordinate system consisting of x-, y-, and z-directions is set in the measurement device 110. A world coordinate system (X-, Y-, and Z-axes) is set relative to the ground. In the world coordinate system (X-, Y-, and Z-axes), when the user is standing upright, the user's lateral direction is set as the X-axis direction (rightward is positive), the direction in front of the user (direction of travel) is set as the Y-axis direction (forward is positive), and the direction of gravity is set as the Z-axis direction (vertically upward is positive). Note that the local coordinate system set in the measurement device 110 is not limited to the example shown in FIG. 3. Any local coordinate system can be set for the measurement device 110.

[0027] FIG. 4 is a conceptual diagram illustrating gait events detected in a step cycle based on the right foot. The horizontal axis in FIG. 4 represents a normalized gait cycle, with one right-foot step cycle starting from the point when the heel of the right foot hits the ground and ending at the point when the heel of the right foot hits the ground, being 100 percent (%). A gait cycle for one leg is broadly divided into a stance phase, in which at least a portion of the sole of the foot is in contact with the ground, and a swing phase, in which the sole of the foot is off the ground. In the example of FIG. 4, the stance phase is normalized to account for 60% and the swing phase to account for 40%. The stance phase is further divided into an early stance phase T1, a mid-stance phase T2, a late stance phase T3, and an early swing phase T4. The swing phase is further divided into an early swing phase T5, a mid-swing phase T6, and a late swing phase T7. The starting point of the gait waveform for one step cycle does not have to be the point at which the heel hits the ground. For example, the starting point of the gait waveform for one step cycle may be set to the center of the stance phase.

[0028] In FIG. 4, gait event E1 represents an event in which the heel of the right foot touches the ground (heel strike) (HS: Heel Strike). Gait event E2 represents an event in which the toe of the left foot leaves the ground (opposite toe off) (OTO: Opposite Toe Off) while the sole of the right foot is in contact with the ground (heel strike). Gait event E3 represents an event in which the heel of the right foot rises (heel rise) while the sole of the right foot is in contact with the ground (HR: Heel Rise). Gait event E4 represents an event in which the heel of the left foot touches the ground (opposite heel strike) (OHS: Opposite Heel Strike). Gait event E5 represents an event in which the toe of the right foot leaves the ground (toe off) while the sole of the left foot is in contact with the ground (TO: Toe Off). Gait event E6 represents an event in which the left and right feet cross (foot crossing) with the sole of the left foot in contact with the ground (FA: Foot Adjacent). Gait event E7 represents an event in which the tibia of the right foot is nearly perpendicular to the ground (TV: Tibia Vertical) with the sole of the left foot in contact with the ground (TV: Tibia Vertical). Gait event E8 represents an event in which the heel of the right foot touches the ground (heel strike) (HS: Heel Strike). Gait event E8 corresponds to the end of the gait cycle that began with gait event E1 and also corresponds to the starting point of the next gait cycle.

[0029] FIG. 5 is a conceptual diagram illustrating feature values ​​extracted from a gait waveform. The graph in FIG. 5 is an example of a gait waveform of the plantar angle around the x-axis. The gait waveform in FIG. 5 is a waveform for one gait cycle (0 to 100%) starting from heel strike. In the example in FIG. 5, feature values ​​are extracted from each of consecutive gait phases that are approximately 40% of the gait cycle. Multiple feature values ​​extracted from consecutive gait phases are integrated into a cluster (also called a gait phase cluster). In the example in FIG. 5, a gait phase cluster GC0 is set by integrating consecutive gait phases that are approximately 40% of the gait cycle. From the gait phase cluster, feature values ​​are extracted according to the feature values ​​extracted from the gait phases that make up that gait phase cluster. In addition, in the example in FIG. 5, feature values ​​are extracted from a single gait phase GP1 just before 80% of the gait cycle. The feature values ​​extracted from sensor data for one gait cycle are integrated into a feature vector for that gait cycle. For example, multiple feature values ​​extracted for each of nine types of walking waveforms related to acceleration in three axes and angular velocity / angles around three axes are integrated into a feature value vector for one walking cycle.

[0030] Fig. 6 is a conceptual diagram for explaining an example of a feature vector extracted from gait data for one step cycle. Features corresponding to gait event indices related to the body state are extracted from the walking waveform. The gait event indices are evaluation indices of gait events and correspond to response variables. For example, a gait event index is a numerical value related to a user's gait parameter. For example, a gait event index is a numerical value (also called a score) related to an evaluation of a user's gait. For example, a gait event index includes a degree of foot pronation / supination, a degree of hallux valgus progression, a degree of knee osteoarthritis progression, muscle strength, balance ability, body flexibility, and other physical condition indicators. For example, a gait event index is the Center of Pressure Excursion Index (CPEI), which is an evaluation index of foot pronation / supination.

[0031] From the gait waveform, features of gait phase clusters composed of temporally consecutive gait phases and features of individual gait phases are extracted. The features extracted from the gait waveform are associated with gait event indices. That is, for the same subject or the same group of subjects, covariance can be used as an indicator of the correspondence between features extracted from the gait waveform and gait event indices. For example, a gait phase cluster GC0 is constructed by integrating temporally consecutive gait phases i to i+4 (i is a natural number). Features are extracted from each of gait phases i to i+4. A feature composition formula is applied to the features extracted from each of gait phases i to i+4 to generate features of gait phase cluster GC0. The feature composition formula is a predetermined formula for generating features of gait phase clusters. For example, the feature composition formula is a formula related to the four basic arithmetic operations. For example, the feature composition formula is a formula related to integral average, arithmetic mean, slope, and variance. In addition, features are extracted from individual gait phase j (j is a natural number). Note that the feature composition equation may be applied without distinguishing between gait phases i to i+4 that make up a gait phase cluster and an individual gait phase j. For example, when the feature composition equation is applied to an individual gait phase j, since it is not possible to calculate slope or variance, a feature composition equation that calculates an integral average or arithmetic average may be used. Multiple feature quantities extracted from sensor data for one gait cycle are integrated as a feature vector associated with that gait cycle. For example, multiple feature quantities extracted from gait waveforms for one gait cycle are integrated using the feature composition equation.

[0032] The measured data processing unit 112 acquires a plurality of paired data sets each including measured gait data and a response variable corresponding to the measured gait data. The measured data processing unit 112 extracts feature amounts from the measured gait data. The measured data processing unit 112 calculates a feature amount vector for each paired data set based on the feature amounts extracted from the measured gait data. The feature amount vector is a vector combining the feature amounts extracted for each paired data set. The feature amount vector is composed of the feature amounts extracted for each step of gait data.

[0033] The measurement data processing unit 112 generates a vector (also called a measurement data set vector) that combines a feature vector and a response variable for each of the acquired plurality of paired data.

[0034] The measurement data processing unit 112 calculates an average vector of feature vectors for multiple paired data. For example, the measurement data processing unit 112 calculates the average value of feature values ​​constituting the feature vector of gait data for each walking phase for each paired data. The measurement data processing unit 112 generates an average vector of feature vectors by combining the average values ​​of feature values ​​calculated for each walking phase. The measurement data processing unit 112 also calculates the average value of response variables for multiple paired data. The numerical values ​​of feature values ​​constituting the one-dimensional feature vector follow a Gaussian distribution. In this embodiment, covariance is used as an index of correspondence between feature values ​​extracted from a gait waveform and gait event indices (response variables), and therefore, an arithmetic mean value is used as the average value of feature values ​​and response variables. When a variance other than covariance is used, an average value appropriate for the variance used may be used.

[0035] FIG. 7 is a conceptual diagram for explaining a feature vector F extracted from a plurality of measured gait data S and a response variable A corresponding to the feature vector F. A plurality of feature vectors F1 to Fn are extracted from each of the plurality of measured gait data S1 to Sn (n is a natural number). A vector combining the average values ​​of a plurality of feature vectors F1 to Fn is the average vector FS of the feature vector F. A plurality of response variables A1 to An are associated with each of the plurality of feature vectors F1 to Fn. In FIG. 7, numerical values ​​such as 0.3, 0.7, and 0.5 are shown to indicate that the response variables A1 to An are numerical values. The average value of the plurality of response variables A1 to An is the average value AS of the response variables. In FIG. 7, the numerical value of the average value AS of the response variables is indicated by x (x is a real number).

[0036] 8 shows an example of generating a measurement data set vector FA by combining a feature vector F and a response variable A. The measurement data processing unit 112 generates measurement data set vectors FA1 to FAn by associating a plurality of response variables A1 to An with a plurality of feature vectors F1 to Fn, respectively.

[0037] The measurement data processing unit 112 generates a covariance matrix between multiple feature quantities and response variables for multiple measurement dataset vectors. If the measurement data processing unit 112 considers each of the multiple measurement dataset vectors as a vector in a linear space, the vector group represented by all of the measurement dataset vectors can be considered to follow a multidimensional normal distribution in this linear space. Therefore, the measurement data processing unit 112 generates a covariance matrix for the measurement dataset vector. There is covariance between each of the multiple feature quantities constituting the measurement dataset vector and the response variable. In some cases, there is also covariance between the multiple feature quantities constituting the dataset vector. A feature vector can be formed by combining only some of the feature quantities. However, because all feature quantities are related to the response variable, combining all feature quantities results in higher covariance accuracy than combining only some of the feature quantities. Furthermore, there is a restriction that the number of dimensions, calculated by adding one to the number of feature quantities, must be less than the number of data. Therefore, when the number of data is small, the accuracy of the covariance may be improved by removing less related feature quantities or combining more related feature quantities.

[0038] The measurement data processing unit 112 performs Cholesky decomposition on the covariance matrix to derive an upper triangular matrix of the covariance matrix. The measurement data processing unit 112 extracts the last column vector of the derived upper triangular matrix. The last column vector of the upper triangular matrix contains the parts related to the response variables.

[0039] 9 is a conceptual diagram showing an example of deriving an upper triangular matrix W1 of a covariance matrix W for a plurality of measurement data set vectors. The measurement data processing unit 112 calculates the covariance matrix W for a plurality of measurement data set vectors FA1 to FAn. The measurement data processing unit 112 performs Cholesky decomposition on the calculated covariance matrix W to calculate the upper triangular matrix W1. The last column vector LC of the upper triangular matrix W1 is used to calculate the deviation C of the pseudo response variable, which will be described later.

[0040] The pseudo data generation unit 113 acquires measured gait data of a plurality of paired data from the acquisition unit 111. The pseudo data generation unit 113 also acquires an average vector of feature vectors and average values ​​of response variables from the measurement data processing unit 112. The average values ​​of feature vectors and average values ​​of response variables may be configured to be calculated in the pseudo data generation unit 113. Features can be extracted from the measured gait data and the pseudo gait data using the same method.

[0041] The pseudo data generation unit 113 generates a walking waveform (also referred to as pseudo gait data) using the acquired multiple pieces of measured gait data. For example, the pseudo data generation unit 113 generates pseudo gait data by adding temporal fluctuations or noise to the measured gait data. For example, the pseudo data generation unit 113 can generate pseudo gait data based on the measured gait data using the method disclosed in Non-Patent Document 1 (Non-Patent Document 1: L. Tran and D. Choi, “Data Augmentation for Inertial Sensor-Based Gait Deep Neural Network,” IEEE Access, Vol. 8, pp. 12364-12378 (2020)).

[0042] FIG. 10 is a conceptual diagram showing an example of generating time-series data of pseudo gait data (also referred to as pseudo waveform) based on time-series data of measured gait data (also referred to as measured waveform) included in paired data. The example in FIG. 10 is an example of generating a pseudo waveform by adding temporal fluctuations by shifting at least a portion of the measured waveform along the axis of the walking cycle. For example, a pseudo waveform may be generated by shifting at least a portion of the measured waveform in a direction perpendicular to or oblique to the axis of the walking cycle. For example, a pseudo waveform may be generated by adding fluctuations in noise level to at least a portion of the measured waveform.

[0043] The pseudo data generation unit 113 extracts features (also referred to as pseudo features) from each of a plurality of pseudo waveforms (also referred to as pseudo gait data). The pseudo data generation unit 113 calculates a feature vector (also referred to as pseudo feature vector) for each pseudo waveform based on the pseudo features extracted from the pseudo waveforms. The pseudo feature vector is a vector combining the pseudo features extracted from the pseudo waveforms. The pseudo feature vector is composed of pseudo features extracted for each pseudo waveform for one walking cycle.

[0044] For the same subject or the same group of subjects, the pseudo gait data is expected to follow a distribution similar to that of the measured gait data. That is, when calculating pseudo response variables corresponding to pseudo features, the covariance between the pseudo features and the pseudo response variables can be used. In this embodiment, it is assumed that the pseudo gait data has the same covariance as the measured gait data. In this embodiment, the pseudo response variables corresponding to the pseudo features are determined using the pseudo features calculated from the pseudo gait data and the covariance between the features calculated from the measured gait data and the response variables.

[0045] The pseudo data generating unit 113 calculates a deviation vector (also called a pseudo deviation vector) of the pseudo feature vectors for each pseudo waveform by subtracting the average vector of the feature vectors from the pseudo feature vector for each of the multiple pseudo waveforms.

[0046] 11 is a conceptual diagram for explaining an example of generating pseudo deviation vectors D based on pseudo gait data PS. The pseudo data generation unit 113 extracts multiple pseudo feature vectors PF1 to PFn (n is a natural number) from each of the multiple pseudo gait data PS1 to PSn. The pseudo data generation unit 113 calculates each of the pseudo deviation vectors D1 to Dn by subtracting the average vector FS of the multiple feature vectors F1 to Fn from each of the multiple pseudo feature vectors PF1 to PFn.

[0047] The pseudo data generator 113 generates random numerical values ​​that follow a normal distribution with a mean value of 0 and a standard deviation of 1. The generated random numerical values ​​correspond to the variances generated by the random processing. The pseudo data generator 113 generates pseudo variance vectors for each of the multiple pseudo waveforms by appending the generated random numerical values ​​to the end of the pseudo deviation vectors extracted for each of the multiple pseudo waveforms.

[0048] The pseudo data generation unit 113 multiplies the pseudo variance vector for each of the generated multiple pseudo gait data by the last column vector of the upper triangular matrix obtained from the covariance matrix, and calculates the deviation of the pseudo response variable corresponding to each of the multiple pseudo gait data.

[0049] FIG. 12 is a conceptual diagram illustrating an example of calculating the deviation C of a pseudo response variable based on a pseudo deviation vector D extracted from multiple pseudo gait data PS. The pseudo data generation unit 113 generates random numerical values ​​R1 to Rn (n is a natural number) that follow a normal distribution with a mean value of 0 and a standard deviation of 1. The pseudo data generation unit 113 generates pseudo variance vectors DR1 to DRn for each of the multiple pseudo gait data PS by adding each of the random numerical values ​​R1 to Rn to the end of each of the pseudo deviation vectors D1 to Dn of the pseudo feature vectors PF1 to PFn. The pseudo data generation unit 113 calculates the deviations C1 to Cn of the pseudo response variables by multiplying each of the multiple generated pseudo variance vectors DR1 to DRn by the column vector LC at the end of the upper triangular matrix W1 obtained by Cholesky decomposition of the covariance matrix W. The numerical values ​​of the deviations C1 to Cn of the pseudo response variables are calculated by multiplying each of the multiple pseudo variance vectors DR1 to DRn by the column vector LC at the end of the upper triangular matrix W1 obtained by Cholesky decomposition of the covariance matrix W. n (y1~yn is a real number).

[0050] The pseudo data generation unit 113 calculates a pseudo response variable for each pseudo waveform by adding the deviation of the pseudo response variable corresponding to each of the multiple pseudo gait data to the average value of the response variable. The pseudo data generation unit 113 generates a pseudo data set vector in which multiple pseudo response variables are associated with the pseudo feature vector extracted for each of the multiple pseudo waveforms.

[0051] 13 is a conceptual diagram showing an example of calculating pseudo response variables PA for each of a plurality of pseudo gait data PS using the pseudo response variable deviation C calculated for a plurality of pseudo gait data PS and the average value AS of the response variables calculated for a plurality of paired data. The pseudo data generation unit 113 generates pseudo response variables PA1 to PAn (n is a natural number) by adding the average value AS of the response variables to the pseudo response variable deviations C1 to Cn corresponding to each of the plurality of pseudo gait data PS1 to PSn.

[0052] 14 is a conceptual diagram showing an example of generating a pseudo data set vector PFT by associating a plurality of pseudo response variables PA with a pseudo feature vector PF extracted for each of a plurality of pseudo gait data PS. The pseudo data generating unit 113 generates pseudo data set vectors PTF1 to PTFn by associating a plurality of pseudo response variables PA1 to PAn with the pseudo feature vectors PF1 to PFn extracted for each of a plurality of pseudo gait data PS1 to PSn (n is a natural number).

[0053] The output unit 115 outputs a plurality of data sets, each of which is composed of the measurement data set vector generated by the measurement data processing unit 112 and the pseudo data set vector generated by the pseudo data generating unit 113, to the learning device 15. The plurality of data sets output from the output unit 115 are used by the learning device 15 to generate an estimation model.

[0054] As described above, the data generating device 11 of this embodiment can expand the dataset used to generate an estimation model by using paired data including gait data based on actually measured sensor data. If the method of Non-Patent Document 1 is used, the dataset can be expanded when the response variable is fixed. However, the method of Non-Patent Document 1 cannot appropriately expand the dataset when the response variable is a variable. The data generating device 11 of this embodiment can calculate pseudo response variables corresponding to pseudo feature vectors extracted from pseudo gait data, even when the response variable is a variable, and therefore can appropriately expand the dataset.

[0055] In the configuration of FIG. 1, the measurement data processing unit 112 generates a measurement data set vector, and the pseudo data generation unit 113 generates a pseudo data set vector. For example, a common feature extraction unit (not shown) may be configured to extract features (pseudo features) from a pseudo waveform generated based on a measurement waveform and from the measurement waveform. In that case, the pseudo response variables may be generated based on the pseudo features using the above-mentioned method. For example, a common dataset vector generation unit (not shown) can generate dataset vectors for the pseudo gait data and the measured gait data.

[0056] The data set can be expanded by the data generating device 11 at any timing. For example, the data generating device 11 can expand the data set by using paired data accumulated in the past. For example, the data generating device 11 can also expand the data set in real time in accordance with measurement of sensor data (gait data) in addition to paired data accumulated in the past.

[0057] [Learning device] The following describes the learning device 15 of the learning system 10. Fig. 15 is a block diagram showing an example of the configuration of the learning device 15. The learning device 15 has a dataset acquisition unit 151, a learning unit 153, and a storage unit 155.

[0058] The dataset acquisition unit 151 acquires a plurality of datasets generated by the data generation device 11. The plurality of datasets include measurement dataset vectors and pseudo dataset vectors. The dataset acquisition unit 151 stores the acquired datasets in the storage unit 155. The measurement dataset vectors and the pseudo dataset vectors may be stored in the storage unit 155 so as to be distinguishable, or may be stored in the storage unit 155 without being distinguishable.

[0059] The learning unit 153 acquires multiple data sets from the storage unit 155. The learning unit 153 extracts feature vectors and response variables from the multiple data sets. If the data set vector is a measured data set vector, the learning unit 153 extracts feature vectors and response variables. If the data set vector is a pseudo data set vector, the learning unit 153 extracts pseudo feature vectors and pseudo response variables. Hereinafter, the pseudo feature vectors and pseudo response variables extracted from the pseudo data set vectors will be referred to as feature vectors and response variables, without distinguishing them from the feature vectors and response variables extracted from the measured data set vectors.

[0060] The learning unit 153 generates an estimation model using feature vectors and response variables of multiple data sets. The learning unit 153 generates an estimation model that outputs response variables (gait event indices) in response to input gait data. For example, the learning unit 153 uses some of the multiple data sets as learning data (training data) and the remaining some as verification data or test data.

[0061] For example, the learning unit 153 may generate an estimation model dedicated to a user using the feature vectors and response variables of a data set generated based on paired data measured according to the user's walking. For example, a personal identifier for uniquely identifying the user from whom the sensor data was acquired may be assigned to the sensor data, and an estimation model may be generated using the sensor data assigned the same personal identifier. A user-specific estimation model may not be versatile enough for everyone, but it can be specialized for that user and accurately estimate the physical state.

[0062] For example, the learning unit 153 may generate an estimation model using a feature vector and a response variable of a data set generated based on paired data measured according to attributes such as gender, age, weight, height, etc. An estimation model according to an attribute may not be general-purpose for all attributes, but it can be specialized for a user with that attribute and accurately estimate a physical condition.

[0063] For example, the learning unit 153 may generate an estimation model using a feature vector and a response variable of a dataset generated based on paired data measured according to a characteristic symptom such as knee or leg pain. Although an estimation model according to a symptom may not be general-purpose for all symptoms, it can be specialized for that symptom and accurately estimate a physical condition.

[0064] The storage unit 155 stores a plurality of data sets. The storage unit 155 also stores an estimation model generated by the learning unit 153. The estimation model stored in the storage unit 155 is implemented in an estimation device (not shown) that estimates the physical state of a user using the user's gait data. There are no particular limitations on the method for implementing the estimation model in the estimation device.

[0065] Learning by the learning device 15 using a data set can be performed at any timing. For example, the learning device 15 can perform learning using a data set that has been expanded using paired data accumulated in the past. For example, the data generating device 11 can perform learning using a data set that has been expanded in real time in response to measurement of sensor data (gait data) in addition to paired data accumulated in the past. If learning is performed using a data set that has been expanded in real time, an estimation model that better reflects the user's current physical state can be constructed.

[0066] (operation) Next, the operation of learning system 10 will be described with reference to the drawings. In the following, the operations of data generating device 11 and learning device 15 included in learning system 10 will be described separately.

[0067] [Data Generator] First, the operation of data generating device 11 will be described with reference to the drawings. Fig. 16 is a flowchart for explaining an example of the operation of data generating device 11. In the explanation following the flowchart of Fig. 16, the data generating device 11 will be described as the subject of the operation.

[0068] 16, first, the data generating device 11 acquires paired data to be processed (step S101). For example, the data generating device 11 acquires paired data stored in a cloud or a server. For example, the data generating device 11 may be configured to acquire paired data directly from a measuring device 110 placed on the user's foot.

[0069] Next, the data generating device 11 executes measurement data processing for the pair data to be processed (step S102). In the measurement data processing, the data generating device 11 generates a measurement data set vector based on the data set to be processed. Details of the measurement data processing will be described later.

[0070] Next, the data generating device 11 executes a pseudo data generation process for the data set to be processed (step S103). In the pseudo data generation process, the data generating device 11 generates a pseudo data set vector based on the data set to be processed. The pseudo data generation process will be described in detail later.

[0071] Next, the data generating device 11 outputs the generated data set (measured data set vector / pseudo data set vector) (step S104). The output data set is used for learning by the learning device 15.

[0072] [Measurement data processing] Next, measurement data processing by the data generating device 11 (step S102 in FIG. 16) will be described with reference to the drawings. FIG. 17 is a flowchart for explaining an example of measurement data processing by the data generating device 11. In the description following the flowchart in FIG. 17, the data generating device 11 will be described as the subject of operations. Vectors and numerical values ​​derived in the description following the flowchart in FIG. 17 are stored in a storage unit (not shown).

[0073] In FIG. 17, first, the data generating device 11 extracts feature amounts from the measured gait data included in the paired data to be processed (step S111).

[0074] Next, the data generating device 11 generates a feature vector for each pair data using the extracted feature (step S112).

[0075] Next, the data generating device 11 calculates the average vector of the feature amount vectors generated for the measured gait data for each paired data to be processed (step S113). The calculated average vector is used in the pseudo data generating process, which will be described later.

[0076] Next, the data generating device 11 calculates the average value of the response variables for the plurality of measured gait data for the data set to be processed (step S114). The calculated average value of the response variables is used in the pseudo data generating process, which will be described later.

[0077] Next, the data generating device 11 generates a feature vector and a measurement dataset vector of the response variable for each pair of data (step S115). The generated measurement dataset vector is output as a dataset in step S104 of FIG.

[0078] Next, the data generating device 11 generates a covariance matrix for the generated measurement data set vectors (step S116).

[0079] Next, the data generating device 11 performs Cholesky decomposition on the calculated covariance matrix to derive an upper triangular matrix (step S117). The last column of the upper triangular matrix derived by performing Cholesky decomposition on the covariance matrix is ​​used in the pseudo data generating process described later.

[0080] [Pseudo data generation process] Next, the pseudo data generation process (step S103 in FIG. 16) by the data generation device 11 will be described with reference to the drawings. FIG. 18 is a flowchart for explaining an example of the pseudo data generation process by the data generation device 11. In the description following the flowchart in FIG. 18, the data generation device 11 will be described as the subject of the operations. Vectors and numerical values ​​derived in the description following the flowchart in FIG. 18 are stored in a storage unit (not shown).

[0081] In FIG. 18, first, the data generating device 11 generates pseudo gait data using measured gait data included in the paired data to be processed (step S121).

[0082] Next, the data generating device 11 extracts pseudo-features from the generated pseudo-gait data (step S122).

[0083] Next, the data generating device 11 uses the extracted pseudo feature amounts to calculate a pseudo feature amount vector for each pseudo gait data (step S123).

[0084] Next, the data generating device 11 calculates a pseudo deviation vector using the pseudo feature vector and the average feature vector for each pseudo gait data (step S124). The data generating device 11 calculates the pseudo deviation vector by subtracting the average feature vector from the pseudo feature vector.

[0085] Next, the data generating device 11 generates a pseudo variance vector by adding a random value to the end of the pseudo deviation vector for each pseudo gait data (step S125).

[0086] Next, the data generating device 11 multiplies the pseudo variance vector for each pseudo gait data by the last column of the upper triangular matrix derived by Cholesky decomposition of the covariance matrix to calculate the deviation of the pseudo response variable (step S126).

[0087] Next, the data generating device 11 calculates the pseudo response variables by adding the deviation of the pseudo response variables for each pseudo gait data to the average value of the response variables (step S127).

[0088] Next, the data generating device 11 generates a pseudo data set vector by combining the pseudo feature quantities and pseudo response variables for each pseudo gait data (step S128). The generated pseudo data set vector is output as a data set in step S104 of FIG.

[0089] [Learning device] Next, the operation of learning device 15 will be described with reference to the drawings. Fig. 19 is a flowchart for explaining an example of the operation of learning device 15. In the explanation following the flowchart of Fig. 19, the learning device 15 will be described as the subject of the operation.

[0090] 16, first, the learning device 15 acquires a data set generated by the data generating device 11 (step S131). The data set includes a measured data set vector and a pseudo data set vector.

[0091] Next, the learning device 15 stores the acquired data set (step S132). Data sets are classified into learning data sets, verification data sets, and test data sets.

[0092] Next, the learning device 15 generates an estimation model using a learning data set from among the stored data sets (step S133).

[0093] Next, the learning device 15 verifies / tests the estimated model using a validation / test dataset from the stored datasets (step S134). For example, the learning device 15 performs validation using the validation dataset followed by testing using a test dataset. For example, the learning device 15 performs cross-validation such as k-fold, holdout, leave-one-out, random iterative subsampling, stratification, and resubstitution using the validation dataset.

[0094] If adjustment of the estimation model is necessary (Yes in step S135), the learning device 15 adjusts the parameters of the estimation model (step S136). For example, the learning device 15 adjusts the parameters of the estimation model if a predetermined index such as accuracy, generalization error, accuracy rate, precision rate, recall rate, F-measure, etc. does not satisfy a standard. After step S136, the process returns to step S134.

[0095] If adjustment of the estimation model is not necessary (No in step S135), the learning device 15 stores the estimation model (step S137). For example, the learning device 15 stores the estimation model when predetermined indices such as accuracy, generalization error, precision, precision, recall, and F-measure satisfy criteria. The estimation model generated by the learning device 15 is implemented in an estimation device (not shown) that estimates a body state using gait data.

[0096] [Example of evaluation of estimation model] Next, an example of the results of evaluating an estimation model generated using the method of this embodiment will be described. The following shows the results of evaluating the correlation between the true values ​​of response variables included in a dataset containing actually measured gait data and the estimated values ​​of the response variables obtained by inputting the measured gait data included in the dataset into an estimation device.

[0097] First, we explain the improvement of intraclass correlation coefficients (ICCs) due to data expansion. Here, the Center of Pressure Excursion Index (CPEI), an index for evaluating foot pronation / supination, was used as the response variable. The original measurement dataset vectors consisted of 500 steps for 32 men. In this evaluation, we generated pseudo-gait data for 6,000 steps for 32 men by introducing random temporal fluctuations into the measured gait data for each step contained in the original measurement dataset vectors. Using leave-one-subject-out (LOSO), we used one subject's measurement dataset vector for testing and combined the other subjects' measurement dataset vectors with the pseudo-dataset vectors to generate an estimation model. LOSO was performed on all 32 subjects, and the predicted values ​​for the test were compared with the true values.

[0098] Figure 20 is a graph showing the correlation between the true and estimated values ​​of CPEI without data augmentation. In the example in Figure 20, the measurement dataset vector was used as is. In the example in Figure 20, the intraclass correlation coefficient (ICC) between the true and estimated values ​​of CPEI was 0.6253. Figure 21 is a graph showing the correlation between the true and estimated values ​​of CPEI with data augmentation. In the example in Figure 21, the measurement dataset vector and the pseudo-measurement dataset vector were used. In the example in Figure 21, the intraclass correlation coefficient (ICC) between the true and estimated values ​​of CPEI was 0.7040. In the examples in Figures 20 and 21, the intraclass correlation coefficient (ICC) between the true and estimated values ​​of CPEI was better when data augmentation was performed. In other words, a more accurate estimation model could be constructed when data augmentation was performed.

[0099] Next, we present the results of evaluating the estimation model generated using the measurement data set vectors of 32 men on the measurement data set vectors of 12 other men.

[0100] Figure 22 is a graph showing the correlation between the true and estimated values ​​of the CPEI when using an estimation model without data augmentation (the estimation model in Figure 20). In the example in Figure 22, the intraclass correlation coefficient (ICC) between the true and estimated values ​​of the CPEI was 0.6486. Figure 23 is a graph showing the correlation between the true and estimated values ​​of the CPEI when using a data-augmented estimation model (the estimation model in Figure 21). In the example in Figure 23, the intraclass correlation coefficient (ICC) between the true and estimated values ​​of the CPEI was 0.6862. In the examples in Figures 22 and 23, the data-augmented estimation model showed a better intraclass correlation coefficient (ICC) between the true and estimated values ​​of the CPEI. In other words, data augmentation allowed the construction of a more versatile estimation model.

[0101] (Application example) Next, an application example of this embodiment will be described with reference to the drawings. In this application example of this embodiment, a measurement device is installed in a user's shoe, and gait data measured by the measurement device is transmitted to a mobile device carried by the user. The sensor data transmitted to the mobile device is processed by an estimation device implemented in the mobile device. For example, the functions of the estimation device are provided as an application that can be installed on the mobile device.

[0102] [Application Example 1-1] 24 and 25 are conceptual diagrams for explaining Application Example 1-1. In this application example, an algorithm for generating a data set using measured gait data is visualized and displayed on the screen of a terminal device that can be viewed by the user.

[0103] Fig. 24 shows an example in which an algorithm for generating measured data set vectors FA1 to FAn using measured gait data S1 to Sn is visualized and displayed on a screen 190 of a terminal device that can be viewed by a user. In the example of Fig. 24, the average vector FS of the feature quantity vectors F1 to Fn extracted from the measured gait data S1 to Sn and the average value AS of the response variables A1 to An are also displayed on screen 190. In the example of Fig. 24, the covariance matrix W related to the measured data set vectors FA1 to FAn and the upper triangular matrix W1 derived by Cholesky decomposition of the covariance matrix W are also displayed on screen 190. By referring to the algorithm visualized on screen 190, the process by which the measured data set vectors FA1 to FAn are generated using the measured gait data S1 to Sn, the covariance matrix, etc. can be intuitively grasped.

[0104] Fig. 25 is an example in which an algorithm for generating pseudo data set vectors PFA1 to PFAn using measured gait data S1 to Sn is visualized and displayed on a screen 190 of a terminal device that can be viewed by a user. Fig. 25 shows the process of calculating pseudo feature vectors PF1 to PFn, pseudo deviation vectors D1 to Dn, pseudo variance vectors DR1 to DRn, and pseudo response variable deviations C1 to Cn to calculate pseudo response variables PA1 to PAn, displayed on screen 190. Fig. 25 also shows the mean vector FS of the feature vectors F1 to Fn, the mean value AS of the response variables A1 to An, and the last column vector LC of the upper triangular matrix W1, which are used to calculate the pseudo response variables PA1 to PAn. By referring to the algorithm visualized on screen 190, the process of generating pseudo data set vectors PFA1 to PFAn using measured gait data S1 to Sn can be intuitively grasped. For example, if there is a problem with the accuracy or versatility of an estimation model generated by learning using a dataset, you can visually verify which process is causing the problem by referring to the algorithm displayed on screen 190.

[0105] [Application Example 1-2] 26 is a conceptual diagram for explaining Application Example 1-2. In this application example, information related to learning using a data set generated by the data generating device 11 is displayed on a screen of a terminal device that can be viewed by a user.

[0106] 26 shows an example in which information about the accuracy and versatility of the estimation model 150 generated when the learning device 15 learns the data set generated by the data generating device 11 is displayed on the screen of a terminal device that can be viewed by the user. In the example of FIG. 26, the measured data set vector FA and the pseudo data set vector PFAThe learning device 15 was made to learn the above, and the state in which the estimation model 150 was generated was displayed on the screen 190. Information regarding the accuracy of the estimation model, such as "The accuracy of the estimation model is XX," was displayed on the screen 190. Information regarding versatility, such as "The accuracy / versatility is improved by using the pseudo dataset vector," was also displayed on the screen 190.

[0107] According to this application example, the effect of learning using the data set generated by the data generating device 11 can be grasped based on the information on accuracy and versatility displayed on the screen 190.

[0108] As described above, the learning system of this embodiment includes a data generating device and a learning device. The data generating device includes an acquiring unit, a measured data processing unit, a pseudo data generating unit, and an output unit. The acquiring unit acquires paired data combining measured gait data related to sensor data measured in response to foot movements of a user and response variables corresponding to the measured gait data. The measured data processing unit generates a measured data set vector by combining a feature vector calculated using features extracted from the measured gait data with the response variables. The measured data processing unit generates a covariance matrix related to the plurality of paired data. The pseudo data generating unit generates pseudo gait data using the measured gait data. The pseudo data generating unit generates a pseudo data set vector by combining a pseudo feature vector calculated using pseudo features extracted from the pseudo gait data with a pseudo response variable generated using a covariance matrix related to the pseudo feature vector. The output unit outputs a data set including the measured data set vector and the pseudo data set vector. The learning device acquires the data set output from the data generating device. The learning device uses the acquired data set to generate an estimation model that outputs a response variable according to the user's physical state in response to input of measured gait data.

[0109] Gait data based on sensor data related to foot movement contains features specific to gait events related to health status. Generating a model (estimation model) trained with learning data associating gait data with gait events corresponding to the gait data allows gait events to be estimated based on measured gait data. Generating an estimation model requires collecting learning data for learning the association between gait data and gait events. Generating an estimation model with sufficient accuracy requires a large amount of learning data. Collecting a large amount of learning data requires a great deal of time and effort. The method of Non-Patent Document 1 can assign labels such as identifiers to pseudo gait data. However, the method of Non-Patent Document 1 cannot associate numerical values ​​(gait event indices) corresponding to gait data with the pseudo gait data. The method of this embodiment can generate numerical values ​​(gait event indices) corresponding to gait data in association with pseudo gait data. In other words, the method of this embodiment can generate a dataset used for learning even if the response variable is a continuous numerical value. Therefore, the method of this embodiment can save time and effort spent on collecting training data and build a highly accurate estimation model.

[0110] In one aspect of this embodiment, the pseudo data generation unit generates a plurality of pseudo gait data by adding fluctuations to a plurality of measured gait data included in a plurality of paired data. According to this aspect, adding fluctuations to the plurality of measured gait data can increase the amount of pseudo gait data.

[0111] In one aspect of this embodiment, the pseudo data generation unit generates a plurality of pseudo gait data by adding noise to a plurality of measured gait data included in a plurality of paired data. According to this aspect, the amount of pseudo gait data can be increased by adding noise to the plurality of measured gait data.

[0112] In one aspect of this embodiment, the pseudo data generation unit generates a plurality of pseudo gait data sets using measured gait data. The pseudo data generation unit extracts at least one pseudo feature from the generated pseudo gait data. The pseudo data generation unit generates a pseudo feature vector for each pseudo gait data set using the pseudo feature extracted from the pseudo gait data. According to this aspect, a feature vector used to generate a pseudo data set vector can be generated for each pseudo gait data set.

[0113] In one aspect of this embodiment, the measured data processing unit extracts at least one feature corresponding to a gait event from measured gait data derived using sensor data for each walking cycle. A feature vector is generated for each measured gait data using the feature extracted from the measured gait data for each walking cycle. A measured data set vector for each measured gait data is generated by adding a response variable associated with the measured gait data to the end of the feature vector generated for each measured gait data. According to this aspect, a measured data set vector can be generated for each data pair.

[0114] In one aspect of this embodiment, the measurement data processing unit calculates an average vector of multiple feature vectors calculated from multiple measured gait data for multiple paired data. The measurement data processing unit generates a covariance matrix for multiple measurement data set vectors generated for each measured gait data. The measurement data processing unit derives an upper triangular matrix of the covariance matrix by Cholesky decomposition of the generated covariance matrix. The measurement data processing unit calculates average values ​​of multiple response variables for the multiple paired data. The pseudo data generation unit calculates a pseudo deviation vector for each of the multiple pseudo gait data by subtracting the average vector of the feature vectors from each of the multiple pseudo gait data. The pseudo data generation unit generates pseudo variance vectors by assigning a random value between 0 and 1 to the end of each of the calculated multiple pseudo deviation vectors. The pseudo data generation unit multiplies each of the generated multiple pseudo variance vectors by the last column of the upper triangular matrix to calculate the deviation of the pseudo response variable for each pseudo gait data. The pseudo data generation unit calculates pseudo response variables corresponding to the pseudo gait data by adding the deviation of the pseudo response variables calculated for each pseudo gait data to the average value of the response variables. According to this aspect, a pseudo data set vector can be generated by combining a pseudo feature vector calculated using pseudo features extracted from the pseudo gait data with a pseudo response variable generated using a covariance matrix related to a plurality of paired data.

[0115] In one aspect of the present embodiment, the output unit displays information about the generated dataset on a screen of the terminal device. According to this aspect, the dataset generated by the data generation unit can be confirmed on the screen of the terminal device.

[0116] (Second embodiment) Next, an estimation system according to a second embodiment will be described with reference to the drawings. The estimation system according to the second embodiment estimates a physical state of a user using an estimation model generated by the learning device according to the first embodiment.

[0117] (composition) 27 is a block diagram showing the configuration of an estimation system 20 according to this embodiment. The estimation system 20 includes a measurement device 21 and an estimation device 25. For example, the estimation system 20 may be configured with only the estimation device 25, excluding the measurement device 21. Below, the measurement device 21 and the estimation device 25 will be described separately.

[0118] [Measuring equipment] 28 is a block diagram showing an example of the configuration of the measurement device 21. The measurement device 21 has a sensor 22 and a measurement unit 23. The sensor 22 includes an acceleration sensor 221 and an angular velocity sensor 222. The measurement unit 23 includes an acquisition unit 231, a storage unit 233, a calculation unit 235, and a transmission unit 237. The measurement device 21 is installed on the foot. For example, the coordinate system and the like set in the measurement device 21 installed on footwear such as a shoe are the same as those in the first embodiment.

[0119] FIG. 29 is a conceptual diagram showing an example in which the measuring device 21 is placed inside the shoe 200. In the example of FIG. 29, the measuring device 21 is placed at a position corresponding to the back of the arch of the foot. For example, the measuring device 21 is placed in an insole inserted into the shoe 200. For example, the measuring device 21 is placed on the bottom surface of the shoe 200. For example, the measuring device 21 is embedded in the body of the shoe 200. The measuring device 21 may be detachable from the shoe 200 or may not be detachable from the shoe 200. Note that the measuring device 21 may be placed at a position other than the back of the arch of the foot as long as it can acquire sensor data related to foot movement. The measuring device 21 may also be placed in socks worn by the user or in an accessory such as an anklet worn by the user. The measuring device 21 may also be attached directly to the foot or embedded in the foot. FIG. 29 shows an example in which the measuring device 21 is placed in the shoes 200 of both feet. The measuring device 21 may also be placed in the shoe 200 of one foot.

[0120] The acceleration sensor 221 is a sensor that measures acceleration in three axial directions (also called spatial acceleration). The acceleration sensor 221 measures acceleration (also called spatial acceleration) as a physical quantity related to foot movement. The acceleration sensor 221 outputs the measured acceleration to the measurement unit 23. For example, a piezoelectric, piezo-resistive, or capacitance type sensor can be used as the acceleration sensor 221. There is no limitation on the measurement type of the sensor used as the acceleration sensor 221 as long as it can measure acceleration.

[0121] The angular velocity sensor 222 is a sensor that measures angular velocity (also called spatial angular velocity) in three axial directions. The angular velocity sensor 222 measures angular velocity (also called spatial angular velocity) as a physical quantity related to foot movement. The angular velocity sensor 222 outputs the measured angular velocity to the measurement unit 23. For example, a vibration type or capacitance type sensor can be used as the angular velocity sensor 222. There is no limitation on the measurement method of the sensor used as the angular velocity sensor 222 as long as it can measure angular velocity.

[0122] The sensor 22 is realized by, for example, an inertial measurement unit that measures acceleration and angular velocity. An example of an inertial measurement unit is an IMU (Inertial Measurement Unit). The IMU includes an acceleration sensor 221 that measures acceleration in three axial directions and an angular velocity sensor 222 that measures angular velocity around three axes. The sensor 22 may be realized by an inertial measurement unit such as a VG (Vertical Gyro) or an AHRS (Attitude Heading). The sensor 22 may also be realized by a GPS / INS (Global Positioning System / Inertial Navigation System). The sensor 22 may be realized by a device other than an inertial measurement unit as long as it can measure physical quantities related to foot movement.

[0123] The acquisition unit 231 acquires acceleration in three axial directions from the acceleration sensor 221. The acquisition unit 231 also acquires angular velocities around three axes from the angular velocity sensor 222. The acquisition unit 231 converts the acquired acceleration and angular velocities into digital data and stores the converted digital data (also referred to as sensor data) in the storage unit 233. The acquisition unit 231 may be configured to directly output the sensor data to the calculation unit 235. The sensor data includes at least acceleration data converted into digital data and angular velocity data converted into digital data. The acceleration data includes acceleration vectors in three axial directions. The angular velocity data includes angular velocity vectors around three axes. The acceleration data and angular velocity data are associated with the acquisition times of the data. The acquisition unit 231 may also apply corrections such as mounting error correction, temperature correction, and linearity correction to the acceleration data and angular velocity data.

[0124] The storage unit 233 stores the sensor data. The sensor data stored in the storage unit 233 is used for calculating gait parameters by the calculation unit 235. When omitting the calculation of gait parameters and transmitting the sensor data directly to the estimation device 25, the sensor data stored in the storage unit 233 may be configured to be transmitted directly from the transmission unit 237.

[0125] The calculation unit 235 acquires sensor data from the storage unit 233. The calculation unit 235 may be configured to acquire sensor data directly from the acquisition unit 231. For example, the calculation unit 235 converts the coordinate system of the acquired sensor data from a local coordinate system to a world coordinate system. When the user is standing upright, the local coordinate system (x-axis, y-axis, z-axis) and the world coordinate system (x-axis, y-axis, z-axis) coincide with each other. While the user is walking, the spatial orientation of the measurement device 21 changes, and therefore the local coordinate system (x-axis, y-axis, z-axis) and the world coordinate system (x-axis, y-axis, z-axis) do not coincide with each other. Therefore, the calculation unit 235 converts the sensor data acquired by the measurement device 21 from the local coordinate system (x-axis, y-axis, z-axis) of the measurement device 21 to the world coordinate system (x-axis, y-axis, z-axis). When a gait event can be detected using sensor data in the local coordinate system, the coordinate conversion from the local coordinate system to the world coordinate system may be omitted.

[0126] The calculation unit 235 uses the acquired sensor data to generate time series data of physical quantities related to foot movement measured as the user walks while wearing the shoe 200 on which the measuring device 21 is installed. For example, the calculation unit 235 generates time series data such as spatial acceleration and spatial angular velocity. The calculation unit 235 also integrates the spatial acceleration and spatial angular velocity to generate time series data such as spatial velocity, spatial angle (sole angle), and spatial trajectory. The calculation unit 235 generates the time series data at predetermined timings or time intervals set in accordance with a general walking cycle or a walking cycle specific to the user. The timing at which the calculation unit 235 generates the time series data can be set arbitrarily. For example, the calculation unit 235 is configured to continue generating time series data for the period during which the user continues walking. For example, the calculation unit 235 may be configured to generate time series data at specific timings.

[0127] When calculating gait parameters, the calculation unit 235 extracts time-series data of sensor data (also referred to as gait waveform) for one walking cycle from the generated time-series data. Here, the gait waveform is not a graph of the time-series data of the sensor data, but the time-series data of the sensor data itself. For example, the calculation unit 235 detects the timing of the center of the stance phase as the start point of the time-series data. For example, the calculation unit 235 may detect the timing of heel strike or toe lift as the start point of the gait waveform. When gait parameters are not calculated, the calculation unit 235 simply outputs the generated time-series data to the transmission unit 237.

[0128] The calculation unit 235 detects gait events from the extracted gait waveform for one step cycle. For example, the calculation unit 235 detects the timing of characteristic changes associated with the occurrence of gait events in the gait waveform. For example, the calculation unit 235 detects the timing of characteristic maximums and minimums associated with the occurrence of gait events in the gait waveform.

[0129] For example, the calculation unit 235 detects gait events such as heel strike, toe off, foot crossing, heel lift, tibia vertical, opposite toe off, and opposite heel strike. In this embodiment, a state in which the toe is positioned below the heel (plantar flexion) is considered positive, and a state in which the toe is positioned above the heel (dorsiflexion) is considered negative. In plantar flexion, the roll angle is maximized at the timing of toe off. For example, the calculation unit 235 detects the timing at which the roll angle is maximized in the gait waveform of a stride cycle as the timing of toe off. In dorsiflexion, the roll angle is minimized at the timing of heel strike. For example, the calculation unit 235 detects the timing at which the roll angle is minimized in the gait waveform of a stride cycle as the timing of heel strike. For example, the calculation unit 235 detects the timing of the center of the stance phase from the gait waveform of the roll angle. In reality, the timings at which the roll angle reaches its maximum / minimum and the timings of toe-off and heel-striking do not perfectly coincide. Therefore, the gait cycle may be normalized so that the timings at which the roll angle reaches its maximum / minimum and the timings of toe-off and heel-striking coincide in the gait waveform for one step cycle. Normalizing the gait waveform makes it possible to align the timings of gait events, which vary from person to person.

[0130] The calculation unit 235 calculates gait parameters based on the detected gait events. For example, the calculation unit 235 calculates the gait parameters using the timing of the detected gait events and the values ​​of the sensor data at the timing of these gait events. For example, the calculation unit 235 calculates the gait parameters for each stride cycle. For example, the calculation unit 235 calculates gait parameters such as walking speed, stride length, contact angle, take-off angle, maximum foot lift height (sensor position), circular movement (traveling direction trajectory), and toe direction. Explanation of how these gait parameters are calculated will be omitted. The gait parameters and sensor data correspond to gait data.

[0131] The transmitter 237 acquires gait data from the calculator 235. The transmitter 237 transmits the acquired gait data to a mobile terminal or the like (not shown) on which the estimation device 25 is implemented. For example, the transmitter 237 transmits the gait data to the mobile terminal or the like via a wired connection such as a cable. For example, the transmitter 237 transmits the gait data to the mobile terminal or the like via wireless communication. For example, the transmitter 237 transmits the gait data to the mobile terminal or the like via a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the transmitter 237 may be conforming to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark).

[0132] For example, the measurement unit 23 is realized by a microcomputer or a microcontroller. For example, the measurement unit 23 has a control circuit and a memory circuit. For example, the control circuit is realized by a CPU (Central Processing Unit). For example, the memory circuit is realized by a volatile memory such as a RAM (Random Access Memory). For example, the memory circuit is realized by a non-volatile memory such as a ROM (Read Only Memory) or an EEPROM (Electrically Erasable and Programmable Read Only Memory).

[0133] [Estimation device] Next, a detailed configuration of the estimation device 25 included in the estimation system 20 will be described with reference to the drawings. FIG. 30 is a block diagram showing an example of the configuration of the estimation device 25. The estimation device 25 has a receiving unit 251, a storage unit 253, an estimation unit 255, and an estimation result output unit 257. For example, the estimation device 25 is implemented in a mobile terminal (not shown) carried by a user. For example, the estimation device 25 is realized by application software or the like installed on the mobile terminal (not shown) carried by the user.

[0134] The receiving unit 251 receives gait data such as gait parameters and sensor data from the measurement device 21. The receiving unit 251 outputs the received gait data to the estimating unit 255. The receiving unit 251 may store the received gait data in the storage unit 253. For example, the receiving unit 251 receives gait data from the measurement device 21 via wireless communication. For example, the receiving unit 251 is configured to receive gait data from the measurement device 21 via a wireless communication function (not shown) that complies with standards such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the receiving unit 251 may be compliant with standards other than Bluetooth (registered trademark) or WiFi (registered trademark). For example, the receiving unit 251 may receive gait data from the measurement device 21 via a wired connection such as a cable.

[0135] The storage unit 253 stores the estimation model generated by the learning device 15 of the first embodiment. The estimation model stored in the storage unit 253 is used for estimation by the estimation unit 255. The storage unit 253 may store data received by the reception unit 251.

[0136] The estimation unit 255 acquires gait data such as time-series data of sensor data and gait parameters from the receiving unit 251. The estimation unit 255 extracts feature amounts from the acquired gait data. The estimation unit 255 inputs the extracted feature amounts to an estimation model and outputs the results (gait event indices) output from the estimation model.

[0137] FIG. 31 is a conceptual diagram for explaining an example of estimation by the estimation unit 255. In the example of FIG. 31, estimation is performed using the estimation model 250 generated by the method of the first embodiment. The estimation model 250 outputs gait event data (response variables) in response to input of feature amounts extracted from gait data based on sensor data measured by the measurement device 21. The estimation model 250 outputs gait event data as an estimation result in response to input of feature amounts FV1 to FVm (m is a natural number). For example, in the case of the estimation model 250 estimating the progression level of hallux valgus, the progression level corresponding to the input feature amounts FV1 to FVm is output from the estimation model 250. For example, in the case of the estimation model 250 estimating the degree of pronation / supination of the foot, the degree corresponding to the input feature amounts FV1 to FVm is output from the estimation model 250. There are no limitations on the estimation result estimated by the estimation model 250 as long as gait event data (response variables) related to the body state can be output as an estimation result in response to input of feature amounts.

[0138] For example, the estimation unit 255 estimates a score related to the user's gait. For example, the score is a numerical value related to the evaluation of the user's gait. For example, the estimation unit 255 estimates the user's physical condition. For example, the physical condition includes the degree of pronation / supination of the foot, the progression of hallux valgus, the progression of knee osteoarthritis, muscle strength, balance ability, body flexibility, etc. The estimation process by the estimation unit 255 is not particularly limited as long as it is related to gait.

[0139] For example, the estimation unit 255 estimates the physical condition according to values ​​related to the physical condition, such as the foot pressure center locus index CPEI and the HV angle. For example, with respect to the foot pressure center locus index CPEI, the estimation device 25 estimates pronation if it is 9 or less, normal if it is 9 to 20, and supination if it is 20 or more. For example, with respect to the HV angle, the estimation unit 255 estimates hallux valgus if it exceeds 20 degrees, estimates a tendency toward hallux valgus if it exceeds a predetermined threshold less than 20 degrees, and estimates normal if it is equal to or less than the predetermined threshold. There are no particular limitations on the estimation of the physical condition by the estimation unit 255 as long as it is related to gait.

[0140] The estimation result output unit 257 acquires the estimation result by the estimation unit 255. The estimation result output unit 257 outputs the estimation result acquired from the estimation unit 255. For example, the estimation result output unit 257 outputs the estimation result by the estimation unit 255 to a display device (not shown). For example, the estimation result by the estimation unit 255 is displayed on the screen of the display device. For example, the estimation result by the estimation unit 255 is output to an external system that uses the estimation result. There are no particular limitations on the use of the estimation result by the estimation unit 255.

[0141] A mobile terminal (not shown) capable of implementing the estimation device 25 is a communication device that can be carried by a user. For example, the mobile terminal is a mobile terminal device with a communication function, such as a smartphone, a smart watch, a tablet, or a mobile phone. The mobile terminal receives gait data from the measurement device 21. The mobile terminal estimates the user's physical state using the received gait data. For example, the measurement device 21 displays the results of data processing of the gait data on the screen of the mobile terminal. For example, the results of data processing of the gait data may be displayed on the screen of a terminal device (not shown) that is visible to the user. For example, the estimation device 25 displays at least some numerical values ​​of the gait data received from the measurement device 21 on the screen of the mobile terminal in real time. For example, the estimation device 25 displays time-series data of the gait data received from the measurement device 21 on the screen of the mobile terminal in real time. Furthermore, the estimation device 25 may transmit an estimation result using the received gait data to a server, a cloud, or the like. There are no particular limitations on the use of the estimation result estimated by the estimation device 25. Furthermore, if the gait data measured by the measuring device 21 can be acquired, the estimation device 25 may be installed on the server or cloud side.

[0142] (operation) Next, the operation of the estimation system 20 will be described with reference to the drawings. In the following, the operations of the measurement device 21 and the estimation device 25 included in the estimation system 20 will be described separately.

[0143] [Measuring equipment] First, the operation of the measuring device 21 will be described with reference to the drawings.

[0144] 32, the measuring device 21 operates in a vibration detection mode (step S201). For example, the measuring device 21 is activated in response to a user operation and begins to operate in the vibration detection mode. For example, the measuring device 21 is set to activate at a preset time period or timing.

[0145] If vibration is detected within the predetermined period (Yes in step S202), the measuring device 21 executes sensor data measurement processing (step S203). In the sensor data measurement processing in step S203, the measuring device 21 measures sensor data. Details of the sensor data measurement processing in step S203 will be described later. If vibration is not detected within the predetermined period (No in step S202), the process proceeds to step S206.

[0146] After step S203, the measurement device 21 executes a gait parameter calculation process (step S204). In the gait parameter calculation process of step S204, the measurement device 21 calculates gait parameters using the sensor data measured in the sensor data measurement process of step S203. The gait parameter calculation process of step S204 will be described in detail later.

[0147] After step S204, if the measurement of sensor data is to be continued (Yes in step S205), the process returns to step S203. Whether to continue the measurement of sensor data may be determined according to preset conditions such as the number of steps and time. If the measurement of sensor data is not to be continued (No in step S205), and the vibration detection mode is to be continued (Yes in step S206), the process returns to step S201. If the vibration detection mode is not to be continued (No in step S206), the process according to the flowchart in FIG. 32 ends. Whether to continue or stop the measurement may be determined according to predetermined timing, a stop operation by the user, etc.

[0148] [Sensor data measurement processing] Next, an example of the sensor data measurement process (step S203 in FIG. 32) by the measurement device 21 will be described with reference to the drawings. Fig. 33 is a flowchart for explaining an example of the sensor data measurement process by the measurement device 21. In the description of the process according to the flowchart in Fig. 33, the measurement device 21 is the subject of the operations.

[0149] 32, first, the measuring device 21 measures sensor data at a specified sampling rate (step S211). The measuring device 21 measures sensor data such as acceleration and angular velocity.

[0150] Next, the measuring device 21 records the acquired sensor data in a buffer (storage unit 233) (step S212).

[0151] Next, the measurement device 21 detects gait events from the sensor data recorded in the buffer (step S213).

[0152] If a predetermined gait event is detected (Yes in step S214) and it is the first step (Yes in step S215), the measurement device 21 detects the starting point of the gait cycle (step S216). For example, the measurement device 21 detects the timing of heel strike, toe lift, or the middle of the stance phase as the starting point of the gait cycle. If it is not the first step (No in step S215), the process proceeds to step S217.

[0153] After step S216, or if the result of step S215 is No, the measurement device 21 performs stride determination (step S217). In the stride determination, the measurement device 21 determines whether to acquire sensor data for one step (one stride).

[0154] If it is time for data communication (Yes in step S218), the process proceeds to step S204 in the flowchart of Fig. 32 (step S221 in Fig. 34). For example, the timing for data communication is set to the swing phase. If it is not time for data communication (No in step S218), the process returns to step S211.

[0155] [Gait parameter calculation process] Next, an example of the gait parameter calculation process (step S204 in FIG. 32) by the measurement device 21 will be described with reference to the drawings. FIG. 34 is a flowchart for explaining an example of the gait parameter calculation process by the measurement device 21. In describing the process according to the flowchart in FIG. 34, the measurement device 21 will be the subject of the operations.

[0156] 32, first, the measurement device 21 temporarily suspends the measurement of sensor data (step S221). In the case of a single-tasking microcomputer, the measurement of sensor data and the calculation of gait parameters cannot be performed simultaneously, so the measurement of sensor data is temporarily suspended. In the case of a multi-tasking microcomputer, the measurement of sensor data and the calculation of gait parameters can be performed simultaneously, so step S221 may be omitted.

[0157] Next, the measurement device 21 calculates gait parameters using the sensor data stored in the buffer (storage unit 233) (step S222). For example, the measurement device 21 calculates gait parameters such as walking speed, stride length, contact angle, take-off angle, maximum foot lift height (sensor position), circular movement (trajectory of traveling direction), and toe direction. If the sensor data itself is transmitted without calculating gait parameters, step S222 is omitted.

[0158] Next, the measurement device 21 transmits the calculated gait parameters (gait data) (step S223). For example, the measurement device 21 transmits gait parameters such as walking speed, stride length, contact angle, take-off angle, maximum foot lift height (sensor position), circular motion (trajectory of direction of travel), and toe direction. For example, the measurement device 21 may transmit the sensor data itself.

[0159] Next, the measurement device 21 clears a portion of the sensor data stored in the buffer (storage unit 233) (step S224). For example, the measurement device 21 deletes the sensor data used to calculate the transmitted gait parameters from the buffer (storage unit 233). After step S224, the process proceeds to step S205 in the flowchart of FIG. 32.

[0160] [Estimation device] Next, the operation of the estimation device 25 will be described with reference to the drawings. Fig. 35 is a flowchart for explaining an example of the operation of the estimation device 25. In the explanation following the flowchart of Fig. 35, the estimation device 25 will be described as the subject of the operation.

[0161] 35, first, the estimating device 25 receives gait data from the measuring device 21 (step S231). The estimating device 25 receives gait data such as sensor data and gait parameters.

[0162] Next, the estimation device 25 extracts feature amounts from the received gait data (step S232).

[0163] Next, the estimation device 25 inputs the extracted feature amounts to an estimation model (step S233). The estimation model outputs a response variable (gait event index) in response to the input feature amounts. In the case of an estimation model that outputs a response variable in response to input gait data such as sensor data or gait parameters, the gait data is input. For example, the estimation model outputs a score related to the user's gait as the response variable. For example, the estimation model outputs a numerical value related to the gait parameters as the response variable. For example, the estimation device 25 outputs a numerical value related to the degree of pronation / supination of the foot, the progression of hallux valgus, the progression of knee osteoarthritis, muscle strength, balance ability, body flexibility, etc. as the response variable. For example, the estimation model outputs a numerical value related to the center of pressure excursion index (CPEI) or the hallux valgus (HV) angle as the response variable.

[0164] Next, the estimation device 25 estimates the physical state according to the output from the estimation model (step S234). The estimation device 25 estimates the physical state according to the response variables output from the estimation model. For example, the estimation device 25 estimates a score related to the user's gait. For example, the estimation device 25 estimates the physical state according to numerical values ​​related to gait parameters. For example, the estimation device 25 estimates the physical state according to numerical values ​​related to the degree of pronation / supination of the foot, the progression of hallux valgus, the progression of knee osteoarthritis, muscle strength, balance ability, body flexibility, etc.

[0165] Next, the estimation device 25 outputs information about the estimated physical state (step S235). For example, the estimation device 25 outputs the estimation result to a display device (not shown). For example, the estimation result by the estimation device 25 is displayed on the screen of the display device. For example, the estimation result by the estimation device 25 is output to a system that uses the estimation result. There are no particular limitations on the use of the estimation result by the estimation device 25.

[0166] (Application example) Next, an application example of this embodiment will be described with reference to the drawings. In this application example of this embodiment, a measurement device is installed in a user's shoe, and gait data measured by the measurement device is transmitted to a mobile device carried by the user. The sensor data transmitted to the mobile device is processed by an estimation device implemented in the mobile device. For example, the functions of the estimation device are provided as an application that can be installed on the mobile device.

[0167] [Application Example 2-1] 36 is a conceptual diagram for explaining Application Example 2-1. In this application example, information according to the estimation result by the estimation device 25 is displayed on the screen of the mobile terminal 260 of a user wearing the shoes 200 on which the measuring device 21 is installed.

[0168] In the example of FIG. 36 , the estimation result, "Your CPEI is +8.5," is displayed on the screen of the mobile device 260 carried by the user. Furthermore, information about the physical condition corresponding to the estimation result, "You tend to pronate," is displayed on the screen of the mobile device 260. Furthermore, recommendation information, "We recommend walking with your toes slightly turned outward," is displayed on the screen of the mobile device 260 according to the physical condition. A user who views the information displayed on the screen of the mobile device 260 can take action according to the information. For example, a user who views the information displayed on the screen of the mobile device 260 can practice exercises and walking styles that are suitable for them according to the information. For example, a user who views the information displayed on the screen of the mobile device 260 can contact a medical institution or the like about their physical condition according to the information.

[0169] The information according to the estimation result by the estimation device 25 is not limited to being displayed on the screen of the mobile terminal 260, but may also be displayed on the screen of a stationary personal computer or a dedicated terminal, as long as the screen is viewable by the user. Furthermore, the information according to the estimation result by the estimation device 25 may be an image according to the estimation result instead of text information. Furthermore, the information according to the estimation result by the estimation device 25 may be notified using a preset pattern such as sound or vibration.

[0170] [Application Example 2-2] 37 and 38 are conceptual diagrams for explaining Application Example 2-2. In this application example, a user wearing shoes 200 on which a measuring device 21 is installed is authenticated according to authentication information transmitted from the user's mobile terminal 260. When performing personal authentication based on gait data, paired data including already accumulated gait data can be expanded using the method of the first embodiment. The identification number is unique to each individual and therefore immutable, but can be considered a variable.

[0171] FIG. 37 shows a situation in which a user wearing shoes 200 equipped with measuring device 21 approaches a door requiring authentication. Above the door, an opening / closing control device 270 is installed. The opening / closing control device 270 receives the identification number (also referred to as authentication information) estimated by estimating device 25 and controls the opening and closing of the door in accordance with the received identification number. The opening / closing control device 270 controls a drive device (not shown) that opens and closes the door in accordance with the received identification number. Estimating device 25 implemented in mobile terminal 260 estimates the user's identification number using features extracted from gait data measured by measuring device 21. Estimating device 25 transmits the estimated identification number. The identification number transmitted from estimating device 25 is received by opening / closing control device 270. The identification number received by opening / closing control device 270 is authenticated by an authentication system implemented in opening / closing control device 270 or a server connected to opening / closing control device 270. If the identification number is an authorized number, opening / closing control device 270 opens the door. If the identification number is not an authorized number, opening / closing control device 270 does not open the door.

[0172] 38 shows a state in which the door is opened by the opening / closing control device 270 in response to the identification number transmitted from the estimation device 25. A user wearing the shoes 200 on which the measurement device 21 is installed can enter the interior of a building or the like through the open door.

[0173] For example, when person B, who is different from the user, walks wearing shoes 200 equipped with measuring device 21, an identification number is estimated according to features extracted from the gait data of person B. If person B's identification number is an authorized number, opening / closing control device 270 opens the door. If person B's identification number is not an authorized number, opening / closing control device 270 does not open the door. For example, when shoes 200 equipped with measuring device 21 are worn by a person other than the user, the door will not open unless the identification number estimated according to the gait data is an authorized number.

[0174] For example, authentication using an identification number estimated based on gait data may be combined with other authentication methods. For example, the opening / closing control device 270 may be configured to open the door when both the identification numbers of the shoes 200 or the measuring device 21 and the identification number estimated based on the gait data are authenticated. This configuration prevents anyone other than the user from passing through the door wearing the shoes 200, thereby improving security. For example, authentication using an identification number based on gait data may be combined with authentication methods such as face authentication, fingerprint authentication, palm print authentication, and vein authentication.

[0175] Furthermore, a variable estimated based on gait data may be used as the identification number. For example, a configuration may be adopted in which a user is authenticated if the variable estimated based on gait data is within a predetermined range from the permitted identification number. With such a configuration, even if a user is permitted to enter, entry can be restricted based on variations in gait data. For example, if a user with a cold exhibits an unusual gait and the estimated variable falls outside a predetermined range from the permitted identification number, the user's entry can be restricted. For example, display information recommending a medical examination at a workplace health management center or recuperating at home based on the user's gait may be displayed near a door or on the screen of the mobile terminal 260. For example, in a location where users must walk slowly, entry can be restricted for users who attempt to run through. For example, display information recommending a slower walk or imposing a penalty for running based on the user's gait may be displayed near a door or on the screen of the mobile terminal 260.

[0176] As described above, the estimation system of this embodiment includes a measurement device and an estimation device. The measurement device is placed on a user's footwear. The measurement device measures spatial acceleration and spatial angular velocity in response to the user's walking and generates sensor data based on the measured spatial acceleration and spatial angular velocity. The measurement device generates measured gait data using the generated sensor data. The measurement device transmits the generated measured gait data to the estimation device. The estimation device includes a receiving unit, a storage unit, an estimation unit, and an estimation result output unit. The receiving unit receives measured gait data derived using sensor data related to the movement of the user's feet. The storage unit stores the estimation model generated by the learning system of the first embodiment. The estimation unit inputs the received measured gait data to the estimation model. The estimation unit estimates the user's physical state according to response variables output from the estimation model. The estimation result output unit outputs information related to the estimated physical state of the user.

[0177] The estimation system of this embodiment estimates a user's physical state using the estimation model generated by the learning system of the first embodiment. The estimation model generated by the learning system of the first embodiment is highly accurate and versatile because it is generated using an extended data set. Therefore, the estimation system of this embodiment can perform estimation with high accuracy and versatility.

[0178] In one aspect of the present embodiment, the estimation result output unit displays information about the estimated physical condition of the user on a screen of a mobile device carried by the user, allowing the user to check the estimation result estimated according to the user's gait in real time.

[0179] In one aspect of the present embodiment, the storage unit stores an estimation model that outputs a response variable related to an identification number in response to input of measured gait data. The estimation unit estimates the user's identification number in response to the response variable output from the estimation model in response to input of the measured gait data. The output unit transmits the estimated identification number to an authentication device that performs authentication using the identification number. According to this aspect, authentication according to gait can be achieved.

[0180] (Third embodiment) Next, a data generating device according to a third embodiment will be described with reference to the drawings. The data generating device according to this embodiment has a simplified configuration of the data generating device according to the first embodiment.

[0181] 39 is a block diagram showing an example of the configuration of a data generating device 31 according to this embodiment. The data generating device 31 includes an acquiring unit 311, a measurement data processing unit 312, a pseudo data generating unit 313, and an output unit 315.

[0182] The acquisition unit 311 acquires paired data that combines measured gait data related to sensor data measured in response to the movement of the user's feet with response variables corresponding to the measured gait data. The measurement data processing unit 312 generates a measured data set vector by combining a feature vector calculated using features extracted from the measured gait data with the response variables. The measurement data processing unit 312 generates a covariance matrix related to the plurality of paired data. The pseudo data generation unit 313 generates pseudo gait data using the measured gait data. The pseudo data generation unit 313 generates a pseudo data set vector by combining a pseudo feature vector calculated using pseudo features extracted from the pseudo gait data with a pseudo response variable generated using a covariance matrix related to the pseudo feature vector. The output unit 315 outputs a dataset including the measured data set vector and the pseudo data set vector.

[0183] As described above, the data generating device of this embodiment generates a pseudo dataset vector by combining a pseudo feature vector calculated using pseudo features extracted from pseudo gait data with a pseudo response variable generated using a covariance matrix related to a plurality of paired data. Therefore, the data generating device of this embodiment can generate a dataset to be used for learning even if the response variable is a continuous numerical value.

[0184] (Hardware) Here, a hardware configuration for executing control and processing according to each embodiment of the present disclosure will be described using an information processing device 90 in Fig. 40 as an example. Note that the information processing device 90 in Fig. 40 is an example configuration for executing control and processing according to each embodiment, and does not limit the scope of the present disclosure.

[0185] As shown in Fig. 40, an information processing device 90 includes a processor 91, a main storage device 92, an auxiliary storage device 93, an input / output interface 95, and a communication interface 96. In Fig. 40, interface is abbreviated as I / F (Interface). The processor 91, the main storage device 92, the auxiliary storage device 93, the input / output interface 95, and the communication interface 96 are connected to each other via a bus 98 so as to be able to communicate data with each other. The processor 91, the main storage device 92, the auxiliary storage device 93, and the input / output interface 95 are also connected to a network such as the Internet or an intranet via the communication interface 96.

[0186] The processor 91 loads a program stored in an auxiliary storage device 93 or the like into a main storage device 92. The processor 91 executes the program loaded into the main storage device 92. In this embodiment, a software program installed in the information processing device 90 may be used. The processor 91 executes control and processing according to each embodiment.

[0187] The main memory device 92 has an area in which programs are loaded. Programs stored in the auxiliary memory device 93 or the like are loaded into the main memory device 92 by the processor 91. The main memory device 92 is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory). Furthermore, a non-volatile memory such as an MRAM (Magnetoresistive Random Access Memory) may be configured / added to the main memory device 92.

[0188] The auxiliary storage device 93 stores various data such as programs. The auxiliary storage device 93 is realized by a local disk such as a hard disk or flash memory. Note that it is also possible to configure the main storage device 92 to store various data, thereby omitting the auxiliary storage device 93.

[0189] The input / output interface 95 is an interface for connecting the information processing device 90 to peripheral devices based on standards and specifications. The communication interface 96 is an interface for connecting to external systems and devices via a network such as the Internet or an intranet based on standards and specifications. The input / output interface 95 and the communication interface 96 may be a common interface for connecting to external devices.

[0190] Input devices such as a keyboard, mouse, and touch panel may be connected to the information processing device 90 as needed. These input devices are used to input information and settings. When a touch panel is used as the input device, the display screen of the display device may also serve as the interface for the input device. Data communication between the processor 91 and the input devices may be mediated by an input / output interface 95.

[0191] The information processing device 90 may also be equipped with a display device for displaying information. When a display device is equipped, the information processing device 90 preferably includes a display control device (not shown) for controlling the display of the display device. The display device may be connected to the information processing device 90 via the input / output interface 95.

[0192] The information processing device 90 may also be equipped with a drive device. The drive device acts as an intermediary between the processor 91 and a recording medium (program recording medium) for reading data and programs from the recording medium, writing the processing results of the information processing device 90 to the recording medium, etc. The drive device may be connected to the information processing device 90 via an input / output interface 95.

[0193] The above is an example of a hardware configuration for enabling control and processing according to each embodiment of the present invention. Note that the hardware configuration in FIG. 40 is an example of a hardware configuration for executing control and processing according to each embodiment and does not limit the scope of the present invention. Furthermore, a program that causes a computer to execute control and processing according to each embodiment is also within the scope of the present invention. Furthermore, a program recording medium on which a program according to each embodiment is recorded is also within the scope of the present invention. The recording medium can be realized, for example, as an optical recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). The recording medium may also be realized as a semiconductor recording medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card. The recording medium may also be realized as a magnetic recording medium such as a flexible disk or other recording medium. When a program executed by a processor is recorded on a recording medium, the recording medium corresponds to a program recording medium.

[0194] The components of each embodiment may be combined in any manner, and may be realized by software or by a circuit.

[0195] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0196] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. (Appendix 1) an acquisition unit that acquires paired data that combines measured gait data related to sensor data measured in accordance with the user's foot movements and response variables corresponding to the measured gait data; a measurement data processing unit that generates a measurement data set vector by combining a feature vector calculated using feature amounts extracted from the measured gait data with the response variables, and generates a covariance matrix for the plurality of paired data; a pseudo data generation unit that generates pseudo gait data using the measured gait data, and generates a pseudo data set vector by combining a pseudo feature vector calculated using pseudo features extracted from the pseudo gait data and a pseudo response variable generated using the covariance matrix related to the pseudo feature vector; an output unit that outputs a dataset including the measured dataset vector and the pseudo dataset vector. (Appendix 2) The pseudo data generation unit 2. The data generating device according to claim 1, wherein the data generating device generates the plurality of pseudo gait data by adding fluctuations to the plurality of measured gait data included in the plurality of paired data. (Appendix 3) The pseudo data generation unit 3. The data generating device according to claim 1, wherein the plurality of pseudo gait data is generated by adding noise to the plurality of measured gait data included in the plurality of paired data. (Appendix 4) The pseudo data generation unit generating a plurality of pseudo gait data using the measured gait data; extracting at least one pseudo-feature from the generated pseudo-gait data; 4. The data generating device according to claim 1, wherein the pseudo feature vector is generated for each of the pseudo gait data by using the pseudo feature extracted from the pseudo gait data. (Appendix 5) The measurement data processing unit extracting at least one of the feature amounts from the measured gait data derived using the sensor data for each walking cycle; generating a feature vector for each of the measured gait data using the feature extracted from the measured gait data for each walking cycle; 5. The data generating device according to claim 1, wherein the response variable associated with the measured gait data is added to the end of the feature vector generated for each of the measured gait data, thereby generating the measured data set vector for each of the measured gait data. (Appendix 6) The measurement data processing unit calculating an average vector of the feature quantity vectors calculated from the plurality of measured gait data for the plurality of paired data; generating the covariance matrix for the plurality of measurement data set vectors generated for each of the measured gait data; deriving an upper triangular matrix of the covariance matrix by Cholesky decomposition of the generated covariance matrix; calculating average values ​​of a plurality of said response variables for a plurality of said paired data; The pseudo data generation unit subtracting the average vector of the feature amount from each of the plurality of pseudo gait data to calculate a pseudo deviation vector for each of the plurality of pseudo gait data; generating a pseudo variance vector by adding a random value between 0 and 1 to the end of each of the calculated pseudo deviation vectors; multiplying each of the generated pseudo variance vectors by the last column of the upper triangular matrix to calculate the deviation of the pseudo response variable for each of the pseudo gait data; 6. The data generating device according to claim 1, wherein the pseudo response variables corresponding to the pseudo gait data are calculated by adding the deviation of the pseudo response variables calculated for each of the pseudo gait data to an average value of the response variables. (Appendix 7) The output unit 7. The data generation device according to claim 1, wherein information relating to the generated data set is displayed on a screen of a terminal device. (Appendix 8) A data generation device according to any one of Supplementary Notes 1 to 7; a learning device that acquires a data set output from the data generating device, and uses the acquired data set to generate an estimation model that outputs response variables according to a user's physical state in response to input of measured gait data. (Appendix 9) a storage unit that stores an estimation model generated by the learning system according to Supplementary Note 8; a receiving unit that receives measured gait data derived using sensor data related to the movement of the user's feet; an estimation unit that inputs the received measured gait data into the estimation model and estimates a physical state of the user in accordance with response variables output from the estimation model; an estimation result output unit that outputs information related to the estimated physical state of the user. (Appendix 10) The estimation result output unit 10. The estimation system according to claim 9, wherein information about the estimated physical condition of the user is displayed on a screen of a mobile terminal carried by the user. (Appendix 11) The storage unit storing the estimation model that outputs the response variable related to the identification number in response to the input of the measured gait data; The estimation unit estimating the identification number of the user according to the response variables output from the estimation model in response to the input of the measured gait data; The output unit 11. The estimation system according to claim 9, wherein the estimated identification number is transmitted to an authentication device that performs authentication using the identification number. (Appendix 12) 12. The estimation system according to any one of appendices 9 to 11, comprising a measurement device that is placed on footwear of the user, measures spatial acceleration and spatial angular velocity in response to walking of the user, generates the sensor data based on the measured spatial acceleration and spatial angular velocity, generates the measured gait data using the generated sensor data, and transmits the generated measured gait data to the receiving unit. (Appendix 13) The computer acquiring paired data that combines measured gait data related to sensor data measured in accordance with the user's foot movements with response variables corresponding to the measured gait data; generating a measurement data set vector by combining a feature vector calculated using the feature amounts extracted from the measured gait data with the response variables; generating a covariance matrix for the plurality of paired data; generating pseudo gait data using the measured gait data; generating a pseudo dataset vector by combining a pseudo feature vector calculated using pseudo features extracted from the pseudo gait data and a pseudo response variable generated using the covariance matrix related to the pseudo feature vector; A data generation method that outputs a data set including the measured data set vector and the pseudo data set vector. (Appendix 14) a process of acquiring paired data in which measured gait data relating to sensor data measured in accordance with the user's foot movements and response variables corresponding to the measured gait data are combined; a process of generating a measurement data set vector by combining a feature vector calculated using the feature amounts extracted from the measured gait data with the response variables; A process of generating a covariance matrix for the plurality of paired data; a process of generating pseudo gait data using the measured gait data; a process of generating a pseudo dataset vector by combining a pseudo feature vector calculated using pseudo features extracted from the pseudo gait data and a pseudo response variable generated using the covariance matrix related to the pseudo feature vector; and outputting a data set including the measured data set vectors and the pseudo data set vectors. [Explanation of symbols]

[0197] 10 Learning System 11, 31 Data generation device 15 Learning Device 20 Estimation System 21 Measuring equipment 22 Sensors 23 Measurement section 25 Estimation device 110 Measuring Equipment 111, 311 Acquisition Department 112, 312 Measurement data processing section 113, 313 Pseudo data generation unit 115, 315 output section 151 Dataset Acquisition Unit 153 Learning Department 155 Storage section 221 Acceleration Sensor 222 Angular rate sensor 231 Acquisition Department 233 Storage section 235 Calculation Department 237 Transmitter 250 Estimated Model 251 Receiving unit 253 Storage section 255 Estimation Department 257 Estimation result output section

Claims

1. an acquisition means for acquiring paired data that combines measured gait data related to sensor data measured in accordance with the movement of the user's feet and a response variable in accordance with the physical state of the user that corresponds to the measured gait data; a measurement data processing means for generating a measurement data set vector by combining a feature vector calculated using the feature amounts extracted from the measured gait data with the response variables, and for generating a covariance matrix for the plurality of paired data; a pseudo data generating means for generating pseudo gait data using the paired data including the measured gait data, generating pseudo feature vectors using pseudo features extracted from the pseudo gait data, generating pseudo response variables using the covariance matrix related to the paired data from which the pseudo feature vectors are generated, and combining the pseudo gait data generated using the paired data and the pseudo response variables to generate a pseudo data set vector; and an output means for outputting a data set including the measured data set vector and the pseudo data set vector.

2. The pseudo data generating means 2. The data generating device according to claim 1, wherein the plurality of pseudo gait data are generated by adding fluctuations to the plurality of measured gait data included in the plurality of paired data.

3. The pseudo data generating means 3. The data generating device according to claim 1, wherein the plurality of pseudo gait data are generated by adding noise to the plurality of measured gait data included in the plurality of paired data.

4. The pseudo data generating means generating a plurality of pseudo gait data using the measured gait data; extracting at least one pseudo-feature from the generated pseudo-gait data; The data generating device according to claim 1 , wherein the pseudo feature vector is generated for each of the pseudo gait data by using the pseudo feature extracted from the pseudo gait data.

5. The measurement data processing means extracting at least one of the feature amounts from the measured gait data derived using the sensor data for each walking cycle; generating a feature vector for each of the measured gait data using the feature extracted from the measured gait data for each walking cycle; 5. The data generating device according to claim 1, wherein the response variable associated with the measured gait data is added to the end of the feature amount vector generated for each of the measured gait data, thereby generating the measured data set vector for each of the measured gait data.

6. The measurement data processing means calculating an average vector of the feature quantity vectors calculated from the plurality of measured gait data for the plurality of paired data; generating the covariance matrix for the plurality of measurement data set vectors generated for each of the measured gait data; deriving an upper triangular matrix of the covariance matrix by Cholesky decomposition of the generated covariance matrix; calculating average values ​​of a plurality of said response variables for a plurality of said paired data; The pseudo data generating means subtracting the average vector of the feature amount from each of the plurality of pseudo gait data to calculate a pseudo deviation vector for each of the plurality of pseudo gait data; generating a pseudo variance vector by adding a random value between 0 and 1 to the end of each of the calculated pseudo deviation vectors; multiplying each of the generated pseudo variance vectors by the last column of the upper triangular matrix to calculate the deviation of the pseudo response variable for each of the pseudo gait data; 6. The data generating device according to claim 1, wherein the pseudo response variables corresponding to the pseudo gait data are calculated by adding the deviation of the pseudo response variables calculated for each of the pseudo gait data to an average value of the response variables.

7. The output means 7. The data generating device according to claim 1, wherein information relating to the generated data set is displayed on a screen of a terminal device.

8. A data generating device according to any one of claims 1 to 7; a learning device that acquires a data set output from the data generating device, and uses the acquired data set to generate an estimation model that outputs response variables according to a user's physical state in response to input of measured gait data.

9. The computer acquiring paired data that combines measured gait data related to sensor data measured in accordance with foot movements of the user and response variables in accordance with the physical state of the user corresponding to the measured gait data; generating a measurement data set vector by combining a feature vector calculated using the feature amounts extracted from the measured gait data with the response variables; generating a covariance matrix for the plurality of paired data; generating pseudo gait data using the paired data including the measured gait data; generating a pseudo feature vector using the pseudo feature extracted from the pseudo gait data; generating a pseudo response variable using the covariance matrix of the paired data from which the pseudo feature vector is generated; generating a pseudo data set vector by combining the pseudo gait data and the pseudo response variables generated using the paired data; A data generation method that outputs a data set including the measured data set vector and the pseudo data set vector.

10. a process of acquiring paired data in which measured gait data relating to sensor data measured in accordance with foot movements of the user is combined with a response variable in accordance with a physical state of the user corresponding to the measured gait data; a process of generating a measurement data set vector by combining a feature vector calculated using the feature amounts extracted from the measured gait data with the response variables; A process of generating a covariance matrix for the plurality of paired data; a process of generating pseudo gait data using the paired data including the measured gait data; generating a pseudo feature vector using pseudo features extracted from the pseudo gait data; generating a pseudo response variable using the covariance matrix of the paired data from which the pseudo feature vector is generated; a process of combining the pseudo gait data and the pseudo response variables generated using the paired data to generate a pseudo data set vector; and outputting a data set including the measured data set vectors and the pseudo data set vectors.

Images