Learning device, estimation system, learning method, and program

Abstract

The learning device according to the present invention trains, to eliminate code redundancy derived from sensor data measured by a plurality of measuring instruments and to efficiently reduce the dimensions of the sensor data, a first encoding model for encoding first sensor data into first code, a second encoding model for encoding second sensor data into second code, and an estimation model for making estimation using the first code and the second code such that an estimation result from the estimation model conforms to correct answer data; trains a first adversarial estimation model that outputs an estimated value of the second code in response to the input of the first code such that the estimated value of the second code estimated by the first adversarial estimation model conforms to the second code outputted from the second encoding model; and trains the first encoding model such that the estimated value of the second code estimated by the first adversarial estimation model does not conform to the second code outputted from the second encoding model.

Description

[Technical field]

[0001] The present disclosure relates to a learning device and the like that performs learning using measurement data obtained by a sensor. [Background technology]

[0002] With the spread of IoT (Internet of Things) technology, various information about people and things can be collected from various IoT devices. Attempts are being made to utilize information collected by IoT devices in fields such as medicine, healthcare, and security. For example, by applying machine learning to information collected by IoT devices, the information can be used for purposes such as estimating health conditions and personal authentication. IoT devices require high levels of power saving. The proportion of power consumed for communication in the total power consumption of IoT devices is relatively large. Therefore, IoT devices have strong constraints on communication. Therefore, it is difficult for IoT devices to transmit large amounts of data frequently.

[0003] Patent Document 1 discloses a data analysis system that analyzes observation data observed by an instrument such as an IoT device. In the system of Patent Document 1, the instrument inputs observation data to an input layer of a trained neural network and processes the data up to a predetermined intermediate layer. The trained neural network is configured so that the number of nodes in the predetermined intermediate layer is smaller than the number of nodes in the output layer. The trained neural network is also trained in advance so that the overlap of probability distributions of low-dimensional observation data for observation data with different analysis results under a predetermined constraint is reduced compared to a case without the predetermined constraint. The instrument transmits the results of processing up to the predetermined intermediate layer to the device as low-dimensional observation data. The device inputs the received low-dimensional observation data to the next intermediate layer after the predetermined intermediate layer and processes the data, thereby analyzing the observation data observed by the instrument. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 203232 Summary of the Invention [Problem to be solved by the invention]

[0005] In the method of Patent Document 1, low-dimensional observation data that has been processed up to a predetermined intermediate layer is transmitted from an instrument to an apparatus. Therefore, according to the method of Patent Document 1, the amount of data transmitted from an instrument to an apparatus can be reduced. For example, in the method of Patent Document 1, by connecting multiple instruments to an apparatus, the observation data observed by multiple instruments can be analyzed. However, when the method of Patent Document 1 is extended to multiple instruments, the neural network of each instrument is trained independently, and the low-dimensional observation data of each instrument may contain redundant information. When the low-dimensional observation data contains redundant information and data is duplicated, communication efficiency decreases in a situation where the communication volume is limited.

[0006] An object of the present disclosure is to provide a learning device and the like that can eliminate code redundancy derived from sensor data measured by multiple measuring instruments and efficiently reduce the dimension of the sensor data. [Means for solving the problem]

[0007] A learning device according to one embodiment of the present disclosure includes an acquisition unit that acquires a training dataset including first sensor data measured by a first measurement device, second sensor data measured by a second measurement device, and ground truth data; an encoding unit that encodes the first sensor data into a first code using a first encoding model and encodes the second sensor data into a second code using a second encoding model; an estimation unit that inputs the first code and the second code into an estimation model and outputs an estimation result output from the estimation model; an adversarial estimation unit that inputs the first code into a first adversarial estimation model that outputs an estimated value of the second code in response to the input of the first code, and estimates an estimated value of the second code; and a learning processing unit that trains the first encoding model, the second encoding model, the estimation model, and the first adversarial estimation model through machine learning. The learning processing unit trains the first encoding model, the second encoding model, and the estimation model so that the estimation result of the estimation model matches the correct data, trains the first adversarial estimation model so that the estimated value of the second code by the first adversarial estimation model matches the second code output from the second encoding model, and trains the first encoding model so that the estimated value of the second code by the first adversarial estimation model does not match the second code output from the second encoding model.

[0008] In a learning method according to one embodiment of the present disclosure, a training dataset including first sensor data measured by a first measurement device, second sensor data measured by a second measurement device, and ground truth data is acquired, the first sensor data is encoded into a first code using a first encoding model, the second sensor data is encoded into a second code using a second encoding model, the first code and the second code are input into an estimation model to output an estimation result output from the estimation model, the first code is input into a first adversarial estimation model that outputs an estimate of the second code in response to the input of the first code, and an estimate of the second code is estimated, the first encoding model, the second encoding model, and the estimation model are trained such that the estimate of the second code by the first adversarial estimation model matches the ground truth data, the first adversarial estimation model is trained such that the estimate of the second code by the first adversarial estimation model matches the second code output from the second encoding model, and the first encoding model is trained such that the estimate of the second code by the first adversarial estimation model does not match the second code output from the second encoding model.

[0009] A program according to an embodiment of the present disclosure includes a process of acquiring a training dataset including first sensor data measured by a first measurement device, second sensor data measured by a second measurement device, and ground truth data; a process of encoding the first sensor data into a first code using a first encoding model, and encoding the second sensor data into a second code using a second encoding model; a process of inputting the first code and the second code into an estimation model and outputting an estimation result output from the estimation model; and a process of inputting an estimation result into a first adversarial estimation model that outputs an estimation value of the second code in response to an input of the first code. The method causes a computer to execute a process of inputting a first code and estimating an estimated value of a second code; a process of training the first encoding model, the second encoding model, and the estimation model so that the estimation result of the estimation model matches the ground truth data; a process of training the first adversarial estimation model so that the estimated value of the second code by the first adversarial estimation model matches the second code output from the second encoding model; and a process of training the first encoding model so that the estimated value of the second code by the first adversarial estimation model does not match the second code output from the second encoding model. Effect of the Invention

[0010] According to the present disclosure, it is possible to provide a learning device etc. that can eliminate code redundancy derived from sensor data measured by multiple measuring instruments and efficiently reduce the dimension of the sensor data. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of a configuration of a learning device according to a first embodiment. [Diagram 2] 2 is a conceptual diagram for explaining an example of a model group included in the learning device according to the first embodiment. FIG. [Diagram 3] 1 is a conceptual diagram for explaining accumulation of training data that is a learning target of a learning device according to a first embodiment. FIG. [Figure 4] FIG. 2 is a conceptual diagram for explaining an example of training a group of models by the learning device according to the first embodiment. [Diagram 5] 4 is a flowchart for explaining an example of an operation of the learning device according to the first embodiment. [Figure 6] 4 is a flowchart for explaining an example of an estimation process performed by the learning device according to the first embodiment. [Figure 7] 4 is a flowchart for explaining an example of a training process by the learning device according to the first embodiment. [Figure 8] FIG. 11 is a block diagram showing an example of the configuration of a learning device according to a second embodiment. [Figure 9] FIG. 11 is a conceptual diagram for explaining an example of a model group included in a learning device according to a second embodiment. [Figure 10] FIG. 11 is a conceptual diagram for explaining an example of training a group of models by a learning device according to a second embodiment. [Figure 11] 10 is a flowchart illustrating an example of an operation of a learning device according to a second embodiment. [Figure 12]13 is a flowchart illustrating an example of an estimation process performed by a learning device according to a second embodiment. [Figure 13] 13 is a flowchart for explaining an example of a training process by a learning device according to a second embodiment. [Figure 14] FIG. 13 is a block diagram showing an example of the configuration of a learning device according to a third embodiment. [Figure 15] FIG. 13 is a block diagram showing an example of a configuration of an estimation system according to a fourth embodiment. [Figure 16] FIG. 13 is a conceptual diagram showing an example of the arrangement of a first measuring device in an estimation system according to a fourth embodiment. [Figure 17] FIG. 13 is a block diagram showing an example of the configuration of a first measuring device included in an estimation system according to a fourth embodiment. [Figure 18] FIG. 13 is a conceptual diagram showing an example of the arrangement of a second measuring device included in an estimation system according to a fourth embodiment. [Figure 19] FIG. 13 is a block diagram showing an example of the configuration of a second measuring device included in an estimation system according to a fourth embodiment. [Figure 20] FIG. 13 is a block diagram showing an example of a configuration of an estimation device included in an estimation system according to a fourth embodiment. [Figure 21] FIG. 13 is a conceptual diagram for explaining estimation of a physical state by an estimation system according to a fourth embodiment. [Figure 22] FIG. 13 is a conceptual diagram showing an example in which a result of a physical condition estimation performed by an estimation system according to a fourth embodiment is displayed on a screen of a mobile terminal. [Figure 23] 13 is a flowchart for explaining an example of an operation of a first measuring apparatus included in the estimation system according to the fourth embodiment. [Figure 24] 13 is a flowchart for explaining an example of an operation of a second measuring device included in the estimation system according to the fourth embodiment. [Diagram 25] 13 is a flowchart for explaining an example of an operation of an estimation device included in an estimation system according to a fourth embodiment. [Figure 26]FIG. 2 is a block diagram showing an example of a hardware configuration for implementing the learning device or the estimation device according to each embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments for carrying out the present invention will be described with reference to the drawings. However, the embodiments described below are limited in a manner that is technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following. In addition, in all the drawings used to explain the following embodiments, the same reference numerals are used for similar parts unless otherwise specified. In addition, in the following embodiments, repeated explanations of similar configurations and operations may be omitted.

[0013] (First embodiment) First, a learning device according to a first embodiment will be described with reference to the drawings. The learning device of this embodiment learns data collected by an IoT (Internet of Things) device (also called a measurement device). The measurement device includes at least one sensor. For example, the measurement device is an inertial measurement unit including an acceleration sensor, an angular velocity sensor, etc. For example, the measurement device is an activity meter including an acceleration sensor, an angular velocity sensor, a pulse sensor, a temperature sensor, etc. In this embodiment, a wearable device attached to the body will be described as an example.

[0014] The learning device of this embodiment learns sensor data (raw data) related to physical activity measured by multiple measuring devices. For example, the learning device of this embodiment learns sensor data related to physical quantities related to leg movements and physical quantities / biological data corresponding to physical activity. The learning device of this embodiment builds a model that estimates a physical state (estimated result) according to input of sensor data by learning using the sensor data. The method of this embodiment can be applied to time-series data of sensor data, image analysis, etc.

[0015] (composition) 1 is a block diagram showing an example of the configuration of a learning device 10 according to this embodiment. The learning device 10 includes an acquisition unit 11, an encoding unit 12, an estimation unit 13, an adversarial estimation unit 14, and a learning processing unit 15. The encoding unit 12 includes a first encoding unit 121 and a second encoding unit 122.

[0016] FIG. 2 is a block diagram for explaining a model constructed by the learning device 10. In FIG. 2, the acquisition unit 11 and the learning processing unit 15 are omitted. The first encoding unit 121 includes a first encoding model 151. The second encoding unit 122 includes a second encoding model 152. The estimation unit 13 includes an estimation model 153. The adversarial estimation unit 14 includes an adversarial estimation model 154 (also called a first adversarial estimation model). The first encoding model 151, the second encoding model 152, the estimation model 153, and the adversarial estimation model 154 are also collectively called a model group. Details of the first encoding model 151, the second encoding model 152, the estimation model 153, and the adversarial estimation model 154 will be described later.

[0017] The acquiring unit 11 acquires a plurality of data sets (also called training data sets) used to construct a model. For example, the acquiring unit 11 acquires the training data sets from a database (not shown) in which the training data sets are accumulated. The training data sets include a data set combining first sensor data (first raw data), second sensor data (second raw data), and correct answer data. The first raw data and the second raw data are sensor data measured by different measuring devices. For example, the correct answer data is a physical condition corresponding to the first raw data and the second raw data. The acquiring unit 11 acquires a training data set including the first raw data, the second raw data, and the correct answer data that correspond to each other from a plurality of training data sets included in the training data set.

[0018] FIG. 3 is a conceptual diagram for explaining an example of collection of a training dataset. FIG. 3 shows an example in which a walking person (also called a subject) wears multiple wearable devices (measuring devices). The physical state (ground truth data) of the estimation target is assumed to have been verified in advance. For example, the training dataset is acquired from multiple subjects. A model constructed using the training dataset acquired from multiple subjects has versatility. For example, the training dataset is acquired from a specific subject. A model constructed using the training dataset acquired from a specific subject enables highly accurate estimation for a specific subject even if it is not versatile.

[0019] The subject in FIG. 3 is wearing footwear 100 in which a first measuring device 111 is installed. That is, the first measuring device 111 is attached to the foot of the subject in FIG. 3. For example, the first measuring device 111 includes a sensor that measures acceleration and angular velocity. The first measuring device 111 generates first sensor data (first raw data) corresponding to the acceleration and angular velocity measured in response to the walking of the subject. The first measuring device 111 transmits the generated first raw data. The first raw data transmitted from the first measuring device 111 is received by a mobile terminal 160 carried by the subject.

[0020] 3, a second measuring device 112 is attached to the wrist of the subject. For example, the second measuring device 112 includes sensors that measure acceleration, angular velocity, pulse, and temperature. The second measuring device 112 generates second sensor data (second raw data) corresponding to the acceleration, angular velocity, pulse, and body temperature measured in accordance with the activity of the walker. The second measuring device 112 transmits the generated second raw data. The second raw data transmitted from the second measuring device 112 is received by a mobile terminal 160 carried by the subject.

[0021] For example, the first measuring device 111 and the second measuring device 112 transmit the first raw data and the second raw data to the mobile terminal 160 via wireless communication. For example, the first measuring device 111 and the second measuring device 112 transmit the raw data to the mobile terminal 160 via a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the first measuring device 111 and the second measuring device 112 may conform to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark). For example, the first measuring device 111 and the second measuring device 112 may transmit the raw data to the mobile terminal 160 via a wired connection such as a cable.

[0022] A data collection application (not shown) installed on the mobile terminal 160 generates a training data set by associating the first raw data and the second raw data with the subject's physical condition (correct answer data). For example, the data collection application generates a training data set by associating the first raw data and the second raw data measured at the same timing with the subject's physical condition (correct answer data). The mobile terminal 160 transmits the training data set to a database 17 built in a cloud or a server via a network 190 such as the Internet. There is no particular limitation on the communication method of the mobile terminal 160. The transmitted training data set is stored in the database 17. For example, the mobile terminal 160 may be configured to transmit the first raw data, the second raw data, and the subject's physical condition to an estimation device implemented in a cloud or a server. In that case, the training data set may be generated by a data collection application built in the cloud or the server. The learning device 10 acquires the training data set stored in the database 17.

[0023] The first raw data and the second raw data may be pre-processed in some way. For example, the first raw data and the second raw data may be pre-processed by noise removal using a low-pass filter or a high-pass filter. For example, the first raw data and the second raw data may be pre-processed by outlier removal or missing value interpolation. For example, the first raw data and the second raw data may be pre-processed by frequency conversion, integration, differentiation, etc. For example, the first raw data and the second raw data may be pre-processed by statistical processing such as averaging or variance calculation. For example, when the first raw data and the second raw data are time-series data, the pre-processing may be performed by cutting out a predetermined section. For example, when the first raw data and the second raw data are image data, the pre-processing may be performed by cutting out a predetermined region. The pre-processing performed on the first raw data and the second raw data is not limited to those listed here.

[0024] For example, the training data set is information combining sensor data (first raw data) related to foot movement, sensor data (second raw data) according to physical activity, and the subject's physical condition (correct answer data). For example, the first raw data includes sensor data such as acceleration and angular velocity. For example, the first raw data may include a velocity, position (trajectory), angle, etc. obtained by integrating the acceleration and angular velocity. For example, the second raw data includes sensor data such as acceleration, angular velocity, pulse, and body temperature. For example, the second raw data may include data calculated using the acceleration, angular velocity, pulse, and body temperature. For example, the body condition (correct answer data) includes the subject's physical condition such as the degree of pronation / supination of the foot, the progress of hallux valgus, and the risk of falling. For example, the body condition may include a score according to the subject's physical condition. The training data set acquired by the acquisition unit 11 is not particularly limited as long as it is information combining explanatory variables (first raw data, second raw data) and objective variables (correct answer data).

[0025] The encoding unit 12 acquires the first raw data and the second raw data from the acquiring unit 11. The encoding unit 12 encodes the first raw data in the first encoding unit 121. The first raw data encoded in the first encoding unit 121 is a first code. The encoding unit 12 also encodes the second raw data in the second encoding unit 122. The second raw data encoded in the second encoding unit 122 is a second code.

[0026] The first encoding unit 121 acquires first raw data. The first encoding unit 121 inputs the acquired first raw data to the first encoding model 151. The first encoding model 151 outputs a first code in response to the input of the first raw data. The first code includes features of the first raw data. That is, the first encoding unit 121 encodes the first raw data to generate a first code including features of the first raw data. For example, the first encoding unit 121 encodes a feature amount extracted from the first raw data to generate a first code including features used to estimate a physical state. For example, the first encoding unit 121 encodes a feature amount extracted from the first raw data to generate a first code. The first code includes features used to estimate a score according to the physical state of the subject.

[0027] The second encoding unit 122 acquires second raw data. The second encoding unit 122 inputs the acquired second raw data to the second encoding model 152. The second encoding model 152 outputs a second code in response to the input of the second raw data. The second code includes the features of the second raw data. That is, the second encoding unit 122 encodes the second raw data to generate a second code including the features of the second raw data. For example, the second encoding unit 122 encodes the feature amount extracted from the second raw data to generate a second code including the features used to estimate the physical condition. For example, the second encoding unit 122 encodes the feature amount extracted from the second raw data to generate the second code. The second code includes the features used to estimate a score according to the physical condition of the subject.

[0028] The first raw data and the second raw data may include overlapping information. For example, both the first measuring device 111 and the second measuring device 112 measure acceleration and angular velocity. Therefore, the first raw data and the second raw data have overlapping information on acceleration and angular velocity. For example, when the subject walks fast, the walking speed calculated based on the first raw data increases. Also, when the subject walks fast, the heart rate increases, and the magnitude and fluctuation of the pulse included in the second raw data increases. Therefore, the first raw data and the second raw data have overlapping information on the increase in heart rate due to the increase in walking speed. For example, there is a physical condition that can be estimated using the first raw data measured by the first measuring device 111 installed in the footwear 100 of one foot. Regarding such a physical condition, information on the first raw data transmitted from the first measuring device 111 installed in the footwear 100 of the left foot and the first measuring device 111 installed in the footwear 100 of the right foot overlap. In this embodiment, a model is constructed for eliminating duplicated information that may be contained in the first raw data and the second raw data at the encoding stage.

[0029] For example, the first encoding model 151 and the second encoding model 152 output 10 Hz time series data (code) in response to input of time series data (raw data) measured at a cycle of 100 Hertz (Hz). For example, the first encoding model 151 and the second encoding model 152 output time series data (code) in which the amount of data has been reduced by averaging or noise removal in response to input of time series data corresponding to raw data. For example, the first encoding model 151 outputs 7×7 pixel image data (code) in response to input of 28×28 pixel image data (raw data). The code only needs to have a smaller amount of data than the raw data and include a feature that enables estimation of correct answer data corresponding to the raw data. There is no limitation on the data capacity or data format of the code.

[0030] The estimation unit 13 acquires the first code and the second code from the encoding unit 12. The estimation unit 13 inputs the acquired first code and second code to the estimation model 153. The estimation model 153 outputs an estimation result regarding the subject's physical condition in response to the input of the first code and the second code. That is, the estimation unit 13 estimates the subject's physical condition using the first code and the second code. The estimation unit 13 outputs the estimation result regarding the subject's physical condition. The estimation result by the estimation unit 13 is compared with the correct answer data of the subject's physical condition by the learning processing unit 15.

[0031] The adversarial estimation unit 14 acquires a first code from the encoding unit 12. The adversarial estimation unit 14 inputs the acquired first code to the adversarial estimation model 154. The adversarial estimation model 154 outputs a second code in response to the input of the first code. That is, the adversarial estimation unit 14 estimates the second code using the first code. The estimate of the second code by the adversarial estimation unit 14 may include a common point with the first code. The estimate of the second code by the adversarial estimation unit 14 is compared by the learning processing unit 15 with the second code encoded by the second encoding unit 122.

[0032] For example, the first coding model 151, the second coding model 152, the estimation model 153, and the adversarial estimation model 154 include a structure of a DNN (Deep Neural Network). For example, the first coding model 151, the second coding model 152, the estimation model 153, and the adversarial estimation model 154 include a structure of a CNN (Convolutional Neural Network). For example, the first coding model 151, the second coding model 152, the estimation model 153, and the adversarial estimation model 154 include a structure of an RNN (Recurrent Neural Network). The structures of the first coding model 151, the second coding model 152, the estimation model 153, and the adversarial estimation model 154 are not limited to DNN, CNN, or RNN. The first coding model 151, the second coding model 152, the estimation model 153, and the adversarial estimation model 154 are trained by machine learning by the learning processing unit 15.

[0033] The learning processing unit 15 trains a group of models, including the first encoding model 151, the second encoding model 152, the estimation model 153, and the adversarial estimation model 154, by machine learning. Fig. 4 is a conceptual diagram for explaining the training of the first encoding model 151, the second encoding model 152, the estimation model 153, and the adversarial estimation model 154 by the learning processing unit 15. In Fig. 4, the acquisition unit 11 and the learning processing unit 15 are omitted.

[0034] The learning processing unit 15 trains the first encoding model 151, the second encoding model 152, and the estimation model 153 so that the estimation result of the estimation model 153 matches the correct answer data. That is, the learning processing unit 15 optimizes the model parameters of the first encoding model 151, the second encoding model 152, and the estimation model 153 so that the error between the estimation result of the estimation model 153 and the correct answer data is reduced. For example, the learning processing unit 15 optimizes the model parameters of the first encoding model 151, the second encoding model 152, and the estimation model 153 so that the error between the estimation result of the estimation model 153 and the correct answer data is minimized. This training improves the accuracy rate of the estimation result output from the estimation model 153.

[0035] Furthermore, the learning processing unit 15 trains the adversarial estimation model 154 so that the estimated value of the second code by the adversarial estimation model 154 matches the second code. That is, the learning processing unit 15 optimizes the model parameters of the adversarial estimation model 154 so that the error between the estimated value of the second code by the adversarial estimation model 154 and the output value of the second code by the second encoding model 152 is reduced. For example, the learning processing unit 15 optimizes the model parameters of the adversarial estimation model 154 so that the error between the estimated value of the second code by the adversarial estimation model 154 and the output value of the second code by the second encoding model 152 is minimized. This training improves the accuracy rate of the estimated value of the second code output from the adversarial estimation model 154.

[0036] Furthermore, the learning processing unit 15 trains the first encoding model 151 so that the estimated value of the second code by the adversarial estimation model 154 does not match the second code. That is, the learning processing unit 15 optimizes the model parameters of the first encoding model so that the error between the estimated value of the second code by the adversarial estimation model 154 and the output value of the second code by the second encoding model 152 becomes large. For example, the learning processing unit 15 optimizes the model parameters of the first encoding model so that the error between the estimated value of the second code by the adversarial estimation model 154 and the output value of the second code by the second encoding model 152 becomes maximum. By this training, features that overlap with the second code are eliminated from the first code output from the first encoding model 151.

[0037] In this embodiment, the adversarial estimation model 154 is trained to improve the accuracy rate of the estimated value of the second code, and the first encoding model 151 is trained to reduce overlap between the first code and the second code. That is, in this embodiment, the first encoding model 151 and the adversarial estimation model 154 are trained adversarially. As a result, common features that may be included in the first code output from the first encoding model 151 and the second code output from the second encoding model 152 are eliminated.

[0038] In this embodiment, an example is shown in which the first encoding model 151 and the adversarial estimation model 154 are adversarially trained using a first code output from the first encoding model 151. This embodiment may be configured to adversarially train the second encoding model 152 and the adversarial estimation model 154 using a second code output from the second encoding model 152. In addition, the method of this embodiment may be used to remove duplication that may be included in sensor data measured by three or more measuring devices.

[0039] For example, the learning processing unit 15 trains the first encoding model 151, the second encoding model 152, and the estimation model 153 so as to minimize a square sum error or a cross entropy error between the output of the estimation model 153 and the ground truth data. For example, the learning processing unit 15 trains the first encoding model 151, the second encoding model 152, and the estimation model 153 so as to minimize a loss function of the following formula 1.

number

[0040] For example, the learning processing unit 15 trains the adversarial estimation model 154 so as to minimize errors such as a square sum error and a cross entropy error between the output (second code) of the second encoding model 152 and an estimate of the second code by the adversarial estimation model 154. For example, the learning processing unit 15 trains the adversarial estimation model 154 so as to minimize a loss function of the following formula 2.

number

[0041] Among the group of models trained by the learning processing unit 15, the first encoding model 151, the second encoding model 152, and the estimation model 153 are implemented in an estimation system (not shown) that performs estimation based on raw data. For example, the estimation system includes a first measurement device that measures first measurement data (first raw data), a second measurement device that measures second measurement data (second raw data), and an estimation device (not shown) that performs estimation using the measurement data. The first encoding model 151 is implemented in the first measurement device. The second encoding model 152 is implemented in the second measurement device. The estimation model 153 is implemented in the estimation device. The first measurement device encodes the first measurement data into a first code using the first encoding model. The first measurement device transmits the encoded first code to the estimation device. The second measurement device encodes the second measurement data into a second code using the first encoding model. The second measurement device transmits the encoded second code to the estimation device. The estimation device inputs the first code received from the first measuring device and the second code received from the second measuring device to an estimation model. The estimation device outputs an estimation result output from the estimation model in response to the input of the first code and the second code. Details of the estimation system using the model trained by the learning processing unit 15 will be described later.

[0042] (operation) Next, the operation of the learning device 10 of this embodiment will be described with reference to the drawings. Figures 5 to 7 are flowcharts for explaining an example of the operation of the learning device 10. In the explanation following the flowchart of Figure 5, the learning device 10 will be described as the subject of the operation.

[0043] In FIG. 5, first, the learning device 10 acquires the first raw data, the second raw data, and the correct answer data from the training data set (step S11).

[0044] Next, the learning device 10 executes an estimation process using a group of models including the first encoding model 151, the second encoding model 152, the estimation model 153, and the adversarial estimation model 154 (step S12). In the estimation process, the first encoding model 151 performs encoding into a first code, the second encoding model 152 performs encoding into a second code, and the estimation model 153 estimates an estimation result. In addition, in the estimation process, the adversarial estimation model 154 estimates the second code. Details of the estimation process in step S12 will be described later.

[0045] Next, the learning device 10 executes training processing of the first encoding model 151, the second encoding model 152, the estimation model 153, and the adversarial estimation model 154 according to the estimation results of the model group (step S13). The model parameters of the model group trained by the learning processing unit 15 are set in the first encoding model 151, the second encoding model 152, and the estimation model 153 implemented in an estimation system (not shown). Details of the training processing in step S13 will be described later.

[0046] If learning is to be continued (Yes in step S14), the process returns to step S11. On the other hand, if learning is to be stopped (No in step S14), the process according to the flowchart in FIG. 5 ends. The continuation / end of learning may be determined based on a preset criterion. For example, the learning device 10 determines whether to continue / end learning depending on the accuracy rate of the estimation result by the estimation model 153. For example, the learning device 10 determines whether to continue / end learning depending on the error between the estimated value of the second code by the adversarial estimation unit 14 and the second code output from the second encoding model 152.

[0047] [Estimation process] Next, the estimation process (step S12 in FIG. 5) by the learning processing unit 15 will be described with reference to the drawings. FIG. 6 is a flowchart for explaining the estimation process by the learning processing unit 15. In the process according to the flowchart in FIG. 6, the learning device 10 will be described as the subject of operations.

[0048] 6, first, the learning device 10 inputs first raw data to the first encoding model 151 and calculates a first code (step S121). In response to the input of the first raw data, the code output from the first encoding model 151 is the first code.

[0049] Next, the learning device 10 inputs the second raw data to the second encoding model 152 to calculate the second code (step S122). In response to the input of the second raw data, the code output from the second encoding model 152 is the second code. Steps S121 and S122 may be interchanged in order, or may be performed in parallel.

[0050] Next, the learning device 10 inputs the first code and the second code to the estimation model 153 and calculates an estimation result (step S123). The result output from the estimation model 153 in response to the input of the first raw data and the second raw data is the estimation result.

[0051] Next, the learning device 10 inputs the first code to the adversarial estimation model 154 and calculates an estimate of the second code (step S124). In response to the input of the first code, the code output from the adversarial estimation model 154 is the estimate of the second code. Steps S123 and S124 may be interchanged in order or may be performed in parallel.

[0052] [Training process] Next, the training process (step S13 in FIG. 5) by the learning processing unit 15 will be described with reference to the drawings. FIG. 6 is a flowchart for explaining the training process by the learning processing unit 15. In the process according to the flowchart in FIG. 6, the learning processing unit 15 will be described as the subject of the operation.

[0053] In FIG. 6, first, the learning processing unit 15 trains the first encoding model 151, the second encoding model 152, and the estimation model 153 so that the estimation result by the estimation model 153 matches the correct data (step S131).

[0054] Next, the learning processing unit 15 trains the adversarial estimation model 154 so that the estimated value of the second code by the adversarial estimation model 154 matches the second code output from the second encoding model 152 (step S132).

[0055] Next, the learning processing unit 15 trains the first encoding model 151 so that the estimated value of the second code by the adversarial estimation model 154 does not match the second code output from the second encoding model 152 (step S133). Steps S132 and S133 may be interchanged in order or may be performed in parallel.

[0056] As described above, the learning device according to the present embodiment includes an acquisition unit, an encoding unit, an estimation unit, an adversarial estimation unit, and a learning processing unit. The encoding unit includes an encoding model. The estimation unit includes an estimation model. The adversarial estimation unit includes an adversarial estimation model. The acquisition unit acquires a training data set including first sensor data measured by a first measurement device, second sensor data measured by a second measurement device, and correct answer data. The encoding unit encodes the first sensor data into a first code using the first encoding model, and encodes the second sensor data into a second code using the second encoding model. The estimation unit inputs the first code and the second code into the estimation model, and outputs an estimation result output from the estimation model. The adversarial estimation unit inputs the first code into a first adversarial estimation model that outputs an estimation value of the second code in response to an input of the first code, and estimates an estimation value of the second code.

[0057] The learning processing unit trains the first encoding model, the second encoding model, the estimation model, and the first adversarial estimation model by machine learning. The learning processing unit trains the first encoding model, the second encoding model, and the estimation model so that an estimation result of the estimation model matches the correct answer data. The learning processing unit trains the first adversarial estimation model so that an estimated value of the second code by the first adversarial estimation model matches the second code output from the second encoding model. The learning processing unit trains the first encoding model so that an estimated value of the second code by the first adversarial estimation model does not match the second code output from the second encoding model.

[0058] The learning device of this embodiment trains the first adversarial estimation model so that the second code output from the first adversarial estimation model in response to the input of the first code matches the second code output from the second encoding device in response to the input of the second sensor data. This training improves the estimation accuracy of the second code by the first adversarial estimation model. The learning device of this embodiment also trains the first encoding model so that the second code output from the first adversarial estimation model in response to the input of the first code does not match the second code output from the second encoding device in response to the input of the second sensor data. This training reduces the estimation accuracy of the second code by the first adversarial estimation model. That is, the learning device of this embodiment eliminates common features that may be included in the first code output from the first encoding model and the second code output from the second encoding model by training the first adversarial estimation model and the first encoding model adversarially. Therefore, according to the learning device of this embodiment, it is possible to eliminate the redundancy of the code derived from the sensor data measured by multiple measuring instruments and to construct a model that can efficiently reduce the dimension of the sensor data.

[0059] In one aspect of the present embodiment, the learning processing unit trains the first encoding model, the second encoding model, and the estimation model so that the error between the estimation result of the estimation model and the correct answer data is reduced. The learning processing unit trains the first adversarial estimation model so that the error between the estimated value of the second code by the first adversarial estimation model and the second code output from the second encoding model is reduced. The learning processing unit trains the first encoding model so that the error between the estimated value of the second code by the first adversarial estimation model and the second code output from the second encoding model is maximized. According to this aspect, a model that can efficiently reduce the dimension of sensor data can be constructed according to the error between the estimated value of the second code by the first adversarial estimation model and the second code output from the second encoding model.

[0060] Second embodiment Next, a learning device according to a second embodiment will be described with reference to the drawings. The learning device according to this embodiment differs from the first embodiment in that it trains both the first encoding model and the second encoding model adversarially. In the following, the description of the same points as the first embodiment will be omitted / simplified.

[0061] (composition) 8 is a block diagram showing an example of the configuration of the learning device 20 according to this embodiment. The learning device 20 includes an acquisition unit 21, an encoding unit 22, an estimation unit 23, an adversarial estimation unit 24, and a learning processing unit 25. The encoding unit 22 includes a first encoding unit 221 and a second encoding unit 222. The adversarial estimation unit 24 includes a first adversarial estimation unit 241 and a second adversarial estimation unit 242.

[0062] FIG. 9 is a block diagram for explaining a model constructed by the learning device 20. In FIG. 9, the acquisition unit 21 and the learning processing unit 25 are omitted. The first encoding unit 221 includes a first encoding model 251. The second encoding unit 222 includes a second encoding model 252. The estimation unit 23 includes an estimation model 253. The first adversarial estimation unit 241 includes a first adversarial estimation model 254. The second adversarial estimation unit 242 includes a second adversarial estimation model 255. The first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 are also collectively referred to as a model group. The second adversarial estimation unit 242 includes a second adversarial estimation model 255. The first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 will be described in detail later.

[0063] The acquiring unit 21 has the same configuration as the acquiring unit 11 of the first embodiment. The acquiring unit 21 acquires a plurality of data sets (also called training data sets) used to construct a model. The training data set includes a data set combining first raw data, second raw data, and supervised answer data. The first raw data and the second raw data are sensor data measured by different measuring devices. The acquiring unit 21 acquires a training data set including corresponding first raw data, second raw data, and supervised answer data from a plurality of training data sets included in the training data set.

[0064] The encoding unit 22 has the same configuration as the encoding unit 12 of the first embodiment. The encoding unit 22 acquires the first raw data and the second raw data from the acquisition unit 21. The encoding unit 22 encodes the first raw data in the first encoding unit 221. The first raw data encoded in the first encoding unit 221 is a first code. The encoding unit 22 also encodes the second raw data in the second encoding unit 222. The second raw data encoded in the second encoding unit 222 is a second code.

[0065] The first encoding unit 221 has a similar configuration to the first encoding unit 121 of the first embodiment. The first encoding unit 221 acquires first raw data. The first encoding unit 221 inputs the acquired first raw data to the first encoding model 251. The first encoding model 251 has a similar configuration to the first encoding model 151 of the first embodiment. The first encoding model 251 outputs a first code in response to the input of the first raw data. The first code includes the characteristics of the first raw data. That is, the first encoding unit 221 encodes the first raw data to generate a first code including the characteristics of the first raw data.

[0066] The second encoding unit 222 has a similar configuration to the second encoding unit 122 of the first embodiment. The second encoding unit 222 acquires second raw data. The second encoding unit 222 inputs the acquired second raw data to the second encoding model 252. The second encoding model 252 has a similar configuration to the second encoding model 152 of the first embodiment. The second encoding model 252 outputs a second code in response to the input of the second raw data. The second code includes the characteristics of the second raw data. That is, the second encoding unit 222 encodes the second raw data to generate a second code including the characteristics of the second raw data.

[0067] The estimation unit 23 has a configuration similar to that of the estimation unit 13 of the first embodiment. The estimation unit 23 acquires the first code and the second code from the encoding unit 22. The estimation unit 23 inputs the acquired first code and the second code to the estimation model 253. The estimation model 253 has a configuration similar to that of the estimation model 153 of the first embodiment. The estimation model 253 outputs an estimation result regarding the subject's physical state in response to the input of the first code and the second code. That is, the estimation unit 23 estimates the subject's physical state using the first code and the second code. The estimation unit 23 outputs an estimation result regarding the subject's physical state. The estimation result by the estimation unit 23 is compared with the correct answer data of the subject's physical state by the learning processing unit 25.

[0068] The adversarial estimation unit 24 acquires the first code and the second code from the encoding unit 22. The adversarial estimation unit 24 inputs the acquired first code to a first adversarial estimation model 254 of the first adversarial estimation unit 241. In addition, the adversarial estimation unit 24 inputs the acquired second code to a second adversarial estimation model 255 of the second adversarial estimation unit 242.

[0069] The first adversarial estimation model 254 outputs a second code in response to an input of a first code. That is, the first adversarial estimation model 254 estimates a second code by using the first code. The estimate of the second code by the first adversarial estimation unit 241 may include a common point with the first code. The estimate of the second code by the first adversarial estimation unit 241 is compared by the learning processing unit 25 with the second code encoded by the second encoding unit 222.

[0070] The second adversarial estimation model 255 outputs a first code in response to an input of a second code. That is, the second adversarial estimation model 255 estimates the first code by using the second code. The estimate of the first code by the second adversarial estimation unit 242 may include a common point with the second code. The estimate of the first code by the second adversarial estimation unit 242 is compared by the learning processing unit 25 with the first code encoded by the first encoding unit 221.

[0071] For example, the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 include a structure of a DNN (Deep Neural Network). For example, the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 include a structure of a CNN (Convolutional Neural Network). For example, the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 include a structure of a RNN (Recurrent Neural Network). The structures of the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 are not limited to DNN, CNN, or RNN. The first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 are trained by machine learning by the learning processing unit 25.

[0072] The learning processing unit 25 trains a group of models, including the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255, by machine learning. Fig. 10 is a conceptual diagram for explaining the training of the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 by the learning processing unit 25. In Fig. 10, the acquisition unit 21 and the learning processing unit 25 are omitted.

[0073] The learning processing unit 25 trains the first encoding model 251, the second encoding model 252, and the estimation model 253 so that the estimation result of the estimation model 253 matches the correct answer data. That is, the learning processing unit 25 optimizes the model parameters of the first encoding model 251, the second encoding model 252, and the estimation model 253 so that the error between the estimation result of the estimation model 253 and the correct answer data is minimized. For example, the learning processing unit 25 trains the first encoding model 251, the second encoding model 252, and the estimation model 253 so that the error such as the square sum error and the cross entropy error between the output of the estimation model 253 and the correct answer data is minimized. By such training, the accuracy rate of the estimation result output from the estimation model 253 is improved.

[0074] For example, the learning processing unit 25 trains the first encoding model 251, the second encoding model 252, and the estimation model 253 so as to minimize a square sum error or a cross entropy error between the output of the estimation model 253 and the ground truth data. For example, the learning processing unit 25 trains the first encoding model 251, the second encoding model 252, and the estimation model 253 so as to minimize a loss function of the following formula 3.

number

[0075] The learning processing unit 25 trains the first adversarial estimation model 254 so that the estimated value of the second code by the first adversarial estimation model 254 matches the second code output from the second encoding model 252. That is, the learning processing unit 25 optimizes the model parameters of the first adversarial estimation model 254 so that the error between the estimated value of the second code by the first adversarial estimation model 254 and the output value of the second code by the second encoding model 252 is reduced. For example, the learning processing unit 25 trains the first adversarial estimation model 254 so that errors such as the sum of squares error and the cross entropy error between the output (second code) of the second encoding model 252 and the estimated value of the second code by the first adversarial estimation model 254 are minimized. By such training, the accuracy rate of the estimated value of the second code output from the first adversarial estimation model 254 is improved.

[0076] The learning processing unit 25 trains the second adversarial estimation model 255 so that the estimated value of the first code by the second adversarial estimation model 255 matches the first code output from the first encoding model 251. That is, the learning processing unit 25 optimizes the model parameters of the second adversarial estimation model 255 so that the error between the estimated value of the first code by the second adversarial estimation model 255 and the output value of the first code by the first encoding model 251 is reduced. For example, the learning processing unit 25 trains the second adversarial estimation model 255 so that errors such as a square sum error and a cross entropy error between the output (first code) of the first encoding model 251 and the estimated value of the first code by the second adversarial estimation model 255 are minimized. By such training, the accuracy rate of the estimated value of the first code output from the second adversarial estimation model 255 is improved.

[0077] For example, the learning processing unit 25 trains the first adversarial estimation model 254 and the second adversarial estimation model 255 so that the loss function of the following equation 4 is minimized.

number

[0078] The learning processing unit 25 trains the first encoding model 251 so that the estimated value of the second code by the first adversarial estimation model 254 does not match the second code. That is, the learning processing unit 25 optimizes the model parameters of the first encoding model 251 so that the error between the estimated value of the second code by the first adversarial estimation model 254 and the output value of the second code by the second encoding model 252 becomes large. For example, the learning processing unit 25 trains the first encoding model 251 so that the error such as the sum of squares error or the cross entropy error between the output (second code) of the second encoding model 252 and the estimated value of the second code by the first adversarial estimation model 254 becomes maximum. By this training, features that overlap with the second code are eliminated from the first code output from the first encoding model 251.

[0079] The learning processing unit 25 trains the second encoding model 252 so that the estimated value of the first code by the second adversarial estimation model 255 does not match the first code. That is, the learning processing unit 25 optimizes the model parameters of the second encoding model 252 so that the error between the estimated value of the first code by the second adversarial estimation model 255 and the output value of the first code by the first encoding model 251 becomes large. For example, the learning processing unit 25 trains the second encoding model 252 so that the error such as the sum of squares error or the cross entropy error between the output (first code) of the first encoding model 251 and the estimated value of the first code by the second adversarial estimation model 255 becomes maximum. By this training, the features overlapping with the first code are eliminated from the second code output from the second encoding model 252.

[0080] In this embodiment, the first adversarial estimation model 254 is trained so as to improve the accuracy rate of the estimated value of the second code, and the first encoding model 251 is trained so as to reduce the overlap between the first code and the second code. In addition, in this embodiment, the second adversarial estimation model 255 is trained so as to improve the accuracy rate of the estimated value of the first code, and the second encoding model 252 is trained so as to reduce the overlap between the first code and the second code. In this manner, in this embodiment, the first encoding model 251 and the first adversarial estimation model 254 are trained adversarially, and the second encoding model 252 and the second adversarial estimation model 255 are trained adversarially. As a result, common features that may be included in the first code output from the first encoding model 251 and the second code output from the second encoding model 252 are eliminated.

[0081] In this embodiment, an example of eliminating duplicates that may be included in sensor data measured by two measurement devices will be described. The method of this embodiment may be used to eliminate duplicates that may be included in sensor data measured by three or more measurement devices.

[0082] Among the group of models trained by the learning processing unit 25, the first encoding model 251, the second encoding model 252, and the estimation model 253 are implemented in an estimation system (not shown) that performs estimation based on raw data. For example, the estimation system includes a first measurement device that measures first measurement data (first raw data), a second measurement device that measures second measurement data (second raw data), and an estimation device (not shown) that performs estimation using the measurement data. The first encoding model 251 is implemented in the first measurement device. The second encoding model 252 is implemented in the second measurement device. The estimation model 253 is implemented in the estimation device. The first measurement device encodes the first measurement data into a first code using the first encoding model. The first measurement device transmits the encoded first code to the estimation device. The second measurement device encodes the second measurement data into a second code using the first encoding model. The second measurement device transmits the encoded second code to the estimation device. The estimation device inputs the first code received from the first measuring device and the second code received from the second measuring device to an estimation model. The estimation device outputs an estimation result output from the estimation model in response to the input of the first code and the second code. Details of the estimation system using the model trained by the learning processing unit 25 will be described later.

[0083] (operation) Next, the operation of the learning device 20 of this embodiment will be described with reference to the drawings. Figures 11 to 13 are flowcharts for explaining an example of the operation of the learning device 20. In the explanation following the flowchart of Figure 11, the learning device 10 will be described as the subject of the operation.

[0084] In FIG. 11, first, the learning device 20 acquires the first raw data, the second raw data, and the correct answer data from the training data set (step S21).

[0085] Next, the learning device 20 executes an estimation process using a group of models including the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 (step S22). In the estimation process, the first encoding model 251 performs encoding to a first code, the second encoding model 252 performs encoding to a second code, and the estimation model 253 estimates an estimation result. In addition, in the estimation process, the first adversarial estimation model 254 estimates a second code, and the second adversarial estimation model 255 estimates a first code. Details of the estimation process in step S22 will be described later.

[0086] Next, the learning device 20 executes training processing of the first encoding model 251, the second encoding model 252, the estimation model 253, the first adversarial estimation model 254, and the second adversarial estimation model 255 according to the estimation results of the model group (step S23). The model parameters of the model group trained by the learning processing unit 25 are set in the first encoding model 251, the second encoding model 252, and the estimation model 253 implemented in an estimation system (not shown). Details of the training processing in step S23 will be described later.

[0087] If the learning is to be continued (Yes in step S24), the process returns to step S21. On the other hand, if the learning is to be stopped (No in step S24), the process according to the flowchart in FIG. 11 ends. The continuation / end of the learning may be determined based on a preset criterion. For example, the learning device 20 determines whether to continue / end the learning according to the accuracy rate of the estimation result by the estimation model 253. For example, the learning device 20 determines whether to continue / end the learning according to the error between the estimated value of the second code by the first adversarial estimation unit 241 and the second code output from the second encoding model 252. For example, the learning device 20 determines whether to continue / end the learning according to the error between the estimated value of the second code by the second adversarial estimation unit 242 and the first code output from the first encoding model 251.

[0088] [Estimation process] Next, the estimation process (step S22 in FIG. 11) by the learning processing unit 25 will be described with reference to the drawings. Fig. 12 is a flowchart for explaining the estimation process by the learning processing unit 25. In the process according to the flowchart in Fig. 12, the learning device 20 will be described as the subject of operations.

[0089] 12, first, the learning device 20 inputs first raw data to the first encoding model 251 and calculates a first code (step S221). In response to the input of the first raw data, the code output from the first encoding model 251 is the first code.

[0090] Next, the learning device 20 inputs the second raw data to the second encoding model 252 to calculate the second code (step S222). In response to the input of the second raw data, the code output from the second encoding model 252 is the second code. Steps S221 and S222 may be interchanged in order or may be performed in parallel.

[0091] Next, the learning device 20 inputs the first code and the second code to the estimation model 253 and calculates an estimation result (step S223). The result output from the estimation model 253 in response to the input of the first raw data and the second raw data is the estimation result.

[0092] Next, the learning device 20 inputs the first code to the first adversarial estimation model 254 and calculates an estimate of the second code (step S224). In response to the input of the first code, the code output from the first adversarial estimation model 254 is the estimate of the second code.

[0093] Next, the learning device 20 inputs the second code to the second adversarial estimation model 255 and calculates an estimate of the first code (step S225). In response to the input of the second code, the code output from the second adversarial estimation model 255 is the estimate of the first code. Steps S223 to S225 may be performed in a reverse order or in parallel.

[0094] [Training process] Next, the training process (step S23 in FIG. 11) by the learning processing unit 25 will be described with reference to the drawings. Fig. 13 is a flowchart for explaining the training process by the learning processing unit 25. In the process according to the flowchart in Fig. 13, the learning processing unit 25 will be described as the subject of the operation.

[0095] In FIG. 13, first, the learning processing unit 25 trains the first encoding model 251, the second encoding model 252, and the estimation model 253 so that the estimation result by the estimation model 253 matches the correct data (step S231).

[0096] Next, the learning processing unit 25 trains the first adversarial estimation model 254 so that the estimated value of the second code by the first adversarial estimation model 254 matches the second code output from the second encoding model 252 (step S232).

[0097] Next, the learning processing unit 25 trains the second adversarial estimation model 255 so that the estimated value of the first code by the second adversarial estimation model 255 matches the first code output from the first encoding model 251 (step S233). Steps S232 and S233 may be interchanged in order or may be performed in parallel.

[0098] Next, the learning processing unit 25 trains the first encoding model 251 so that the estimated value of the second code by the first adversarial estimation model 254 does not match the second code output from the second encoding model 252 (step S234).

[0099] Next, the learning processing unit 25 trains the second encoding model 252 so that the estimated value of the first code by the second adversarial estimation model 255 does not match the first code output from the first encoding model 251 (step S235). Steps S234 and S235 may be interchanged in order or may be performed in parallel.

[0100] As described above, the learning device according to the present embodiment includes an acquisition unit, an encoding unit, an estimation unit, an adversarial estimation unit, and a learning processing unit. The encoding unit includes a first encoding model and a second encoding model. The estimation unit includes an estimation model. The adversarial estimation unit includes a first adversarial estimation model and a second adversarial estimation model. The acquisition unit acquires a training data set including first sensor data measured by a first measurement device, second sensor data measured by a second measurement device, and correct answer data. The encoding unit encodes the first sensor data into a first code using the first encoding model, and encodes the second sensor data into a second code using the second encoding model. The estimation unit inputs the first code and the second code into an estimation model and outputs an estimation result output from the estimation model. The adversarial estimation unit inputs the first code into a first adversarial estimation model that outputs an estimation value of the second code in response to an input of the first code, and estimates an estimation value of the second code. The adversarial estimation unit inputs the second code to a second adversarial estimation model that outputs an estimate of the first code in response to an input of the second code, and estimates an estimate of the first code.

[0101] The learning processing unit trains the first encoding model, the second encoding model, the estimation model, the first adversarial estimation model, and the second adversarial encoding model by machine learning. The learning processing unit trains the first encoding model, the second encoding model, and the estimation model so that an estimation result of the estimation model matches the correct answer data. The learning processing unit trains the first adversarial estimation model so that an estimated value of the second code by the first adversarial estimation model matches the second code output from the second encoding model. The learning processing unit trains the second adversarial estimation model so that an estimated value of the first code by the second adversarial estimation model matches the first code output from the first encoding model. The learning processing unit trains the first encoding model so that an estimated value of the second code by the first adversarial estimation model does not match the second code output from the second encoding model. The learning processing unit trains the second encoding model so that an estimated value of the first code by the second adversarial estimation model does not match the first code output from the first encoding model.

[0102] The learning device of this embodiment trains the first adversarial estimation model so that the second code output from the first adversarial estimation model in response to the input of the first code matches the second code output from the second encoding device in response to the input of the second sensor data. The learning device of this embodiment trains the second adversarial estimation model so that the first code output from the second adversarial estimation model in response to the input of the second code matches the first code output from the first encoding device in response to the input of the first sensor data. These trainings improve the estimation accuracy of the second code by the first adversarial estimation model and the estimation accuracy of the first code by the second adversarial estimation model.

[0103] The learning device of this embodiment trains the first encoding model so that the second code output from the first adversarial estimation model in response to the input of the first code does not match the second code output from the second encoding device in response to the input of the second sensor data. The learning device of this embodiment trains the second encoding model so that the first code output from the second adversarial estimation model in response to the input of the second code does not match the first code output from the first encoding device in response to the input of the first sensor data. These trainings reduce the estimation accuracy of the second code by the first adversarial estimation model and the estimation accuracy of the first code by the second adversarial estimation model. That is, the learning device of this embodiment trains the first adversarial estimation model and the first encoding model adversarially, and trains the second adversarial estimation model and the second encoding model adversarially. As a result, common features that may be included in the first code output from the first encoding model and the second code output from the second encoding model are eliminated. Therefore, according to the learning device of this embodiment, it is possible to construct a model that can eliminate code redundancy derived from sensor data measured by multiple measuring instruments and efficiently reduce the dimensionality of the sensor data.

[0104] In one aspect of the present embodiment, the learning processing unit trains the second adversarial estimation model so that an error between an estimated value of the first code by the second adversarial estimation model and the first code output from the first encoding model is reduced. The learning processing unit trains the second encoding model so that an error between an estimated value of the first code by the second adversarial estimation model and the first code output from the first encoding model is increased. According to this aspect, a model that can efficiently reduce the dimension of sensor data can be constructed according to the error between an estimated value of the first code by the second adversarial estimation model and the first code output from the first encoding model.

[0105] The adversarial estimation of this embodiment may be applied to three or more measurement devices. For example, when there are three measurement devices, adversarial estimation is performed between all the measurement devices. By performing adversarial estimation in this manner, duplication of codes according to the sensor data measured is eliminated for all the measurement devices. For example, when there are three measurement devices, at least one pair of two measurement devices may be selected from the three measurement devices, and adversarial estimation may be performed for the pair of measurement devices. By performing adversarial estimation in this manner, duplication of codes according to the sensor data measured is eliminated between the measurement devices for which adversarial estimation has been performed.

[0106] (Third embodiment) Next, a learning device according to a third embodiment will be described with reference to the drawings. The learning device of this embodiment has a simplified configuration of the learning device of the first and second embodiments.

[0107] (composition) 14 is a block diagram showing an example of the configuration of a learning device 30 according to this embodiment. The learning device 30 includes an acquisition unit 31, an encoding unit 32, an estimation unit 33, an adversarial estimation unit 34, and a learning processing unit 35. The encoding unit 32 includes an encoding model. The estimation unit 33 includes an estimation model. The adversarial estimation unit 34 includes an adversarial estimation model.

[0108] The acquisition unit 31 acquires a training data set including first sensor data measured by a first measurement device, second sensor data measured by a second measurement device, and correct answer data. The encoding unit 32 encodes the first sensor data into a first code using a first encoding model, and encodes the second sensor data into a second code using a second encoding model. The estimation unit 33 inputs the first code and the second code into an estimation model and outputs an estimation result output from the estimation model. The adversarial estimation unit 34 inputs the first code into a first adversarial estimation model that outputs an estimation value of the second code in response to the input of the first code, and estimates an estimation value of the second code. The learning processing unit 35 trains the first encoding model, the second encoding model, the estimation model, and the first adversarial estimation model by machine learning. The learning processing unit 35 trains the first encoding model, the second encoding model, and the estimation model so that the estimation result of the estimation model fits the correct answer data. The learning processing unit 35 trains the first adversarial estimation model so that an estimated value of the second code by the first adversarial estimation model matches the second code output from the second encoding model. The learning processing unit 35 trains the first encoding model so that an estimated value of the second code by the first adversarial estimation model does not match the second code output from the second encoding model.

[0109] The learning device of this embodiment eliminates common features that may be included in the first code output from the first encoding model and the second code output from the second encoding model by adversarially training the first adversarial estimation model and the first encoding model. Therefore, the learning device of this embodiment can eliminate code redundancy derived from sensor data measured by multiple measuring instruments and construct a model that can efficiently reduce the dimension of the sensor data.

[0110] (Fourth embodiment) Next, an estimation system according to a fourth embodiment will be described with reference to the drawings. The estimation system of this embodiment includes an estimation device including a first encoding model, a second encoding model, and an estimation model constructed by the learning device of the first to third embodiments. The estimation system of this embodiment includes a first measuring device installed in footwear worn by a user. The first measuring device measures a physical quantity (first sensor data) corresponding to foot movement. The estimation system of this embodiment includes a second measuring device worn on the wrist of the user. The second measuring device measures a physical quantity and biological data (second sensor data) corresponding to physical activity. The estimation system of this embodiment performs estimation regarding the user's physical condition based on the measured first sensor data and second sensor data.

[0111] The first measuring device and the second measuring device may be attached to a body part other than the foot or the wrist. For example, the first measuring device may be attached to the foot of the left foot, and the second measuring device may be attached to the foot of the right foot. For example, the first measuring device may be attached to the wrist of the left hand, and the second measuring device may be attached to the wrist of the right hand. For example, the first measuring device and the second measuring device may be attached to the same body part. The attachment locations of the first measuring device and the second measuring device are not limited as long as appropriate physical quantities / biological data can be measured according to the physical activity of the user.

[0112] (composition) 15 is a block diagram showing an example of a configuration of an estimation system 40 according to this embodiment. The estimation system 40 includes a first measuring device 41, a second measuring device 42, and an estimation device 47. The first measuring device 41 and the estimation device 47 may be connected to each other by wire or wirelessly. Similarly, the second measuring device 42 and the estimation device 47 may be connected to each other by wire or wirelessly.

[0113] [First measuring device] The first measurement device 41 is installed on the foot. For example, the first measurement device 41 is installed on footwear such as shoes. In this embodiment, an example in which the first measurement device 41 is placed at a position on the back side of the arch of the foot will be described.

[0114] FIG. 16 is a conceptual diagram showing an example of arranging the first measuring device 41 in the footwear 400. In the example of FIG. 16, the first measuring device 41 is arranged at a position that hits the back side of the arch of the foot. For example, the first measuring device 41 is arranged in an insole inserted into the footwear 400. For example, the first measuring device 41 is arranged on the bottom surface of the footwear 400. For example, the first measuring device 41 is embedded in the body of the footwear 400. The first measuring device 41 may be detachable from the footwear 400, or may not be detachable from the footwear 400. Note that the first measuring device 41 may be arranged at a position other than the back side of the arch of the foot, as long as it can acquire sensor data related to the movement of the foot. Also, the first measuring device 41 may be arranged in socks worn by the user or in an accessory such as an anklet worn by the user. Also, the first measuring device 41 may be directly attached to the foot or embedded in the foot. FIG. 17 shows an example in which the first measuring device 41 is installed in footwear 400 for both the left and right feet, but the first measuring device 41 may be installed in footwear 400 for one foot.

[0115] 17 is a block diagram showing an example of a detailed configuration of the first measuring device 41. The first measuring device 41 has a sensor 410, a control unit 415, a first encoding unit 416, and a transmission unit 417. The sensor 410 includes an acceleration sensor 411 and an angular velocity sensor 412. The sensor 410 may include sensors other than the acceleration sensor 411 and the angular velocity sensor 412. The first encoding unit 416 includes a first encoding model 451. The first measuring device 41 also includes a real-time clock and a power supply, which are not shown.

[0116] Acceleration sensor 411 is a sensor that measures acceleration in three axial directions (also called spatial acceleration). Acceleration sensor 411 outputs the measured acceleration to control unit 415. For example, a piezoelectric type, a piezoresistance type, a capacitance type, or other type of sensor can be used for acceleration sensor 411. Note that the measurement type of the sensor used for acceleration sensor 411 is not limited as long as it can measure acceleration.

[0117] Angular velocity sensor 412 is a sensor that measures angular velocity (also called spatial angular velocity) in three axial directions. Angular velocity sensor 412 outputs the measured angular velocity to control unit 415. For example, a vibration type, a capacitance type, or other type of sensor can be used for angular velocity sensor 412. Note that the measurement method of the sensor used for angular velocity sensor 412 is not limited as long as it can measure angular velocity.

[0118] The first measuring device 41 has an inertial measurement device including, for example, an acceleration sensor 411 and an angular velocity sensor 412. An example of the inertial measurement device is an IMU (Inertial Measurement Unit). The IMU includes an acceleration sensor that measures acceleration in three axial directions and an angular velocity sensor that measures angular velocity around three axes. The first measuring device 41 may be realized by an inertial measurement device such as a VG (Vertical Gyro) or an AHRS (Attitude Heading). The first measuring device 41 may be realized by a GPS / INS (Global Positioning System / Inertial Navigation System).

[0119] The control unit 415 acquires the acceleration in three axial directions and the angular velocity around the three axes from each of the acceleration sensor 411 and the angular velocity sensor 412. The control unit 415 converts the acquired acceleration and angular velocity into digital data, and outputs the converted digital data (also referred to as first sensor data) to the first encoding unit 416. The first sensor data includes at least the acceleration data converted into digital data and the angular velocity data converted into digital data. The acceleration data includes acceleration vectors in three axial directions. The angular velocity data includes angular velocity vectors around the three axes. The first sensor data is associated with the acquisition time of those data. The control unit 415 may be configured to output the first sensor data obtained by applying corrections such as mounting error, temperature correction, and linearity correction to the acquired acceleration data and angular velocity data. The control unit 415 may also generate angle data around the three axes using the acquired acceleration data and angular velocity data.

[0120] For example, the control unit 415 is a microcomputer or a microcontroller that performs overall control of the first measuring device 41 and data processing. For example, the control unit 415 has a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, and the like. The control unit 415 controls the acceleration sensor 411 and the angular velocity sensor 412 to measure the angular velocity and acceleration. For example, the control unit 415 performs AD conversion (Analog-to-Digital Conversion) of physical quantities (analog data) such as the measured angular velocity and acceleration, and stores the converted digital data in a flash memory. Note that the physical quantities (analog data) measured by the acceleration sensor 411 and the angular velocity sensor 412 may be converted into digital data in each of the acceleration sensor 411 and the angular velocity sensor 412. The digital data stored in the flash memory is output to the first encoding unit 416 at a predetermined timing.

[0121] The first encoding unit 416 acquires the first sensor data from the control unit 415. The first encoding unit 416 includes a first encoding model 451. The first encoding model 451 is a first encoding model constructed by the learning device of the first to third embodiments. For example, a model parameter set by the learning device of the first or third embodiment is set in the first encoding model 451. The first encoding unit 416 inputs the acquired first sensor data to the first encoding model 451 and encodes the data into a first code. The first encoding unit 416 outputs the encoded first code to the transmission unit 417.

[0122] The transmitting unit 417 acquires the first code from the first encoding unit 416. The transmitting unit 417 transmits the acquired first code to the estimation device 47. The transmitting unit 417 may transmit the first code to the estimation device 47 via a wired connection such as a cable, or may transmit the first code to the estimation device 47 via wireless communication. For example, the transmitting unit 417 is configured to transmit the first code to the estimation device 47 via a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the transmitting unit 417 may conform to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark). The transmitting unit 417 also has a function of receiving data transmitted from the estimation device 47. For example, the transmitting unit 417 receives update data of model parameters, universal time data, and the like from the estimation device 47. The transmitting unit 417 outputs the received data to the control unit 415.

[0123] For example, the first measuring device 41 is connected to the estimation device 47 via a mobile terminal (not shown) carried by a user. When the communication between the first measuring device 41 and the mobile terminal is successful and the first code is transmitted from the first measuring device 41 to the mobile terminal, the measurement in that measurement time period is completed. For example, when the communication between the first measuring device 41 and the mobile terminal is successful, the time of the clock of the first measuring device 41 may be synchronized with the time of the mobile terminal. When the communication between the first measuring device 41 and the mobile terminal is unsuccessful and the first code is not transmitted from the first measuring device 41 to the mobile terminal, the first code in that measurement time period may be retransmitted in the next or subsequent measurement time period. For example, when the communication between the first measuring device 41 and the mobile terminal is unsuccessful, the transmission of the first code in that measurement time period may be repeated until the communication is successful. For example, when the communication between the first measuring device 41 and the mobile terminal is unsuccessful, the transmission of the first code in that measurement time period may be repeated within a predetermined time. The first code for the measurement time period in which the transmission failed may be stored in a storage device (not shown) such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) until the next transmission timing.

[0124] When the first measuring devices 41 are mounted on both the left and right feet, the time of the first measuring devices 41 mounted on both feet can be synchronized by synchronizing the clock times of the first measuring devices 41 with the time of the mobile terminal. The first measuring devices 41 mounted on both feet may perform measurements at the same timing or at different timings. For example, if the measurement timings of the first measuring devices 41 mounted on both feet are significantly different based on the measurement times of both feet or the number of measurement failures, a correction may be made to reduce the difference in the measurement timing. The correction of the measurement timing may be made in the estimation device 47 capable of processing the first code transmitted from the first measuring devices 41 installed on both feet, or in a higher-level system.

[0125] [Second measuring device] The second measuring device 42 is placed on the wrist. The second measuring device 42 collects information according to the user's physical activity. For example, the second measuring device 42 is a wristwatch-type wearable device worn on the wrist. For example, the second measuring device 42 is realized by an activity meter. For example, the second measuring device 42 is realized by a smart watch. For example, the second measuring device 42 may include a Global Positioning System (GPS).

[0126] 18 is a conceptual diagram showing an example in which the second measuring device 42 is placed on the wrist. The second measuring device 42 may be worn on a part other than the wrist as long as it can collect information according to the user's physical activity. For example, the second measuring device 42 may be worn on the head, neck, chest, back, waist, abdomen, thighs, lower legs, ankles, etc. There is no particular limitation on the part where the second measuring device 42 is worn. In addition, the second measuring device 42 may be worn on multiple body parts.

[0127] 19 is a block diagram showing an example of a detailed configuration of the second measuring device 42. The second measuring device 42 has a sensor 420, a control unit 425, a second encoding unit 426, and a transmission unit 427. The sensor 420 includes an acceleration sensor 421, an angular velocity sensor 422, a pulse sensor 423, and a temperature sensor 424. The sensor 420 may include sensors other than the acceleration sensor 421, the angular velocity sensor 422, the pulse sensor 423, and the temperature sensor 424. The second encoding unit 426 includes a second encoding model 452. The second measuring device 42 also includes a real-time clock and a power supply, which are not shown.

[0128] Acceleration sensor 421 is a sensor that measures acceleration in three axial directions (also called spatial acceleration). Acceleration sensor 421 outputs the measured acceleration to control unit 425. For example, a piezoelectric type, a piezoresistance type, a capacitance type, or other type of sensor can be used for acceleration sensor 421. Note that the measurement type of the sensor used for acceleration sensor 421 is not limited as long as it can measure acceleration.

[0129] Angular velocity sensor 422 is a sensor that measures angular velocity (also called spatial angular velocity) in three axial directions. Angular velocity sensor 422 outputs the measured angular velocity to control unit 425. For example, a vibration type, a capacitance type, or other type of sensor can be used for angular velocity sensor 422. Note that the measurement method of the sensor used for angular velocity sensor 422 is not limited as long as it can measure angular velocity.

[0130] The pulse sensor 423 measures the pulse of the user. For example, the pulse sensor 423 is a sensor using a photoplethysmography method. For example, the pulse sensor 423 is realized by a reflective pulse sensor. The reflective pulse sensor receives the reflected light of light irradiated toward a living body with a photodiode or a phototransistor. The reflective pulse sensor measures the pulse wave according to the change in intensity of the received reflected light. For example, the reflective pulse sensor measures the pulse wave using light in the infrared, red, or green wavelength band. The light reflected in the living body is absorbed by oxyhemoglobin contained in the blood in the artery. The reflective pulse sensor measures the pulse wave according to the periodicity of the blood flow rate that changes with the pulsation of the heart. For example, the pulse wave is used to evaluate the pulse rate, oxygen saturation, stress level, vascular age, and the like. Note that the measurement method of the sensor used in the pulse sensor 423 is not limited as long as it can measure the pulse.

[0131] The temperature sensor 424 measures the body temperature (skin temperature) of the user. For example, the temperature sensor 424 is realized by a contact-type temperature sensor such as a thermistor, a thermocouple, or a resistance temperature detector. For example, the temperature sensor 424 is realized by a non-contact-type temperature sensor such as a radiation temperature sensor or a color temperature sensor. For example, the temperature sensor 424 may be a sensor that estimates the body temperature based on the measured values ​​of biological data such as pulse and blood pressure. For example, the temperature sensor 424 measures the temperature of the user's body surface. For example, the temperature sensor 424 estimates the body temperature of the user according to the temperature of the user's body surface. Note that there is no limitation on the measurement method of the sensor used for the temperature sensor 424 as long as it can measure temperature.

[0132] The control unit 425 acquires the acceleration in three axial directions from the acceleration sensor 421, and the angular velocity around three axes from the angular velocity sensor 422. The control unit 425 also acquires a pulse signal from the pulse sensor 423, and a temperature signal from the temperature sensor 424. The control unit 425 converts the acquired physical quantities such as acceleration and angular velocity, and biological information such as a pulse signal and a temperature signal into digital data. The control unit 425 outputs the converted digital data (also called second sensor data) to the second encoding unit 426. The second sensor data includes at least the acceleration data, angular velocity data, pulse data, and temperature data converted into digital data. The second sensor data is associated with the acquisition time of those data. The control unit 425 may also be configured to output the second sensor data obtained by applying corrections such as mounting error, temperature correction, and linearity correction to the acquired acceleration data, angular velocity data, pulse data, and temperature data.

[0133] For example, the control unit 425 is a microcomputer or microcontroller that performs overall control of the second measuring device 42 and data processing. For example, the control unit 425 has a CPU, a ROM, a flash memory, etc. The control unit 425 controls the acceleration sensor 421 and the angular velocity sensor 422 to measure angular velocity and acceleration. The control unit 425 controls the pulse sensor 423 and the temperature sensor 424 to measure pulse and temperature. For example, the control unit 425 performs AD conversion of angular velocity data, acceleration data, pulse data, and temperature data. The control unit 425 stores the converted digital data in a flash memory. The physical quantity (analog data) measured by the acceleration sensor 421 and the angular velocity sensor 422 may be converted into digital data in each of the acceleration sensor 421 and the angular velocity sensor 422. The biological information (analog data) measured by the pulse sensor 423 and the temperature sensor 424 may be converted into digital data in each of the pulse sensor 423 and the temperature sensor 424. The digital data stored in the flash memory is output to the second encoding unit 426 at a predetermined timing.

[0134] The second encoding unit 426 acquires the second sensor data from the control unit 425. The second encoding unit 426 includes a second encoding model 452. The second encoding model 452 is a second encoding model constructed by the learning device of the first to third embodiments. For example, the model parameters set by the learning device of the first to third embodiments are set in the second encoding model 452. The second encoding unit 426 inputs the acquired second sensor data to the second encoding model 452 and encodes the data into a second code. The second encoding unit 426 outputs the encoded second code to the transmission unit 427.

[0135] The transmitting unit 427 acquires the second code from the second encoding unit 426. The transmitting unit 427 transmits the acquired second code to the estimation device 47. The transmitting unit 427 may transmit the second code to the estimation device 47 via a wired connection such as a cable, or may transmit the second code to the estimation device 47 via wireless communication. For example, the transmitting unit 427 is configured to transmit the second code to the estimation device 47 via a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the transmitting unit 427 may conform to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark). The transmitting unit 427 also has a function of receiving data transmitted from the estimation device 47. For example, the transmitting unit 427 receives update data of model parameters, universal time data, and the like from the estimation device 47. The transmitting unit 427 outputs the received data to the control unit 425.

[0136] For example, the second measuring device 42 is connected to the estimation device 47 via a mobile terminal (not shown) carried by a user. When the communication between the second measuring device 42 and the mobile terminal is successful and the second code is transmitted from the second measuring device 42 to the mobile terminal, the measurement in that measurement time period is completed. For example, when the communication between the second measuring device 42 and the mobile terminal is successful, the time of the clock of the second measuring device 42 may be synchronized with the time of the mobile terminal. When the communication between the second measuring device 42 and the mobile terminal is unsuccessful and the second code is not transmitted from the second measuring device 42 to the mobile terminal, the second code in that measurement time period may be retransmitted in the next or subsequent measurement time period. For example, when the communication between the second measuring device 42 and the mobile terminal is unsuccessful, the transmission of the second code in that measurement time period may be repeated until the communication is successful. For example, when the communication between the second measuring device 42 and the mobile terminal is unsuccessful, the transmission of the second code in that measurement time period may be repeated within a predetermined time. The second code for the measurement time period in which the transmission failed may be stored in a storage device (not shown) such as an EEPROM until the next transmission timing.

[0137] A mobile terminal (not shown) connected to the first measuring device 41 and the second measuring device 42 is realized by a communication device that can be carried by a user. For example, the mobile terminal is a mobile communication device having a communication function such as a smartphone, a smart watch, or a mobile phone. If the mobile terminal is a smart watch, the second measuring device 42 may be implemented in the smart watch. The mobile terminal receives first sensor data related to the movement of the user's feet from the first measuring device 41. The mobile terminal receives second sensor data corresponding to the user's physical activity from the second measuring device 42. The mobile terminal transmits the received code to a cloud or a server in which the estimation device 47 is implemented. Note that the function of the estimation device 47 may be realized by application software or the like (also called an app) installed in the mobile terminal. In that case, the mobile terminal processes the received code by the app installed in itself.

[0138] For example, when starting to use the estimation system 40 of this embodiment, an app that executes the functions of the estimation system 40 is downloaded to the user's mobile terminal and user information is registered. For example, when registering user information in the first measuring device 41 and the second measuring device 42, the clock times of the first measuring device 41 and the second measuring device 42 are synchronized with the time of the mobile terminal. If synchronized in this way, the unique times of the first measuring device 41 and the second measuring device 42 can be set to match the universal time.

[0139] The measurement timing of the first measuring device 41 and the second measuring device 42 may or may not be synchronized. If time data is associated with the measurement data measured by the first measuring device 41 and the second measuring device 42, the measurement data measured by the first measuring device 41 and the second measuring device 42 can be temporally associated. For this reason, it is preferable that the times of the first measuring device 41 and the second measuring device 42 are synchronized. For example, the estimation device 47 may be configured to synchronize the time difference between the first measuring device 41 and the second measuring device 42.

[0140] [Estimation device] 20 is a block diagram showing an example of the configuration of the estimation device 47. The estimation device 47 has a receiving unit 471, an estimation unit 473, and an output unit 475. The estimation unit 473 includes an estimation model 453.

[0141] The receiving unit 471 acquires the first code from the first measuring device 41. The receiving unit 471 also acquires the second code from the second measuring device 42. The receiving unit 471 receives the code from the first measuring device 41. The receiving unit 471 outputs the received first code and second code to the estimating unit 473. For example, the receiving unit 471 receives the first code from the first measuring device 41 and receives the first code from the second measuring device 42 via wireless communication. For example, the receiving unit 471 is configured to receive the first code from the first measuring device 41 and receive the first code from the second measuring device 42 via a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the receiving unit 471 may be conforming to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark). For example, the receiving unit 471 may receive the first code from the first measuring device 41 and receive the first code from the second measuring device 42 via a wired connection such as a cable. For example, the receiving unit 471 may have a function of transmitting data to the first measuring equipment 41 and the second measuring equipment .

[0142] The estimation unit 473 acquires the first code and the second code from the receiving unit 471. The estimation unit 473 includes an estimation model 453. The estimation model 453 is an estimation model constructed by the learning device of the first or third embodiment. The estimation unit 473 implements the estimation model 453 constructed by the learning device of the first to third embodiments. The estimation model 453 is set with model parameters set by the learning device of the first to third embodiments.

[0143] The estimation unit 473 inputs the acquired first code and second code to the estimation model 453. The estimation model 453 outputs an estimation result regarding the user's physical condition in response to the input of the first code and the second code. The estimation unit 473 outputs the estimation result by the estimation model 453. For example, the estimation unit 473 estimates a score regarding the user's physical condition. For example, the score is a value obtained by indexing an evaluation regarding the user's physical condition.

[0144] For example, the estimation unit 473 estimates the user's physical condition using a first code derived from sensor data related to foot movement measured by the first measuring device 41. For example, the physical condition includes the degree of pronation / supination of the foot, the progress of hallux valgus, the progress of knee osteoarthritis, muscle strength, balance ability, body flexibility, and the like. For example, the estimation unit 473 estimates the physical condition using physical quantities such as acceleration, velocity, trajectory (position), angular velocity, and angle measured by the first measuring device 41. There are no particular limitations on the estimation by the estimation unit 473 as long as it is an estimation related to the physical condition. The estimation unit 473 outputs the estimation result to the output unit 475.

[0145] For example, the estimation unit 473 may be configured to estimate the user's emotion using pulse data measured by the second measuring device 42. The user's emotion can be estimated from the intensity and fluctuation of the pulse. For example, the estimation device 47 estimates the degree of emotion such as joy, anger, sadness, and happiness according to the fluctuation of the time series data of the pulse. For example, the estimation device 47 may estimate the user's emotion according to the fluctuation of the baseline of the time series data regarding the pulse. For example, when the user's "anger" gradually increases, an upward trend appears in the baseline according to an increase in the user's excitement level (arousal level). For example, when the user's "sadness" gradually increases, a downward trend appears in the baseline according to a decrease in the user's excitement level (arousal level).

[0146] The heart rate fluctuates under the influence of autonomic nerve activity such as the sympathetic and parasympathetic nerves. Similarly, the pulse rate fluctuates under the influence of autonomic nerve activity such as the sympathetic and parasympathetic nerves. For example, low and high frequency components can be extracted by frequency analysis of the time series data of the pulse rate. The influence of the sympathetic and parasympathetic nerves is reflected in the low frequency components. The influence of the parasympathetic nerves is reflected in the high frequency components. Therefore, for example, the activity state of the autonomic nerve function can be estimated according to the ratio between the high frequency components and the low frequency components.

[0147] For example, the estimation device 47 estimates the user's emotion according to the arousal level and valence. The sympathetic nerves tend to be active when the user is excited. When the user's sympathetic nerves are active, the pulse rate becomes faster. That is, the higher the pulse rate, the higher the arousal level. The parasympathetic nerves tend to be active when the user is relaxed. When the user is relaxed, the pulse rate becomes slower. That is, the lower the pulse rate, the lower the arousal level. In this way, the estimation device 47 can measure the arousal level according to the pulse rate. For example, the valence level can be evaluated according to the fluctuation of the pulse interval. The more pleasant the emotional state, the more stable the emotion and the smaller the fluctuation of the pulse interval. That is, the smaller the fluctuation of the pulse interval, the higher the valence. On the other hand, the more unpleasant the emotional state, the more unstable the emotion and the larger the fluctuation of the pulse interval. That is, the larger the fluctuation of the pulse interval, the higher the valence. In this way, the estimation device 47 can measure the valence according to the pulse interval.

[0148] For example, the estimation device 47 estimates that the greater the valence and the greater the arousal level, the greater the degree of "happiness." For example, the estimation device 47 estimates that the lower the valence and the greater the arousal level, the greater the degree of "anger." For example, the estimation device 47 estimates that the lower the valence and the smaller the arousal level, the greater the degree of "sadness." For example, the estimation device 47 estimates that the greater the valence and the smaller the arousal level, the greater the degree of "happiness." For example, the user's emotions may be classified into more detailed emotional states rather than being classified into four emotional states such as joy, anger, sadness, and happiness.

[0149] The output unit 475 acquires the estimation result by the estimation unit 473. The output unit 475 outputs the estimation result by the estimation unit 473. For example, the output unit 475 outputs the estimation result by the estimation unit 473 to a display device (not shown). For example, the estimation result by the estimation unit 473 is displayed on the screen of the display device. For example, the estimation result by the estimation unit 473 is output to a system that uses the estimation result. There are no particular limitations on the use of the estimation result by the estimation unit 473.

[0150] For example, the estimation device 47 is implemented in a cloud or a server (not shown). For example, the estimation device 47 may be realized by an application server. For example, the estimation device 47 may be realized by an app installed on a mobile terminal (not shown). For example, the estimation result by the estimation device 47 is displayed on the screen of a mobile terminal (not shown) or a terminal device (not shown) carried by a user. For example, the estimation result by the estimation device 47 is output to a system that uses the result. There is no particular limitation on the use of the estimation result by the estimation device 47.

[0151] FIG. 21 is a conceptual diagram for explaining the setting of model parameters to the model group implemented in the estimation system 40, the estimation process of the user's physical state by the estimation system 40, and the like. In the example of FIG. 21, the estimation device 47 and the learning device 45 are implemented in a cloud or a server. FIG. 21 shows a state in which a user is walking while carrying a mobile terminal 460. The first measuring device 41 is installed in footwear 400 worn by the user. The second measuring device 42 is installed on the user's wrist. For example, the first measuring device 41 and the second measuring device 42 are wirelessly connected to the mobile terminal 460. The mobile terminal 460 is connected to the estimation device 47 implemented in the cloud or a server via a network 490. In addition, a learning device 45 similar to the learning device of the first to third embodiments is implemented in the cloud or a server. For example, during initial setting or when updating software or model parameters, the learning device 45 transmits update data of the model parameters to the first measuring device 41, the second measuring device 42, and the estimation device 47.

[0152] The first measuring device 41 measures sensor data related to foot movement such as acceleration and angular velocity as the user walks. The first encoding unit 416 of the first measuring device 41 inputs the measured sensor data to the first encoding model 451 and encodes it into a first code. The first measuring device 41 transmits the first code obtained by encoding the sensor data to the mobile terminal 460. The first code transmitted from the first measuring device 41 is transmitted to the estimation device 47 via the mobile terminal 460 carried by the user and the network 490. When the first measuring device 41 obtains update data for the model parameters of the first encoding model 451 from the learning device 45, it updates the model parameters of the first encoding model 451.

[0153] The second measuring device 42 measures sensor data corresponding to physical activity such as acceleration, angular velocity, pulse, and body temperature as the user walks. The second encoding unit 426 of the second measuring device 42 inputs the measured sensor data to the second encoding model 452 and encodes it into a second code. The second measuring device 42 transmits the second code obtained by encoding the sensor data to the mobile terminal 460. The second code transmitted from the second measuring device 42 is transmitted to the estimation device 47 via the mobile terminal 460 carried by the user and the network 490. When the second measuring device 42 acquires update data for the model parameters of the second encoding model 452 from the learning device 45, it updates the model parameters of the second encoding model 452.

[0154] The estimation device 47 receives the first code from the first measuring device 41 via the network 490. The estimation device 47 also receives the second code from the second measuring device 42 via the network 490. The estimation unit 473 of the estimation device 47 inputs the received first code and second code to the estimation model 453. The estimation model 453 outputs an estimation value according to the input of the first code and the second code. The estimation unit 473 outputs the estimation result output from the estimation model 453. For example, the estimation result output from the estimation device 47 is transmitted to the mobile terminal 460 carried by the user via the network 490. When the estimation device 47 acquires update data for the model parameters of the estimation model 453 from the learning device 45, the estimation device 47 updates the model parameters of the estimation model 453.

[0155] FIG. 22 is an example in which information on the estimation result by the estimation device 47 is displayed on the screen of the mobile terminal 460 carried by the user. In the example of FIG. 22, the estimation results of the walking score and the calories burned are displayed on the screen of the mobile terminal 460. In addition, in the example of FIG. 22, an evaluation result according to the estimation result by the estimation device 47, such as "Your physical condition is good," is displayed on the screen of the mobile terminal 460. Furthermore, in the example of FIG. 22, recommendation information according to the estimation result by the estimation device 47, such as "It is recommended that you take a break for about 10 minutes," is displayed on the screen of the mobile terminal 460. A user who views the screen of the mobile terminal 460 can recognize the walking score related to his / her gait and the calories burned according to his / her physical activity. Furthermore, a user who views the screen of the mobile terminal 460 can recognize the evaluation result and recommendation information according to the estimation result of his / her physical condition. Information such as the estimation result by the estimation device 47 and the evaluation result and recommendation information according to the estimation result may be displayed on a screen that can be viewed by the user. For example, the information may be displayed on the screen of a stationary personal computer or a dedicated terminal. Furthermore, the information may be not text information but images representing the information. Furthermore, the information may be notified by a preset pattern such as sound or vibration.

[0156] (operation) Next, the operation of the estimation system 40 of this embodiment will be described with reference to the drawings. In the following, the operations of the first measuring device 41, the second measuring device 42, and the estimation device 47 will be described individually.

[0157] [First measuring device] Fig. 23 is a flowchart for explaining an example of the operation of the first measuring equipment 41. In the explanation following the flowchart of Fig. 23, the first measuring equipment 41 will be described as the subject of the operation.

[0158] 23, first, the first measuring device 41 measures a physical quantity related to the movement of the foot (step S411). For example, the physical quantity related to the movement of the foot is an acceleration in three axial directions or an angular velocity around three axes.

[0159] Next, the first measuring device 41 converts the measured physical quantity into digital data (sensor data) (step S412).

[0160] Next, the first measuring device 41 inputs the sensor data (first raw data) to the first encoding model 451 to calculate the first code (step S413).

[0161] Next, the first measuring device 41 transmits the calculated first code to the estimation device 47 (step S414).

[0162] If the measurement is to be stopped (Yes in step S415), the process according to the flowchart in Fig. 23 ends. The measurement may be stopped at a preset timing, or may be stopped in response to an operation by the user. If the measurement is not to be stopped (No in step S415), the process returns to step S411.

[0163] When the first measuring device 41 receives the update data, it updates the model parameters of the first encoding model 451. The model parameters of the first encoding model 451 are updated at a timing set in advance or at a timing according to a request from a user.

[0164] [Second measuring device] Fig. 24 is a flowchart for explaining an example of the operation of the second measuring device 42. In the explanation following the flowchart of Fig. 24, the second measuring device 42 will be described as the subject of the operation.

[0165] 24, first, the second measuring device 42 measures physical quantities / biological data corresponding to physical activity (step S421). For example, the physical quantities corresponding to physical activity are acceleration in three axial directions and angular velocity around three axes. For example, the biological data corresponding to physical activity are pulse data and body temperature data.

[0166] Next, the second measuring device 42 converts the measured physical quantity / biological data into digital data (sensor data) (step S422).

[0167] Next, the second measuring device 42 inputs the sensor data (second raw data) to the second encoding model 452 to calculate the second code (step S423).

[0168] Next, the second measuring device 42 transmits the calculated second code to the estimation device 47 (step S424).

[0169] If the measurement is to be stopped (Yes in step S425), the process according to the flowchart in Fig. 24 ends. The measurement may be stopped at a preset timing, or may be stopped in response to an operation by the user. If the measurement is not to be stopped (No in step S425), the process returns to step S411.

[0170] When the second measuring device 42 receives the update data, it updates the model parameters of the second encoding model 452. The model parameters of the second encoding model 452 are updated at a timing set in advance or at a timing according to a request from a user.

[0171] [Estimation device] Fig. 25 is a flowchart for explaining an example of the operation of the estimating device 47. In the explanation following the flowchart of Fig. 25, the estimating device 47 will be described as the subject of the operation.

[0172] In FIG. 25, first, the estimation device 47 receives the first code and the second code from the first measuring device 41 and the second measuring device 42, respectively (step S471).

[0173] Next, the estimation device 47 inputs the first code and the second code to the estimation model 453 and calculates an estimation result (step S472).

[0174] Next, the estimation device 47 outputs the calculated estimation result (step S473).

[0175] If the estimation is to be stopped (Yes in step S474), the process according to the flowchart in Fig. 25 ends. The estimation may be stopped at a preset timing, or may be stopped in response to an operation by the user. If the estimation is not to be stopped (No in step S474), the process returns to step S471.

[0176] When the estimation device 47 receives the update data, it updates the model parameters of the estimation model 453. The model parameters of the estimation model 453 are updated at a preset timing or at a timing according to a request from a user.

[0177] As described above, the estimation system of this embodiment includes a first measurement device, a second measurement device, and an estimation device. The first measurement device includes at least one first sensor. The first measurement device inputs first sensor data measured by the first sensor to a first encoding model. The first measurement device transmits a first code output from the first encoding model in response to the input of the first sensor data. The second measurement device includes at least one second sensor. The second measurement device inputs second sensor data measured by the second sensor to a second encoding model. The second measurement device transmits a second code output from the second encoding model in response to the input of the second sensor data. The estimation device includes an estimation model. The estimation device receives the first code transmitted from the first measurement device and the second code transmitted from the second measurement device. The estimation device inputs the received first code and second code to the estimation model. The estimation device outputs an estimation result output from the estimation model in response to the input of the first code and the second code.

[0178] The estimation system of this embodiment includes a first coding model, a second coding model, and an estimation model constructed by the learning device of the first to third embodiments. According to this embodiment, the code coded by the first coding model and the second coding model is communicated, so that the amount of data in communication can be reduced. That is, according to this embodiment, the redundancy of the code derived from the sensor data measured by a plurality of measuring instruments is eliminated, so that the communication capacity between the first measuring device, the second measuring device, and the estimation device can be reduced.

[0179] In one aspect of the present embodiment, the first measuring device and the second measuring device are attached to different body parts of a user whose physical state is to be estimated. According to this aspect, redundancy of sensor data measured by the first measuring device and the second measuring device attached to different body parts, such as a foot and a wrist, can be eliminated, and the dimensionality of the sensor data can be efficiently reduced.

[0180] In one aspect of the present embodiment, the first measuring device and the second measuring device are attached to a pair of body parts of a user whose physical state is to be estimated. According to this aspect, redundancy of sensor data measured by the first measuring device and the second measuring device attached to a pair of body parts such as the left and right feet or wrists can be eliminated, and the dimensionality of the sensor data can be efficiently reduced.

[0181] In one aspect of this embodiment, the estimation device transmits information about the estimation result to a terminal device having a screen that can be viewed by a user. For example, the information about the estimation result transmitted to a mobile device is displayed on the screen of the mobile terminal. A user who views the information about the estimation result displayed on the screen of the mobile terminal can recognize the estimation result.

[0182] In this embodiment, an example is given in which an encoding model is implemented in each of two measurement devices. The encoding model may be implemented in either one of the two measurement devices. It is difficult to change the internal algorithm of a general-purpose measurement device (assumed to be the second measurement device). Therefore, it is sufficient to train the first encoding model included in the first measurement device using the method of the first embodiment so that data of the general-purpose second measurement device cannot be estimated from the first measurement device whose internal algorithm can be changed. Also, in this embodiment, an example is given in which the estimation system includes two measurement devices. The estimation system of this embodiment may include three or more measurement devices.

[0183] In the present embodiment, an example is given in which the first measuring device 41 is installed on the foot and the second measuring device 42 is installed on the wrist. In such a case, the foot corresponds to the first part and the wrist corresponds to the second part. For example, the first measuring device 41 may be installed on the right foot and the second measuring device 42 may be installed on the left foot. In such a case, one of the right foot and the left foot corresponds to the first part and the other corresponds to the second part. For example, the first measuring device 41 may be installed on the right wrist and the second measuring device 42 may be installed on the left wrist. In such a case, one of the right wrist and the left wrist corresponds to the first part and the other corresponds to the second part. Note that the attachment parts of the first measuring device 41 and the second measuring device 42 are not limited to the foot or other parts. The first measuring device 41 and the second measuring device 42 may be attached to the body part to be measured.

[0184] (Hardware) Here, a hardware configuration for executing the processing of the learning device or estimation device according to each embodiment of the present disclosure will be described using an information processing device 90 in Fig. 26 as an example. Note that the information processing device 90 in Fig. 26 is an example of a configuration for executing the processing of the learning device or estimation device according to each embodiment, and does not limit the scope of the present disclosure.

[0185] As shown in Fig. 26, an information processing device 90 includes a processor 91, a main storage device 92, an auxiliary storage device 93, an input / output interface 95, and a communication interface 96. In Fig. 26, the interface is abbreviated as I / F (Interface). The processor 91, the main storage device 92, the auxiliary storage device 93, the input / output interface 95, and the communication interface 96 are connected to each other via a bus 98 so as to be able to communicate data with each other. In addition, the processor 91, the main storage device 92, the auxiliary storage device 93, and the input / output interface 95 are connected to a network such as the Internet or an intranet via the communication interface 96.

[0186] The processor 91 loads a program stored in the auxiliary storage device 93 or the like into the main storage device 92. The processor 91 executes the program loaded into the main storage device 92. In this embodiment, a software program installed in the information processing device 90 may be used. The processor 91 executes processing by the learning device and the estimation device according to this embodiment.

[0187] The main memory device 92 has an area in which a program is loaded. The processor 91 loads a program stored in the auxiliary memory device 93 or the like in the main memory device 92. The main memory device 92 is realized by a volatile memory such as a dynamic random access memory (DRAM). Furthermore, a non-volatile memory such as a magnetoresistive random access memory (MRAM) may be configured / added to the main memory device 92.

[0188] The auxiliary storage device 93 stores various data such as programs. The auxiliary storage device 93 is realized by a local disk such as a hard disk or a flash memory. Note that it is also possible to configure the main storage device 92 to store various data and omit the auxiliary storage device 93.

[0189] The input / output interface 95 is an interface for connecting the information processing device 90 to peripheral devices based on standards and specifications. The communication interface 96 is an interface for connecting to external systems and devices through a network such as the Internet or an intranet based on standards and specifications. The input / output interface 95 and the communication interface 96 may be a common interface for connecting to external devices.

[0190] Input devices such as a keyboard, a mouse, and a touch panel may be connected to the information processing device 90 as necessary. These input devices are used to input information and settings. When a touch panel is used as an input device, the display screen of the display device may also serve as an interface for the input device. Data communication between the processor 91 and the input devices may be mediated by an input / output interface 95.

[0191] The information processing device 90 may be equipped with a display device for displaying information. When the display device is equipped, the information processing device 90 is preferably equipped with a display control device (not shown) for controlling the display of the display device. The display device may be connected to the information processing device 90 via the input / output interface 95.

[0192] The information processing device 90 may also be provided with a drive device. The drive device mediates between the processor 91 and a recording medium (program recording medium) for reading data and programs from the recording medium, writing the processing results of the information processing device 90 to the recording medium, and the like. The drive device may be connected to the information processing device 90 via an input / output interface 95.

[0193] The above is an example of a hardware configuration for enabling the learning device or the estimation device according to each embodiment of the present invention. The hardware configuration in FIG. 26 is an example of a hardware configuration for executing the arithmetic processing of the learning device or the estimation device according to each embodiment, and does not limit the scope of the present invention. In addition, a program for causing a computer to execute processing related to the learning device or the estimation device according to each embodiment is also included in the scope of the present invention. Furthermore, a program recording medium on which the program according to each embodiment is recorded is also included in the scope of the present invention. The recording medium can be realized, for example, by an optical recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). The recording medium may be realized by a semiconductor recording medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card. In addition, the recording medium may be realized by a magnetic recording medium such as a flexible disk or other recording medium. When the program executed by the processor is recorded on a recording medium, the recording medium corresponds to a program recording medium.

[0194] The components of the learning device and the estimation device of each embodiment may be combined in any manner. Furthermore, the components of the learning device and the estimation device of each embodiment may be realized by software or a circuit.

[0195] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0196] 10, 20, 30, 45 Learning Device 11, 21, 31 Acquisition part 12, 22, 32 encoder 13, 23, 33 Estimation part 14, 24, 34 Adversarial Estimation 15, 25, 35 Learning processing unit 17 Database 40 Estimation System 41 First Measurement Device 42 Second measuring device 47 Estimation device 111 First Measurement Device 112 Second Measurement Device 121, 221 First encoding section 122, 222 Second encoding section 151, 251, 451 First coding model 152, 252, 452 Second encoding model 153, 253, 453 Estimation Model 154 Adversarial Inference Model 241 First Adversarial Estimation Division 242 Second Adversarial Estimation Unit 254 First Adversarial Estimation Model 255 Second Adversarial Estimation Model 410, 420 Sensor 411, 421 Acceleration sensor 412, 422 Angular velocity sensor 415, 425 Control section 416 1st encoding section 417, 427 Transmitter 423 Pulse Sensor 424 Temperature Sensor 426 Second encoding section 471 Receiving unit 473 Estimation Department 475 Output section

Claims

1. an acquisition means for acquiring a training data set including first sensor data measured by a first measuring device, second sensor data measured by a second measuring device, and correct answer data; encoding means for encoding the first sensor data into a first code using a first encoding model and for encoding the second sensor data into a second code using a second encoding model; an estimation means for inputting the first code and the second code to an estimation model and outputting an estimation result output from the estimation model; an adversarial estimation means for estimating an estimate of the second code by inputting the first code to a first adversarial estimation model that outputs an estimate of the second code in response to an input of the first code; a learning processing means for training the first encoding model, the second encoding model, the estimation model, and the first adversarial estimation model by machine learning; The learning processing means training the first encoding model, the second encoding model, and the estimation model so that an estimation result of the estimation model matches the ground truth data; training the first adversarial estimation model such that an estimate of the second code by the first adversarial estimation model matches the second code output from the second encoding model; A learning device that trains the first encoding model so that an estimate of the second code by the first adversarial estimation model does not match the second code output from the second encoding model.

2. The learning processing means training the first encoding model, the second encoding model, and the estimation model so that an error between an estimation result of the estimation model and the correct answer data is reduced; training the first adversarial estimation model so that an error between an estimate of the second code by the first adversarial estimation model and the second code output from the second encoding model is reduced; The learning device according to claim 1 , wherein the first encoding model is trained so that an error between an estimated value of the second code by the first adversarial estimation model and the second code output from the second encoding model becomes large.

3. The adversarial estimation means includes: inputting the second code into a second adversarial estimation model that outputs an estimate of the first code in response to an input of the second code, thereby estimating an estimate of the first code; The learning processing means training the second adversarial estimation model such that an estimate of the first code by the second adversarial estimation model matches the first code output from the first encoding model; The learning device according to claim 1 or 2, wherein the second encoding model is trained such that an estimate of the first code by the second adversarial estimation model does not match the first code output from the first encoding model.

4. The learning processing means training the second adversarial estimation model so that an error between an estimate of the first code by the second adversarial estimation model and the first code output from the first encoding model is reduced; The learning device according to claim 3 , wherein the second encoding model is trained so that an error between an estimate of the first code by the second adversarial estimation model and the first code output from the first encoding model becomes large.

5. An estimation system in which a first encoding model, a second encoding model, and an estimation model constructed by the learning device according to any one of claims 1 to 4 are implemented, a first measurement device including at least one first sensor, inputting first sensor data measured by the first sensor to the first encoding model, and transmitting a first code output from the first encoding model in response to the input of the first sensor data; a second measurement device including at least one second sensor, inputting second sensor data measured by the second sensor to the second encoding model, and transmitting a second code output from the second encoding model in response to the input of the second sensor data; an estimation device that includes the estimation model, receives the first code transmitted from the first measuring device and the second code transmitted from the second measuring device, inputs the received first code and second code to the estimation model, and outputs an estimation result output from the estimation model in response to the input of the first code and the second code.

6. The estimation system according to claim 5 , wherein the first measuring device and the second measuring device are attached to different body parts of a user who is the subject of physical state estimation.

7. The estimation system according to claim 5 , wherein the first measuring device and the second measuring device are attached to a pair of body parts of a user who is the subject of physical state estimation.

8. The estimation device includes: The estimation system according to claim 6 or 7, wherein information relating to the estimation result is transmitted to a terminal device having a screen viewable by the user.

9. The computer Obtaining a training data set including first sensor data measured by a first measuring device, second sensor data measured by a second measuring device, and ground truth data; encoding the first sensor data into a first code using a first encoding model and encoding the second sensor data into a second code using a second encoding model; inputting the first code and the second code into an estimation model, and outputting an estimation result output from the estimation model; inputting the first code into a first adversarial estimation model that outputs an estimate of the second code in response to an input of the first code, thereby estimating an estimate of the second code; training the first encoding model, the second encoding model, and the estimation model so that an estimation result of the estimation model matches the ground truth data; training the first adversarial estimation model such that an estimate of the second code by the first adversarial estimation model matches the second code output from the second encoding model; A learning method for training the first encoding model such that an estimate of the second code by the first adversarial estimation model does not match the second code output from the second encoding model.

10. A process of acquiring a training data set including first sensor data measured by a first measuring device, second sensor data measured by a second measuring device, and ground truth data; encoding the first sensor data into a first code using a first encoding model and encoding the second sensor data into a second code using a second encoding model; a process of inputting the first code and the second code into an estimation model and outputting an estimation result output from the estimation model; A process of inputting the first code to a first adversarial estimation model that outputs an estimate of the second code in response to an input of the first code, thereby estimating an estimate of the second code; a process of training the first encoding model, the second encoding model, and the estimation model so that an estimation result of the estimation model matches the ground truth data; training the first adversarial estimation model such that an estimate of the second code by the first adversarial estimation model matches the second code output from the second encoding model; and training the first encoding model so that an estimate of the second code by the first adversarial estimation model does not match the second code output from the second encoding model.

Images