Learning device, estimation device, learning method, estimation method, and program

Abstract

To provide a learning device that suppresses deterioration in accuracy of operation estimation caused by lack of information on an operation entity.SOLUTION: A learning device includes a control unit that executes a first mathematical model that estimates an operation entity on the basis of operation data that is a time series of quantity indicating operation. The control unit updates estimation accuracy of the first mathematical model by learning including execution of operation information amount attenuation processing, which is processing to obtain time series using operation data as an execution target, processing that attenuates an amount of information about a type of operation from the execution target, and processing where the amount of information attenuation of the type of operation possessed by the execution target is greater than the amount of information attenuated by the execution target, that is, the amount of information attenuated by the operation entity. The control unit updates the first mathematical model so as to increase similarity between an amount of feature of the operation data used for learning and an amount of feature of time series obtained by executing operation information attenuation processing on the operation data.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a learning device, an estimation device, a learning method, an estimation method, and a program.

Background Art

[0002] In recent years, with the spread of sensor devices such as motion capture devices and wearable devices, and the improvement of image recognition technology, it has become easier to sense the movements of people. As an example of the motion data obtained by sensing, there are data recorded by acceleration sensors mounted on wearable devices or smartphones, and posture data of people in camera images. The movements of people can be recognized using these data. Such recognition processing can be applied to various services such as person authentication, life log applications, and automatic monitoring systems. Note that motion data is a time series of quantities indicating motion.

[0003] Recognition of human movements using sensed data is often realized by a prediction model using machine learning. As a problem in this case, there is a point that the recognition accuracy for data other than the people used for learning decreases. This is because the movements of each person have individuality. For example, even considering the movement of walking, there is diversity in the walking speed, the magnitude of arm swinging, etc., and the recognition accuracy of the movement decreases in a prediction model trained with another person.

[0004] To solve such problems, various techniques have been disclosed. For example, Non-Patent Document 1 discloses a method of training a prediction model using data of a target person. Also, Patent Document 1 discloses a method of detecting a person similar to a target person from a large number of accumulated human motion data and training a prediction model using the motion data of that person. Further, Non-Patent Documents 2 and 3 disclose methods of adapting a prediction model to people other than those included in the learning examples using active learning, transfer learning, etc.

Prior Art Documents

Patent Document

[0005]

Patent Document 1

Non - Patent Document

[0006]

Non - Patent Document 1

Non - Patent Document 2

Non - Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0007] In order to achieve highly accurate motion recognition, various technologies have been proposed. However, these technologies have a problem that the cost of creating a training dataset for utilization is extremely high.

[0008] First, in the methods described in Non-Patent Document 1, Non-Patent Document 2, and Non-Patent Document 3, it is necessary to acquire data from the target person and further assign a label indicating which action each data corresponds to. In addition, since a model trained with a specific person does not generalize to other people, when the target person changes, it is necessary to acquire data and perform labeling again.

[0009] Also, regarding the method described in Patent Document 1 of using data of similar people from among motion data of a large number of people, it is necessary to acquire data and labels for all actions that one wants to recognize for each person, which is very costly.

[0010] Furthermore, when creating training data, there are cases where a person label indicating which person each motion data belongs to cannot be obtained due to issues such as privacy. In such cases, since any of the above technologies has a technical configuration premised on being able to identify a person, there is a problem that they cannot be used.

[0011] As outlined above, under realistic conditions considering issues such as privacy, since it may be difficult to obtain information on the subject of motion, the fact is that no technology has been proposed to realize human motion recognition while suppressing the cost of creating training data.

[0012] Moreover, such circumstances were common even when the subject of motion, which is the actor, is not a person such as an animal or a robot. Therefore, in the learning of a mathematical model for estimating the motion of motion data, there was a problem that the estimation accuracy of the mathematical model obtained by learning deteriorates when training data that does not include information indicating the subject of motion is included in the dataset of training data.

[0013] In view of the above circumstances, an object of the present invention is to provide a technique for suppressing deterioration in the accuracy of estimating an operation caused by lack of information on the subject of the operation.

Means for Solving the Problems

[0014] One aspect of the present invention includes a control unit that executes a first mathematical model for estimating an operation subject based on operation data that is a time series of an amount indicating an operation. The control unit updates the accuracy of the estimation of the first mathematical model by learning that includes execution of an operation information amount attenuation process, which is a process of obtaining a time series with the operation data as an execution target, a process of attenuating the information amount of the type of operation from the execution target, and a process in which the amount of attenuation of the information amount of the type of operation possessed by the execution target is larger than the amount of attenuation of the information amount of the operation subject that is the subject of the operation and is the information amount possessed by the execution target. The control unit updates the first mathematical model so as to increase the similarity between the feature amount of the operation data used in the learning and the feature amount of the time series obtained by executing the operation information amount attenuation process with the operation data as the execution target. The control unit is a learning device that updates the first mathematical model so that the similarity between the feature amount of the operation data and the feature amount of the time series obtained by executing the operation information amount attenuation process with other operation data different from the operation data as the execution target does not increase.

[0015] One aspect of the present invention includes a control unit that executes a first mathematical model for estimating an operating entity based on operation data that is a time series of an amount indicating an operation. The control unit updates the accuracy of the estimation of the first mathematical model by learning that includes executing an operation information amount attenuation process, which is a process of obtaining a time series with the operation data as an execution target, a process of attenuating the amount of information on the type of operation from the execution target, and a process in which the amount of attenuation of the amount of information on the type of operation possessed by the execution target is larger than the amount of attenuation of the amount of information on the operating entity that is the subject of the operation and is the amount of information possessed by the execution target. The control unit updates the first mathematical model so as to increase the similarity between the feature amount of the operation data used in the learning and the feature amount of the time series obtained by executing the operation information amount attenuation process with the operation data as the execution target. The control unit updates the first mathematical model so that the similarity between the feature amount of the operation data and the feature amount of the time series obtained by executing the operation information amount attenuation process with other operation data different from the operation data as the execution target does not increase. An estimation device includes an estimation unit that executes a learned second mathematical model obtained by a learning device that updates a second mathematical model for estimating an operation based on operation data using the estimation result of the first mathematical model.

[0016] One aspect of the present invention includes a control unit that executes a first mathematical model for estimating an operating entity based on operation data that is a time series of an amount indicating an operation. The control unit updates the accuracy of the estimation of the first mathematical model through learning that includes execution of an operation information amount attenuation process, which is a process of obtaining a time series with the operation data as an execution target, a process of attenuating the amount of information on the type of operation from the execution target, and a process in which the amount of attenuation of the amount of information on the type of operation possessed by the execution target is larger than the amount of attenuation of the amount of information on the operating entity that is the subject of the operation and is the amount of information possessed by the execution target. The control unit updates the first mathematical model so as to increase the similarity between the feature amount of the operation data used in the learning and the feature amount of the time series obtained by executing the operation information amount attenuation process with the operation data as the execution target. The control unit updates the first mathematical model so that the similarity between the feature amount of the operation data and the feature amount of the time series obtained by executing the operation information amount attenuation process with other operation data different from the operation data as the execution target does not increase. An estimation device includes an estimation unit that executes a sixth mathematical model for estimating the operation indicated by the estimation target based on the estimation result of the learned first mathematical model obtained by the learning device and the operation data of the estimation target.

[0017] One aspect of the present invention has a control step in which a computer executes a first mathematical model for estimating an operation subject based on operation data that is a time series of an amount indicating an operation. The control step includes updating the accuracy of the estimation of the first mathematical model by learning that includes executing an operation information amount attenuation process, which is a process of obtaining a time series with the operation data as an execution target, a process of attenuating the amount of information on the type of operation from the execution target, and a process in which the amount of attenuation of the amount of information on the type of operation possessed by the execution target is larger than the amount of attenuation of the amount of information on the operation subject that is the operation subject possessing the execution target. The control step updates the first mathematical model so as to increase the similarity between the feature amount of the operation data used in the learning and the feature amount of the time series obtained by executing the operation information amount attenuation process with the operation data as the execution target. The control step updates the first mathematical model so that the similarity between the feature amount of the operation data and the feature amount of the time series obtained by executing the operation information amount attenuation process with other operation data different from the operation data as the execution target does not increase. This is a learning method.

[0018] One aspect of the present invention includes a control unit that executes a first mathematical model for estimating an operating entity based on operation data that is a time series of an amount indicating an operation. The control unit updates the accuracy of the estimation of the first mathematical model through learning that includes executing an operation information amount attenuation process, which is a process of obtaining a time series with the operation data as an execution target, a process of attenuating the amount of information on the type of operation from the execution target, and a process in which the amount of attenuation of the amount of information on the type of operation possessed by the execution target is greater than the amount of attenuation of the amount of information on the operating entity that is the subject of the operation and is the amount of information possessed by the execution target. The control unit updates the first mathematical model so as to increase the similarity between the feature amount of the operation data used in the learning and the feature amount of the time series obtained by executing the operation information amount attenuation process with the operation data as the execution target. The control unit updates the first mathematical model so that the similarity between the feature amount of the operation data and the feature amount of the time series obtained by executing the operation information amount attenuation process with other operation data different from the operation data as the execution target does not increase. The estimation method includes an estimation step of executing a learned second mathematical model obtained by a learning device that updates a second mathematical model for estimating an operation based on operation data using the estimation result of the first mathematical model.

[0019] One aspect of the present invention includes a control unit that executes a first mathematical model for estimating an operating entity based on operation data that is a time series of an amount indicating an operation. The control unit updates the accuracy of the estimation of the first mathematical model through learning that includes execution of an operation information amount attenuation process, which is a process of obtaining a time series with the operation data as an execution target, a process of attenuating the amount of information on the type of operation from the execution target, and a process in which the amount of attenuation of the amount of information on the type of operation possessed by the execution target is larger than the amount of attenuation of the amount of information on the operating entity that is the subject of the operation and is the amount of information possessed by the execution target. The control unit updates the first mathematical model so as to increase the similarity between the feature amount of the operation data used in the learning and the feature amount of the time series obtained by executing the operation information amount attenuation process with the operation data as the execution target. The control unit updates the first mathematical model so that the similarity between the feature amount of the operation data and the feature amount of the time series obtained by executing the operation information amount attenuation process with other operation data different from the operation data as the execution target does not increase. An estimation unit that executes a sixth mathematical model for estimating the operation indicated by the estimation target based on the estimation result of the learned first mathematical model obtained by a learning device and the operation data of the estimation target.

[0020] One aspect of the present invention is a program for causing a computer to function as either the above learning device or the above estimation device.

Advantages of the Invention

[0021] According to the present invention, it is possible to suppress deterioration in the accuracy of estimating an operation caused by lack of information on the operating entity.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0023] (Embodiment) FIG. 1 is an explanatory diagram for explaining the outline of the estimation system 100 of the embodiment. The estimation system 100 includes a learning device 1 and an estimation device 2. The learning device 1 learns the accuracy of the estimation of a mathematical model (hereinafter referred to as the "main operation estimation model") that estimates the operation indicated by the operation data based on the operation data, and updates it until a predetermined end condition regarding the update is satisfied. The operation data is used for learning. Note that updating the mathematical model means updating the value of the parameter of the mathematical model. The value of the parameter of the mathematical model is stored in a predetermined storage device. Hereinafter, the predetermined end condition regarding the update of the mathematical model is referred to as the learning end condition.

[0024] The learning end condition may be, for example, a condition that a predetermined number of updates have been performed, or may be, for example, a condition that the degree of change in the mathematical model to be updated is within a predetermined degree due to the update. The mathematical model at the time when the learning end condition is satisfied is a learned mathematical model. Therefore, the main operation estimation model at the time when the learning end condition is satisfied is a learned main operation estimation model. Hereinafter, the learned main operation estimation model will be referred to as the learned main operation estimation model. The estimation device 2 estimates the operation indicated by the input operation data using the learned main operation estimation model.

[0025] The operation data is a time series of quantities indicating the operation. The operation is, for example, the same type of operation such as walking, and the details depend on the subject of the operation (hereinafter referred to as the "operation subject"). Therefore, the operation data includes information indicating the type of operation and information indicating the operation subject.

[0026] The operation data is, for example, the result of sensing the operation to be sensed by a sensor device such as a motion capture device or a wearable device. In such a case, the quantity indicating the operation is, for example, a tensor indicating the position coordinates of each part of the measurement target.

[0027] The operation data may be, for example, a time series of measurement results by an acceleration sensor mounted on a device such as a wearable device or a smartphone. In such a case, the quantity indicating the operation is, for example, the acceleration at each timing measured by the acceleration sensor.

[0028] The operation data may be, for example, the posture data of the operation subject in the camera video. Note that the operation subject means the subject of the operation. The operation subject may be, for example, a person, an animal, or a robot.

[0029] The estimation device 2 estimates the operation of the input operation data using the learned main operation estimation model obtained by the learning device 1.

[0030] First, the learning device 1 will be described in more detail, and then the estimation device 2 will be described in more detail.

[0031] <<Learning Device 1>> In the learning of the active operation estimation model by the learning device 1, a dataset of training data (hereinafter referred to as the "training dataset") is used. The training dataset includes supervised operation data. The supervised operation data is training data that includes operation data and an operation class label. That is, the operation class label in the supervised operation data is the correct data for the operation data included in the supervised operation data.

[0032] The operation class label is information indicating whether the operation indicated by the operation data belongs to any one of one or more predetermined types, and if so, which type of operation it is. For the sake of easy understanding of the following description, the term "operation class" is defined here.

[0033] <Definition of Operation Class> The operation class is one type of set, and there are at least two of them. Each set is either an unknown operation class or a known operation class. The known operation class is a set of operation data that satisfies the condition of making the types of operations the same, and the type of operation is a predetermined operation. On the other hand, the unknown operation class is a set of operation data that does not belong to the known operation class. That is, it is a set of operation data indicating an operation that is not predetermined. Therefore, the unknown operation class is the complement of the set of known operation classes. Using the expression "operation class", the operation class label is information indicating the operation class.

[0034] The training dataset may include unsupervised operation data. The unsupervised operation data is training data that does not include an operation class label.

[0035] The training data included in the set of teacher-assisted motion data may be training data that includes a motion subject label or may be training data that does not include a motion subject label. A motion subject label is information indicating a motion subject.

[0036] The training data included in the set of unsupervised motion data may be training data that includes a motion subject label or may be training data that does not include a motion subject label.

[0037] FIG. 2 is an explanatory diagram for explaining an overview of the operation of the learning device 1 in an embodiment. The learning device 1 performs a main motion subject estimation model learning process. The main motion subject estimation model learning process is a process for learning a main motion subject estimation model. The main motion subject estimation model is a mathematical model that estimates a motion subject based on motion data. In the learning of the main motion subject estimation model, training data including motion data is used as the training data.

[0038] Specifically, estimating a motion subject means estimating the probability that motion data belongs to each of a predetermined set of motion subject classes. A motion subject class is a set of motion data that satisfies the condition of making the motion subjects identical. Therefore, estimating the probability that motion data belongs to each of the predetermined motion subject classes means estimating the probability that the motion subject of the motion data is among the predetermined candidates for the motion subject. Hereinafter, the probability that motion data belongs to each motion subject class is referred to as the motion subject class membership probability.

[0039] <More detailed explanation about the motion subject label> Here, a more detailed explanation of the motion subject label will be given. The motion subject label may indicate the motion subject in any form as long as it can indicate the motion subject. For example, it may indicate the motion subject class membership probability. The motion subject label may indicate only one motion subject, which is equivalent to indicating that the motion subject class membership probability for the indicated motion subject is 100%. Therefore, the motion subject label is information indicating the motion subject class membership probability.

[0040] <Regarding the learning process of the main action subject estimation model> The learning process of the main action subject estimation model will be described. In the learning process of the main action subject estimation model, the first feature quantity estimation model is executed. The first feature quantity estimation model is a mathematical model that acquires the feature quantities of the time series of the execution target.

[0041] In the learning process of the main action subject estimation model, the first feature quantity estimation model is executed on the operation data. Also, in the learning process of the main action subject estimation model, it is also executed on the time series obtained as a result of the operation information quantity attenuation process for the operation data.

[0042] The operation information quantity attenuation process is a process of obtaining a time series with the operation data as the execution target. The operation information quantity attenuation process is a process of attenuating the information quantity of the type of operation from the operation data of the execution target, and the amount of attenuation of the information quantity of the type of operation possessed by the operation data of the execution target is larger than the amount of attenuation of the information quantity of the operation subject possessed by the operation data of the execution target.

[0043] Therefore, the information quantity of the type of operation possessed by the time series (hereinafter referred to as "operation information quantity attenuation time series") obtained by executing the operation information quantity attenuation process is less than the information quantity of the type of operation possessed by the operation data that is the execution target of the operation information quantity attenuation process. Also, the magnitude of the operation information quantity difference is smaller than the magnitude of the subject information quantity difference.

[0044] The operation information quantity difference is the difference between the information quantity of the type of operation possessed by the operation information quantity attenuation time series and the information quantity of the type of operation possessed by the operation data that is the execution target of the operation information quantity attenuation process. The subject information quantity difference is the difference between the information quantity of the operation subject possessed by the operation information quantity attenuation time series and the information quantity of the operation subject possessed by the operation data that is the execution target of the operation information quantity attenuation process.

[0045] Hereinafter, the feature amount obtained by executing the first feature amount estimation model for the operation data is referred to as the first type of first feature amount. Hereinafter, the feature amount obtained by executing the first feature amount estimation model for the time series obtained as a result of the operation information amount attenuation process for the operation data is referred to as the second type of first feature amount. Hereinafter, when not distinguishing between the two types of feature amounts, namely the first type of first feature amount and the second type of first feature amount, it is referred to as the first feature amount.

[0046] Thus, in the main operation subject estimation model learning process, the first type of first feature amount and the second type of first feature amount are obtained. In the main operation subject estimation model learning process, based on the similarity between the obtained first type of first feature amount and the second type of first feature amount, the main operation subject estimation model is updated to increase the similarity. The update of the main operation subject estimation model specifically means the update of the first feature amount estimation model.

[0047] Therefore, in the main operation subject estimation model learning process, based on the similarity between the first type of first feature amount and the second type of first feature amount, the first feature amount estimation model is updated to increase the similarity.

[0048] <Regarding the main operation subject estimation model and the first feature amount> More specifically, the first operation subject estimation model is a mathematical model including a first feature amount estimation model and a first secondary operation subject estimation model. The first secondary operation subject estimation model is a mathematical model for estimating the operation subject based on the first type of first feature amount. Here too, estimating the operation subject means estimating the operation subject class membership probability for each of the predetermined operation subject classes. Note that the result of estimating the operation subject by the first operation subject estimation model is specifically the result of the estimation by the first secondary operation subject estimation model.

[0049] The information indicating the probability of each operation subject class estimated by the first secondary operation subject estimation model is an example of an operation subject label. When estimating the operation subject class membership probability for each of the operation subject classes, for example, the operation subject class with the highest probability among the operation subject class membership probabilities estimated by the first secondary operation subject estimation model is the operation subject indicated by the operation subject label.

[0050] Generally, the accuracy of information determination is higher as the amount of information to be determined included in the data used as a basis is larger. Therefore, the accuracy of the estimation by the first sub-operation entity estimation model is higher as the ratio of the amount of information of the operation entity included in the first type of feature amount to the total amount of information of the first type of feature amount is larger. Thus, while explaining the first type of feature amount regarding the fact that the ratio of the amount of information of the operation entity included in the first type of feature amount to the total amount of information of the first type of feature amount becomes higher by the update of the first feature amount estimation model, further explanation will be provided.

[0051] As described above, the first type of feature amount is a feature amount obtained from operation data. Therefore, the first type of feature amount includes information indicating the operation entity. Thus, the first type of feature amount is a quantity having a correlation with the operation entity label indicating the operation entity.

[0052] And as described above, in the main operation entity estimation model learning process, learning is performed so as to increase the similarity between two types of first feature amounts obtained from the same operation data. This is learning to make prominent the amount of information of the operation entity indicated by the first type of feature amount, as will be described in detail later. Therefore, as learning progresses, the ratio of the amount of information indicating the operation entity to the total amount of information of the first type of feature amount increases.

[0053] Thus, performing the first feature amount estimation model learning so that the similarity between two types of first feature amounts obtained from the same operation data increases means increasing the accuracy of the estimation by the main operation entity estimation model. More specifically, increasing the accuracy of the estimation by the main operation entity estimation model means increasing the accuracy with which the main operation entity estimation model estimates the same operation entity as being the same and different operation entities as being different.

[0054] In the main operation entity estimation model learning process, for the first feature amounts obtained from different operation data, learning is performed so that the similarity does not increase.

[0055] Note that the first type of feature amount is represented by, for example, a numerical vector. Therefore, the first type of feature amount is, for example, a positive integer value n 1 determined by n1 It is represented by a vector having dimensional elements. Although the representation of the first feature amount is not limited to a vector, in the following description, for simplicity, the case where the first feature amount is represented by a vector will be taken as an example for explanation. The operation information amount decay time series can be obtained, for example, by executing an operation information amount decay process on the replication of operation data.

[0056] <Example of operation information amount decay process: Phase randomization process> An example of the operation information amount decay process will be described. Since the operation data is time-series data, according to the theory of Fourier transform, the operation data can be expressed as the sum of one or more sine waves (i.e., sine wave components) with different amplitudes, phases, or frequencies. The operation information amount decay process is, for example, a process of randomizing the phase of the sine wave component of the operation data (hereinafter referred to as the "phase randomization process").

[0057] That is, the phase randomization process is an example of a process that attenuates the information amount of the type of operation, and the amount of attenuation of the information amount of the type of operation is larger than the amount of attenuation of the information amount of the operation subject. For the sake of simplicity of explanation, the case where the operation subject is a person will be taken as an example for explanation. First, the process of obtaining the average value of the operation data in the time direction and the process of randomly rearranging the time-series order of the operation frames of the operation data will be described.

[0058] The process of obtaining the average value of the operation data in the time direction and the process of randomly rearranging the time-series order of the operation frames of the operation data lose the information for identifying the operation by losing the time-series information of the operation data. On the other hand, the process of obtaining the average value of the operation data in the time direction and the process of randomly rearranging the time-series order of the operation frames of the operation data preserve the information representing physical characteristics such as a person's height and proportions.

[0059] However, the personal characteristics preserved in the process of obtaining the time-direction average value of motion data or the process of randomly rearranging the time-series order of the motion frames of motion data are limited to static physical characteristics, and dynamic personal characteristics such as the speed of motion are largely lost. Therefore, in learning using the process of obtaining the time-direction average value of motion data or the process of randomly rearranging the time-series order of the motion frames of motion data, there is a risk that person identification with sufficient accuracy cannot be learned.

[0060] On the other hand, when randomizing the phase of the sine wave component of motion data, based on the finding that the mechanical influence caused by the physical differences of people is strongly reflected in the amplitude of motion, dynamic personal characteristics are also retained in addition to static physical characteristics. For example, the skeletal structure of a person, which is deeply related to personal characteristics, can be regarded as a pendulum model in which bones are connected at joint points. It is known that an inverse proportional relationship holds between the length and amplitude of a pendulum in free vibration. From this, the dynamic characteristics resulting from the body size, which is one of the characteristics of a person, are more likely to occur in the amplitude rather than the phase of motion.

[0061] Therefore, in the phase randomization process, by randomizing the phase, the characteristics for identifying motion are lost, and by retaining the amplitude, the dynamic personal characteristics are preserved. In the phase randomization process, static characteristics such as a person's height and proportion are not damaged. From the above, the phase randomization process is a process that preserves both static and dynamic characteristics for identifying a person while losing the characteristics for identifying motion.

[0062] <Mathematical Explanation of Phase Randomization Process> Taking as an example the case of receiving a time series \(x(t)\) of \(m\) dimensions (\(m\) is an integer greater than or equal to 1) as operation data, a mathematical explanation of the phase randomization process will be given. In the phase randomization process, first, a pre-data transformation \(P\) is executed on the time series \(x(t)\). The pre-data transformation \(P\) is a transformation process that enables the execution of the Fourier transform described later. Therefore, the pre-data transformation \(P\) is, for example, a normalization process that enables the execution of the Fourier transform described later. The pre-data transformation \(P\) is, for example, a process of converting to a relative coordinate system from the lumbar joint point when the operation data is a time series recording the time series of joint coordinate points. This is because the raw data obtained by the sensor would be the absolute coordinates of the joint points and the Fourier transform could not be executed. By the pre-data transformation \(P\), the time series \(x(t)\) is converted into the \(m\)-dimensional operation data \(Px(t)\).

[0063] In the phase randomization process, next, for the \(i\)-th dimensional (\(i\) is an integer greater than or equal to 1 and less than or equal to \(m\)) time series \(Px\) i (t) in the operation data \(Px(t)\), a Fourier transform \(F\) is executed. The frequency spectrum \(X\) i (ω) obtained by the Fourier transform of the time series \(Px\) i (t) is represented by the following equation (1). That is, the frequency spectrum \(X\) i (ω) is an equation representing the frequency representation of the time series \(Px\) i (t).

[0064]

Number

[0065] Therefore, the amplitude component \(R\) i (ω) and the phase component \(A\) i (ω) of the time series \(Px\) i (t) are respectively the following equations (2) and (3).

[0066]

Number

[0067]

Number

[0068] absolute(a) and angle(a) represent the absolute value and the argument of the complex number a, respectively. In the phase randomization process, for the phase A i (ω), a random number value α i (ω) generated randomly from the range [-πr, πr] defined by the variable r is added. Hereinafter, the phase A i (ω) to which the random number is added is referred to as the phase A i ’(ω) after the randomization process.

[0069] In the phase randomization process, using the amplitude R i (ω) and the phase A i ’(ω) after the randomization process, the following sequence X’ i (ω) is obtained. Note that j represents the imaginary unit.

[0070]

Equation

[0071] Next, in the phase randomization process, the inverse operation transformation P i of the pre-data transformation P and the inverse Fourier transform F -1 are performed on the frequency spectrum X’ -1 (ω). That is, the process represented by the following equation (5) is performed.

[0072]

Equation

[0073] This is the process of obtaining the time series x i ’(t) whose frequency spectrum is the frequency spectrum X’ i (ω). The time series x i ’(t) is more specifically the time series represented by the following equation (6).

[0074] [Number]

[0075] Time series x i The process of obtaining x’(t) is executed for each dimension i, but it may be obtained by parallel processing or sequentially in the order of dimensions.

[0076] Next, in the phase randomization process, an ordered set defined by the magnitude of dimension i, where each time series x i ’(t) is used as an element, an ordered set x’(t) is obtained. The ordered set x’(t) obtained as a result of the phase randomization process for the operation data x(t) is an example of a time series with decaying operation information amount and an example of an object for obtaining the first feature amount. Thus, the phase randomization process is a process of randomizing the phase of the sine wave component of the operation data.

[0077] Note that the phase randomization process may be any process as long as it randomizes the phase of the sine wave component of the operation data. Therefore, in the phase randomization process, for example, instead of adding a randomly generated random value α i (ω) to the original phase A i (ω), it may be performed to replace the phase A i (ω) with the random value α i (ω).

[0078] Since the phase randomization process is a process of randomizing the phase of the sine wave component of the operation data x(t), hereinafter, the operation data after the operation information amount attenuation process is referred to as a de-randomized time series. That is, the de-randomized time series is a time series in which the phase of the sine wave component of the operation data is randomized. Also, the de-randomized time series is an example of a time series with decaying operation information amount. The de-randomized time series obtained based on the operation data x(t) is, for example, the above-mentioned ordered set x’(t).

[0079] <Regarding the loss and update rules in the main operation entity estimation model learning process> The rules of loss and update in the learning process of the main actor estimation model will be described. As described above, in the learning process of the main actor estimation model, the first feature amount is obtained from each of the motion data and the randomized time series. In the learning process of the main actor estimation model, learning is performed with the similarity between the first feature amounts as the loss. Hereinafter, the similarity between the first feature amounts is referred to as the contrast loss evaluation value.

[0080] The similarity between the first feature amounts indicated by the contrast loss evaluation value is, for example, the similarity between the first type first feature amount and the second type first feature amount obtained from the same motion data. Note that the first feature amounts obtained from the same motion data mean that both of the two first feature amounts to be evaluated for similarity are the first feature amounts obtained from the same motion data.

[0081] For the sake of certainty, specific examples of the first type first feature amount and the second type first feature amount obtained from the same motion data will be described. In this case, if the first type first feature amount is the first type first feature amount obtained from the motion data x(t), the second type first feature amount means the second type first feature amount obtained from the randomized time series x'(t) obtained as a result of the randomization process for the motion data x(t).

[0082] Note that in the learning process of the main actor estimation model, the first type first feature amount and the second type first feature amount are obtained from a plurality of motion data in the training data set. Therefore, when updating the actor estimation model, it is not necessarily the case that only the similarity between the first feature amounts obtained from the same motion data is used, and the similarity between the first type feature amounts obtained from different motion data may be used. However, at least once, the similarity between the first type first feature amount and the second type first feature amount obtained from the same motion data is used.

[0083] In the main action subject estimation model learning process, when the obtained similarity is the similarity between the first feature amounts obtained from the same operation data, the operation subject estimation model is updated so as to increase the similarity. On the other hand, in the main action subject estimation model learning process, when the obtained similarity is the similarity between the first feature amounts obtained from different operation data, the operation subject estimation model is updated so as to decrease the similarity.

[0084] This update rule is a rule that increases the probability that the operation subject estimation model determines that the first feature amounts obtained from the same operation data are the same, and increases the probability that the operation subject estimation model determines that the first feature amounts obtained from different operation data are different. As described above, the first feature amount obtained by the phase randomization process is a process that preserves both static features and dynamic features for identifying the operation subject. Therefore, such an update rule is a rule that improves the accuracy of identifying the operation subject by the operation subject estimation model.

[0085] Whether the obtained similarity is from the same operation data is determined based on whether the identifiers assigned to each data are the same, for example, under the rule that the same identifier is assigned to the same operation data.

[0086] An example of the process of assigning the same identifier to the same operation data (hereinafter referred to as "identifier assignment process") will be described. As an example, an example of the identifier assignment process will be described using the case where a plurality of the same operation data are generated by replicating one operation data. In such a case, the identifier assignment process is a process of adding the same identifier to both the operation data before replication and the operation data generated by replication. Such replication process and identifier assignment process are executed, for example, by the main action estimation model learning unit 111 described later.

[0087] The identifier assignment process may be executed in advance, for example, before the operation data is input to the learning device 1. In such a case, the same identifier has been added to the same operation data before the operation data is input to the learning device 1.

[0088] <Mathematical Explanation of Loss in the Active Agent Estimation Model Learning Process> The active agent estimation model is a function that takes motion data x as input and outputs a vector z, which is the first feature, and is represented by a function with parameters θ z and is represented by a function having.

[0089] For simplicity of explanation, as an example, assume a scenario where a set of motion data with N data points is given for learning the active agent estimation model. Also, for simplicity of the following explanation, assume a scenario where both the first type of first feature and the second type of first feature are represented by vectors.

[0090] In the active agent estimation model learning process, for each element of the given set of motion data, the first type of first feature and the second type of first feature are obtained. That is, in the active agent estimation model learning process, 2N vectors are obtained. In this set of vectors, for a certain vector z j there is another vector z i with the same motion data. And for the vector z j all vectors other than the vector z i are vectors z k generated from different motion data.

[0091] That is, there are 2(N - 1) vectors z k . In the learning of the active agent estimation model, learning is performed so that vectors extracted from the same motion data have a large similarity and vectors generated from different motion data have a small similarity. As a result, learning is performed so that motion data of the same active agent are located in the vicinity in the vector space of z.

[0092] As an example of the loss for realizing such learning, there is the following formula (7) using sim(u, v) for measuring the similarity between vectors.

[0093]

Equation

[0094] Eb[a] is the expected value of a with respect to the probability b. In the main operating entity estimation model learning process, Equation (7) is approximately equal to the equation shown in the following Equation (8), which represents the average of N pieces of operation data. In Equation (8), z i and z i+1 are the first feature quantities extracted from the same piece of operation data. The value of the function L ctr in Equation (8) is an example of the contrast loss evaluation value.

[0095]

Number

[0096] In the main operating entity estimation model learning process, the main operating entity estimation model is updated so that the value of Equation (8) becomes smaller with respect to the parameter θ z . An example of a method for obtaining the value of the parameter θ z will be described. When the function representing the main operating entity estimation model is differentiable with respect to the parameter θ z , it is known that local minimization is possible. Therefore, by executing an algorithm such as the gradient descent method, the value of the parameter θ z that locally minimizes the value of Equation (8) can be obtained.

[0097] By the way, it has been described above that the main operating entity estimation model is a function that takes the operation data x as input and outputs the vector z that is the first feature quantity, and is represented by a function having the parameter θ z . The function representing the main operating entity estimation model may further have the condition of being differentiable with respect to the parameter θ z .

[0098] In the main operating entity estimation model learning process, for example, the main operating entity estimation model is updated so that the contrast loss evaluation value becomes smaller.

[0099] <An example of the first sub-operating entity estimation model> An example of the first secondary action agent estimation model will be described. The first secondary action agent estimation model performs clustering on, for example, a set of first feature quantities. By performing clustering, the action agent class membership probability is obtained. The clustering performed by the first secondary action agent estimation model may be any known clustering. The clustering performed by the first secondary action agent estimation model may be, for example, clustering provided by open source software such as Scikit-learn.

[0100] <Regarding the relationship between the main action estimation model and the main action agent estimation model> FIG. 3 is an explanatory diagram for explaining the relationship between the main action estimation model and the first-order secondary action estimation model in the embodiment.

[0101] The main action agent estimation model is included in the main action estimation model. By the way, if the main action estimation model includes a mathematical model (hereinafter referred to as the "first-order secondary action estimation model") that estimates an action based on action data, it is possible to estimate the action based on the action data. And if the first-order secondary action estimation model is a mathematical model in which learning is performed so that the estimation accuracy is improved by using the estimation result of the main action agent estimation model, the higher the estimation accuracy of the main action agent estimation model, the higher the estimation accuracy of the main action estimation model. Therefore, an example of the first-order secondary action estimation model learning process, which is a process for learning the first-order secondary action estimation model, will be described.

[0102] For the sake of simplicity of explanation, in the following example of the first-order secondary action estimation model learning process, estimating an action means estimating the probability that the action data belongs to each of an unknown action class and one or more predetermined known action classes. Hereinafter, the probability that the action data belongs to each action class will be referred to as the action class membership probability.

[0103] <More detailed explanation about the action class label> Here, the operation class label will be described in more detail. As described above, the operation class label may be information indicating whether the operation indicated by the operation data belongs to any one of one or more predetermined types, and if so, which type of operation it is. Therefore, for example, the operation class label may indicate the operation class attribution probability. The operation class label may be information indicating only one operation class among the unknown operation class and one or more known operation classes. Actually, this is equivalent to indicating that the operation class attribution probability for the indicated operation class is 100%. Therefore, the operation class label is information indicating the operation class attribution probability.

[0104] Therefore, in other examples of the first-order secondary operation estimation model and other examples of the main operation subject estimation model learning process, estimating an operation may be a process of estimating the operation class with the highest operation class attribution probability.

[0105] <An example of the first-order secondary operation estimation model learning process> The learning device 1 performs the first-order secondary operation estimation model learning process. As described above, the first-order secondary operation estimation model learning process is a process of learning the first-order secondary operation estimation model. The first-order secondary operation estimation model is a mathematical model that estimates an operation based on operation data, an operation subject label indicating the operation subject such as the estimation result of the main operation subject estimation model, and an operation class label.

[0106] FIG. 4 is an explanatory diagram for explaining an example of the first-order secondary operation estimation model learning process in the embodiment. The first-order secondary operation estimation model includes a second feature quantity estimation model and a second secondary operation subject estimation model. When learning the first secondary operation estimation model, not only the second feature quantity estimation model and the second-order secondary operation estimation model, but also the second secondary operation subject estimation model is used.

[0107] <Regarding the second feature quantity estimation model and the second secondary operation subject estimation model> The second feature quantity estimation model is a mathematical model that estimates the feature quantity that is the output of the second secondary operation subject estimation model and is the feature quantity of the operation data to be executed. That is, the output destination of the feature quantity estimated by the second feature quantity estimation model is different from the output destination of the feature quantity estimated by the first feature quantity estimation model. Hereinafter, the feature quantity estimated by the second feature quantity estimation model is referred to as the second feature quantity. The second feature quantity may be the same as or different from the first feature quantity.

[0108] The second feature quantity is a feature quantity obtained from the operation data. Therefore, the second feature quantity is represented by a vector having elements of n 2 dimensions. n 2 is an arbitrary positive integer. Although the expression of the second feature quantity is not limited to a vector, in the following description, for simplicity, the case where the second feature quantity is represented by a vector will be taken as an example for explanation.

[0109] The second secondary operation subject estimation model is a mathematical model that estimates the operation subject based on the second feature quantity. Specifically, estimating the operation subject means estimating the operation subject class membership probability for each of the predetermined operation subject classes.

[0110] <Regarding the update rules of the second feature quantity estimation model and the second secondary operation subject estimation model> The second feature quantity estimation model and the second secondary operation subject estimation model are updated by learning through the first secondary operation estimation model learning process. At the time of update, the second secondary operation subject estimation model is updated so as to improve the accuracy of the estimation of the operation subject, and the second feature quantity estimation model is updated so as to reduce the accuracy of the estimation by the second secondary operation subject estimation model.

[0111] More specifically, the second secondary action agent estimation model is updated to increase the similarity between the action agent class attribution probability estimated by the second secondary action agent estimation model and the result of the main action agent estimation model. The second feature quantity estimation model is updated to reduce the similarity between the result of the estimation of the second secondary action agent estimation model based on the second feature quantity estimated by the second feature quantity estimation model and the result of the estimation of the main action agent estimation model. The second feature quantity estimation model updated in this way estimates a second feature quantity with less dependence on the action agent.

[0112] <Regarding the second secondary action estimation model> The second secondary action estimation model estimates an action based on the second feature quantity. Here, estimating an action also means estimating the action class attribution probability for each of the predetermined action classes. The second secondary action estimation model is also updated by learning through the first secondary action estimation model learning process.

[0113] <Regarding the update rule of the second secondary action estimation model> In the learning of the second secondary action estimation model, the second secondary action estimation model is updated so as to reduce the difference between the action class attribution probability estimated by the second secondary action estimation model and the action class label given as the correct data. The action class label of the correct data is the action class label included in the supervised action data when the training data is supervised action data.

[0114] On the other hand, when the training data is unsupervised action data, the action class label generated according to a predetermined rule regarding the generation of the action class label is used as the action class label of the correct data. An example of the predetermined rule (hereinafter referred to as the "pseudo-generation rule") regarding the generation of the action class label is, for example, a rule for determining the action class label based on the history of the estimation of the action class. Details of an example of the pseudo-generation rule will be described in the description of the branching process described later.

[0115] <Regarding the timing of learning of the second feature quantity estimation model, the second secondary action agent estimation model, and the second secondary action estimation model> In the first sub-action estimation model learning process, for example, before the learning end condition regarding the learning of the second feature quantity estimation model and the second sub-action subject estimation model is satisfied, the learning of the second sub-action estimation model is also performed. In the learning in the first sub-action estimation model learning process, for example, first, the update of the second feature quantity estimation model and the second sub-action subject estimation model is performed until the learning end condition is satisfied, and then, the update of the second sub-action estimation model may be performed.

[0116] When the learning of the second sub-action estimation model is also performed before the learning end condition regarding the learning of the second feature quantity estimation model and the second sub-action subject estimation model is satisfied, the update of the second feature quantity estimation model is performed in relation to the second sub-action subject estimation model and the second sub-action estimation model. Therefore, the second feature quantity estimation model is updated so that the estimation accuracy of the second sub-action estimation model is also increased. Therefore, the estimation accuracy of the first sub-action estimation model is higher than when the update of the second feature quantity estimation model and the second sub-action subject estimation model is performed until the learning end condition is satisfied and then the update of the second sub-action estimation model is performed.

[0117] Thus, the update in the first sub-action estimation model learning process is performed based on a loss function expressed using an action subject label and an action class label.

[0118] FIG. 5 is a diagram showing an example of the hardware configuration of the learning device 1 in the embodiment. The learning device 1 includes a control unit 11 including a processor 91 such as a CPU (Central Processing Unit) and a memory 92 connected by a bus, and executes a program. The learning device 1 functions as a device including a control unit 11, an input unit 12, a communication unit 13, a storage unit 14, and an output unit 15 by executing a program.

[0119] More specifically, the processor 91 reads out the program stored in the storage unit 14 and stores the read program in the memory 92. By executing the program stored in the memory 92, the learning device 1 functions as a device including the control unit 11, the input unit 12, the communication unit 13, the storage unit 14, and the output unit 15.

[0120] The control unit 11 controls the operations of various functional units included in the learning device 1. The control unit 11 performs, for example, the process of learning the main operation subject estimation model. The control unit 11 performs, for example, the main operation subject estimation model learning process. The control unit 11 performs, for example, the first sub-operation estimation model learning process.

[0121] The input unit 12 is configured to include input devices such as a mouse, a keyboard, and a touch panel. The input unit 12 may be configured as an interface for connecting these input devices to the learning device 1. The input unit 12 receives the input of various information to the learning device 1. For example, training data is input to the input unit 12.

[0122] The communication unit 13 is configured to include a communication interface for connecting the learning device 1 to an external device. The communication unit 13 communicates with the external device via wired or wireless means. The external device is, for example, a device that is the source of the training data. The communication unit 13 acquires the training data through communication with the source of the training data. The external device is, for example, the estimation device 2. The communication unit 13 transmits the program of the learned main operation estimation model to the estimation device 2 through communication with the estimation device 2.

[0123] The storage unit 14 is configured using a computer-readable storage medium device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 14 stores various information related to the learning device 1. The storage unit 14 stores, for example, the information input via the input unit 12 or the communication unit 13. The storage unit 14 stores, for example, various information generated by learning.

[0124] The output unit 15 outputs various types of information. The output unit 15 is configured to include a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The output unit 15 may be configured as an interface for connecting these display devices to the learning device 1. The output unit 15 outputs, for example, the information input to the input unit 12 or the communication unit 13.

[0125] FIG. 6 is a diagram showing an example of the configuration of the control unit 11 included in the learning device 1 according to the embodiment. The control unit 11 includes a main operation estimation model learning unit 111, an input control unit 112, a communication control unit 113, a memory control unit 114, and an output control unit 115.

[0126] The main operation estimation model learning unit 111 executes learning of the main operation estimation model. Therefore, the main operation estimation model learning unit 111 executes at least the main operation subject estimation model learning process. The main operation estimation model learning unit 111 may further execute the first-order sub-operation estimation model learning process.

[0127] The input control unit 112 controls the operation of the input unit 12. The communication control unit 113 controls the operation of the communication unit 13. The memory control unit 114 controls the operation of the memory unit 14. That is, the memory control unit 114 records information in the memory unit 14, for example. The memory control unit 114 reads out the information stored in the memory unit 14, for example. The output control unit 115 controls the operation of the output unit 15.

[0128] FIG. 7 is a flowchart showing an example of the flow of the main operation subject estimation model learning process executed by the main operation estimation model learning unit 111 according to the embodiment. The main operation estimation model learning unit 111 acquires a set of operation data (step S101). The operation data includes two or more pieces of operation data. Next, the main operation estimation model learning unit 111 acquires a first-type first feature amount and a second-type first feature amount for each piece of operation data (step S102).

[0129] Next, the main operation estimation model learning unit 111 acquires a contrast loss evaluation value (step S103). Next, the main operation estimation model learning unit 111 updates the main operation subject estimation model so that the contrast loss evaluation value becomes smaller (step S104). Next, the main operation estimation model learning unit 111 determines whether a learning end condition is satisfied (step S105). If the learning end condition is satisfied (step S105: YES), the process ends. If the learning end condition is not satisfied (step S105: NO), the process returns to the process of step S101.

[0130] FIG. 8 is a flowchart showing an example of the flow of the first secondary operation estimation model learning process executed by the main operation estimation model learning unit 111 in the embodiment. The main operation estimation model learning unit 111 acquires a set of supervised operation data, a set of unsupervised operation data, and information indicating the operation subject estimated by the main operation subject estimation model (step S101). Next, the main operation estimation model learning unit 111 determines whether the number of updates of the first secondary operation estimation model is equal to or less than a predetermined number (step S102). Hereinafter, the number of updates of the first secondary operation estimation model is referred to as the number of repetitions.

[0131] When the number of repetitions is equal to or less than the predetermined number (step S202: YES), the main operation estimation model learning unit 111 obtains an operation class identification loss evaluation value, an operation subject class identification loss evaluation value, and a result of identifying the operation class for each of the supervised operation data and the unsupervised operation data (step S203). Specifically, the result of identifying the operation class is the result of estimation by the second secondary operation estimation model.

[0132] On the other hand, when the number of repetitions is more than the predetermined number (step S202: NO), the main operation estimation model learning unit 111 reads out the result of identifying the operation class for the unsupervised operation data of the past T times (T is a predetermined integer of 1 or more) (step S204). Next, the main operation estimation model learning unit 111 obtains an operation class identification loss evaluation value, an operation subject class identification loss evaluation value, and a result of identifying the operation class (step S205).

[0133] Next to step S203 or step S205, the main operation estimation model learning unit 111 updates the first secondary operation estimation model (step S206). In the update, for the second secondary operation subject estimation model, the update is performed so as to improve the accuracy of estimating the operation subject. For the second feature quantity estimation model, the update is performed so as to reduce the accuracy of estimation by the second secondary operation subject estimation model. For the second secondary operation estimation model, the update of the second secondary operation estimation model is performed so as to reduce the difference between the operation class attribution probability estimated by the second secondary operation estimation model and the operation class label given as the correct data.

[0134] Next, the main operation estimation model learning unit 111 controls the operation of the memory control unit 114 to record the result of identifying the operation class in a predetermined storage device such as the storage unit 14 (step S207). Next, the main operation estimation model learning unit 111 determines whether the learning end condition is satisfied (step S208). If the learning end condition is satisfied (step S208: YES), the process ends. If the learning end condition is not satisfied (step S208: NO), the process returns to the process of step S201.

[0135] <Regarding the branch> Here, the branch in the first secondary operation estimation model learning process illustrated in the above flowchart will be described.

[0136] The reason for executing the branch process is to perform learning of the second secondary operation estimation model so as to improve the accuracy of identification by the second secondary operation estimation model, that is, the accuracy of identifying an unknown operation class in which there is no operation class label. The learning for the second secondary operation estimation model is performed for the purpose of improving the accuracy of identification of (K + 1) operation classes including K known operation classes and an unknown operation class.

[0137] By the way, regarding known operation classes, operation data with operation class labels exists in the set of supervised operation data, but for unknown operation classes, there may be no operation data with a label indicating that it is unknown. Therefore, when learning using only a given operation class label, the accuracy of identifying K known operation classes is high, but the accuracy of identifying unknown operation classes may be low. Thus, in the primary sub-operation estimation model learning process illustrated in the above flowchart, when the number of iterations is less than or equal to the planned number, a process of identifying whether it is an unknown operation class is performed on the set of unsupervised operation data based on the output result of the secondary sub-operation estimation model.

[0138] In the primary sub-operation estimation model learning process illustrated in the above flowchart, the result of identifying (K + 1) classes is recorded in a predetermined storage device such as the storage unit 14. When recording, it is not overwritten, but recorded so that a history of the identification results is generated. That is, a predetermined storage device such as the storage unit 14 stores a history of the identification results (hereinafter referred to as "identification history").

[0139] After that, when the number of iterations is more than the predetermined number, a pseudo-operation class label is generated based on the result of identifying the operation class recorded in the identification history. The definition of the pseudo-operation class label is information indicating an operation class that satisfies a predetermined condition among the operation classes shown in the identification history. The predetermined condition is, for example, the condition that it is the operation class that appears most frequently among the operation classes recorded in a certain recent period. The rule for generating a pseudo-operation class label in this way is an example of a pseudo-generation rule.

[0140] In the primary sub-operation estimation model learning process illustrated in the above flowchart, using the generated pseudo-operation class label, the secondary sub-operation estimation model is learned so that the accuracy of identifying (K + 1) classes by the secondary sub-operation estimation model is improved. The result of identifying the operation class is additionally recorded in the identification history. The identification history with the identification result added is used for generating the operation class label during the next iterative learning.

[0141] Based on the output result of the secondary sub - operation estimation model used when the number of repetitions is less than or equal to a predetermined number, by identifying the unknown operation class, formally, it is possible to identify the unknown operation class. However, there is a problem that this method alone contains many errors. Therefore, in the primary sub - operation estimation model learning process illustrated in the above flowchart, for the unsupervised operation data, a pseudo - operation class label (hereinafter referred to as "pseudo - teacher operation class label") including the unknown operation class is generated.

[0142] And then, in the primary sub - operation estimation model learning process illustrated in the above flowchart, thereafter, the learning of the secondary sub - operation estimation model and the generation of a pseudo - operation class label including the unknown operation class are mutually performed. As a result, the accuracy of identifying the unknown operation class is improved.

[0143] Here, a more detailed example of the primary sub - operation estimation model learning process illustrated in the above flowchart is given using mathematics.

[0144] [Regarding the case where the number of repetitions is less than or equal to a predetermined number] First, the case where the number of repetitions is less than or equal to a predetermined number is explained. In step S202, the processes executed by the second feature quantity estimation model, the second sub - operation subject estimation model, and the secondary sub - operation estimation model when the number of repetitions is less than or equal to a predetermined number are explained.

[0145] [Regarding the operation class identification loss function] The second feature quantity estimation model is a function that takes the operation data x as input and outputs a vector f representing the second feature quantity, and is represented by a function F having parameters θ e The secondary sub - operation estimation model is a function that takes the vector f as input and outputs the operation class membership probability y, and is represented by a function having parameters θ m

[0146] ​The action class belonging probability y is represented by the following formula (9) as a probability function using the function F.

[0147] [Number]

[0148] Formula (9) is the probability of y appearing given θ e , θ m and x. The learned second feature quantity estimation model and the second-order sub-action estimation model estimate, for example, the action class belonging probability t of each action class indicated by the correct data of the action data s when the action data s is obtained from a set of supervised action data. That is, the learned second feature quantity estimation model and the second-order sub-action estimation model are, for example, a second feature quantity estimation model and a second-order sub-action estimation model that obtain a belonging probability by which the correct action class can be identified.

[0149] Hereinafter, the action class belonging probability of each action class indicated by the correct data is referred to as the teacher action class belonging probability. Expressing the appearance probability of the teacher action class belonging probability t corresponding to the action data s as p(s, t), in the first-order sub-action estimation model learning process, the parameter θ e or θ m is updated so that the value of the following formula (10) becomes small.

[0150] [Number]

[0151] As described above, the set of supervised action data is a set of pairs of action data and action class labels, and the action class labels are information indicating the action class belonging probability. Therefore, formula (10) is approximately replaced by the following formula (11).

[0152] [Number]

[0153] Note that both S and T are each a set of pairs of one or more pieces of operation data and corresponding teacher operation class attribution probabilities. Equation (11) is part of the loss function in the first-order sub-operation estimation model learning process. Hereinafter, the function of Equation (11) is referred to as the operation class discrimination loss function. Hereinafter, the value obtained by evaluating the operation class discrimination loss function for arbitrary S and T is referred to as the operation class discrimination loss function value. That is, hereinafter, the value of the operation class discrimination loss function is referred to as the operation class discrimination loss function value.

[0154] In the first-order sub-operation estimation model learning process, the first-order sub-operation estimation model is updated so that the value of Equation (11) becomes smaller with respect to the parameters θ e and θ m . An example of a method for obtaining the values of the parameters θ e and θ m will be described. If the function representing the first-order sub-operation estimation model is differentiable with respect to the parameters θ e and θ m , it is known that local minimization is possible. Therefore, by executing an algorithm such as the gradient descent method, the values of the parameters θ e and θ m that minimize the value of Equation (11) can be obtained.

[0155] Incidentally, it was described above that the second feature quantity estimation model is a function that outputs a vector f representing the second feature quantity with the operation data x as an input and has the parameter θ e . The function representing the second feature quantity estimation model may further satisfy the condition of being differentiable with respect to the parameter θ e .

[0156] [Regarding the operation subject class discrimination loss function] The second sub-operation subject estimation model is a function that takes the feature vector f output by the second feature quantity estimation model as an input and outputs the operation subject class attribution probability and has the parameter θ c . The operation subject class attribution probability estimated by the second sub-operation subject estimation model is represented by the following Equation (12) as a probability function using the above-mentioned function F.

[0157]

Number

[0158] Equation (12) is the probability of the occurrence of c given θ e , θ c and x. The second secondary actor estimation model is updated to increase the similarity between the estimated actor class membership probability of the second secondary actor estimation model for the given action data x from the set of supervised action data and the set of unsupervised action data and the estimation result of the primary actor estimation model.

[0159] Hereinafter, the estimated actor class membership probability estimated by the primary actor estimation model is referred to as the teacher actor class membership probability. Expressing the occurrence probability of the actor class membership probability c corresponding to the action data x as p(x, c), in the learning of the second secondary actor estimation model, the parameter θ c is updated so that the following equation (13) becomes smaller.

[0160]

Number

[0161] On the other hand, in the learning of the second feature quantity estimation model, the parameter θ e is updated so that the value of equation (13) becomes larger.

[0162] The significance of the second secondary actor estimation model learning to make the value of equation (13) smaller while the second feature quantity estimation model learning to make the value of the same equation (13) larger will be explained. In the first secondary action estimation model learning process, the second feature quantity estimation model extracts the feature quantities invariant to the actor, aiming to make the learning less affected by the differences of the actors in the discrimination by the second secondary action estimation model.

[0163] In the case of the second secondary action subject estimation model, learning is performed to reduce the value of Equation (13), thereby improving the accuracy of identifying the action subject. On the other hand, in the case of the second feature quantity estimation model, learning is performed to increase the value of (13), so that the features output by the second feature quantity estimation model change to features that make it more difficult to identify the action subject. By continuing this learning, the second feature quantity estimation model can extract invariant feature quantities for action subjects with little influence of the differences between action subjects.

[0164] In the first-order secondary action estimation model learning process, similar to Equation (10), Equation (13) is approximately equal to the following Equation (14).

[0165]

Equation

[0166] Note that both X and C are sets of pairs of one or more pieces of action data and the corresponding teacher action subject class membership probabilities. Hereinafter, the function of Equation (14) is referred to as the action subject class identification loss function. Hereinafter, the value obtained by evaluating the action subject class identification loss function for arbitrary X and C is referred to as the action subject class identification loss function value. That is, hereinafter, the value of the action subject class identification loss function is referred to as the action subject class identification loss function value.

[0167] Incidentally, it was described above that the second secondary action subject estimation model is a function that takes as input the feature vector f output by the second feature quantity estimation model and outputs the action subject class membership probability, and has the parameter θ c The function representing the second secondary action subject estimation model may further have the condition of being differentiable with respect to the parameter θ c

[0168] [Regarding the process executed by the second-order secondary action estimation model and the information entropy] ​For the motion data x, the probability that the motion data belongs to any of the known motion classes is the motion class membership probability y output by the secondary sub-motion estimation model, and can be expressed using the above formula (9). Regarding the motion class membership probability y output by the secondary sub-motion estimation model, the information entropy H(y|x), which is a value indicating the uncertainty of the output motion class membership probability y, is expressed as the following formula (15).

[0169]

Number

[0170] Therefore, the secondary sub-motion estimation model determines whether the unlabeled motion data u in the set of unlabeled motion data belongs to the motion data of the unknown motion class by whether the value of the information entropy H is greater than the threshold σ. The threshold σ is a predetermined value.

[0171] Based on the information entropy H, the secondary sub-motion estimation model determines the motion data determined to belong to the unknown motion class as the unknown motion class K+1. Note that K+1 is an identifier indicating the unknown motion class. In this case, the known motion classes are identified by any value from 1 to K. On the other hand, the secondary sub-motion estimation model determines the motion data not determined to belong to the unknown motion class as belonging to the class with the largest motion class membership probability y estimated by the secondary sub-motion estimation model.

[0172] The result y of identifying the motion class of the unlabeled motion data u at the e-th iteration u、e is expressed by the following formula (16).

[0173]

Number

[0174] So far, the explanation for the case where the number of iterations is greater than the predetermined number of times ends. Next, the case where the number of iterations is greater than the predetermined number of times will be explained.

[0175] [[Regarding the case where the number of repetitions is greater than a predetermined number]] First, the process in step S204 will be described. In the process of step S204, for each piece of motion data in the set of unsupervised motion data, their past identification results are read, and based on the read identification results, a process of obtaining the probability of belonging to a pseudo-teacher motion class for the unsupervised motion data is executed. The probability of belonging to a pseudo-teacher motion class represents the probability of which class among the motion classes including unknown motions the motion data belongs to, and is a pseudo probability of belonging to a teacher motion class. Note that in this specification, the definition of "pseudo" means that it is obtained as a result of estimation by the main motion estimation model learning unit 111. Therefore, for example, the pseudo probability of belonging to a teacher motion class means that it is not a probability of belonging to a teacher motion class given in advance by a user or the like, but a probability of belonging to a teacher motion class obtained as a result of estimation by the main motion estimation model learning unit 111. Note that the above-mentioned pseudo motion class label is also information indicating the motion class estimated by the main motion estimation model learning unit 111 to be a motion class that satisfies a predetermined condition, such as the motion class that appears most frequently among the motion classes recorded in a recent fixed period. Therefore, the pseudo motion class label is also obtained as a result of estimation by the main motion estimation model learning unit 111.

[0176] Regarding the motion data u in the set of unsupervised motion data, the result of identifying the motion class at the time of the number of repetitions t is y in step S207 described later u、t and is recorded in a predetermined recording destination such as the storage unit 14.

[0177] In step S204, the results of the past T identifications are read from a predetermined recording destination, and the probability of belonging to the class that is the most frequent in the past T times is set to 1, and the probability of belonging to other classes is set to 0, and this is used as the pseudo-teacher motion class belonging probability. Therefore, the pseudo-teacher motion class probability y of the unsupervised motion data u at the number of repetitions e u is represented by, for example, the following formula (17) using mode(·) that represents the most frequent value.

[0178] [[Equation]]

[0179] Next, the evaluation process according to step S205 will be described.

[0180] [Handling of the action class discrimination loss function] In the first-order secondary action estimation model learning process, a sum is obtained for the supervised action data, the set (S, T) of supervised action class membership probabilities, the unsupervised action data, and the union of the set (S', T') of pseudo-supervised action class membership probabilities obtained in step 204. In the first-order secondary action estimation model learning process, based on the obtained sum, the value of the action class discrimination loss function is obtained. The value of the action class discrimination loss function is represented by the following formula (18).

[0181] [Equation]

[0182] [Processing of the action subject class discrimination loss function] When obtaining the value of the action subject class discrimination loss function when the number of iterations is greater than a predetermined number, the same process as the value of the action subject class discrimination loss function in step S203 is executed.

[0183] [Regarding the processing executed by the second-order secondary action estimation model and the information entropy] In step S205, the second-order secondary action estimation model outputs the action class membership probabilities for a total of (K + 1) classes including the unknown action class and K known action classes. In such a case, the result of identifying the action class by the second-order secondary action estimation model is the class with the largest action class membership probability among the (K + 1) classes. The identification y of the action class of the unsupervised action data u at the e-th iteration u、e is represented by the following formula (19).

[0184] [Equation]

[0185] [Learning] The learning related to step S206 will be described. Regarding the second feature quantity estimation model, for the operation class discrimination loss evaluation value L s is small, and learning is performed so that the value of the operation subject class discrimination loss evaluation value L c becomes large. Regarding the secondary sub-operation estimation model, learning is performed so that the operation class discrimination loss evaluation value L s becomes small. Regarding the second sub-operation subject estimation model, learning is performed so that the operation subject class discrimination loss evaluation value becomes small. Specifically, in the learning related to step S206, the problems shown in equations (20) and (21) are sequentially optimized.

[0186] [Number]

[0187] [Number]

[0188] When each function expressing the second feature quantity estimation model, the secondary sub-operation estimation model, and the second sub-operation subject estimation model is a function that satisfies the loss differentiable condition, learning by the error gradient descent method is possible. The loss differentiable condition is that the class discrimination loss evaluation value L s and the domain discrimination loss evaluation value L c are differentiable with respect to the parameters θ e , θ m and θ c , which means satisfying the condition.

[0189] [Recording of Parameters] After updating the values of the parameters, in the process related to step S206, the parameters θ e , θ m , θ c are recorded in a predetermined recording destination such as the storage unit 14.

[0190] [Recording of Learning Results] In the process according to step S207, the operation class identification result y u、e is recorded in a predetermined recording destination such as the storage unit 14.

[0191] The above learning processes from step S201 to S207 may be repeated until the learning end condition is satisfied. The learning end condition may be, for example, a condition of repeating a predetermined number of times, a condition that the value of the objective function does not change more than a certain level, or a condition that the accuracy for evaluation data prepared separately from the training data does not change more than a certain level.

[0192] In this way, the learning device 1 includes a control unit 11 that executes a main operation subject estimation model for estimating an operation subject based on operation data. The control unit 11 updates the estimation accuracy of the main operation subject estimation model by learning including the execution of operation information amount attenuation processing. The control unit 11 updates the main operation subject estimation model so as to increase the similarity between the feature amount of the operation data used for learning and the time-series feature amount obtained by executing the operation information amount attenuation processing for the operation data as the execution target. The control unit 11 updates the main operation subject estimation model so that the similarity between the feature amount of the operation data and the time-series feature amount obtained by executing the operation information amount attenuation processing for other operation data different from the operation data as the execution target does not increase.

[0193] Also, in this way, the control unit 11 executes a first-order sub-operation estimation model for estimating the operation indicated by the operation data based on the operation data. The estimation accuracy of the first-order sub-operation estimation model is improved by learning using the estimation result of the main operation subject estimation model.

[0194] Also, as described above, in the learning of the primary sub-operation estimation model, the control unit 11 executes a second feature quantity estimation model that estimates feature quantities based on the operation data. In the learning of the primary sub-operation estimation model, the control unit 11 executes a second sub-operation subject estimation model that estimates the operation subject based on the estimation result of the second feature quantity estimation model. In the learning of the primary sub-operation estimation model, the control unit 11 executes a secondary sub-operation estimation model that estimates the operation based on the estimation result of the second feature quantity estimation model.

[0195] The control unit 11 updates based on the estimation result of the second feature quantity estimation model, the estimation result of the second sub-operation subject estimation model, the estimation result of the secondary sub-operation estimation model, and the execution result of the main operation subject estimation model for the operation data. In the update, the control unit 11 updates the second sub-operation subject estimation model and the secondary sub-operation estimation model so that the estimation accuracy is improved. In the update, the control unit 11 updates the second feature quantity estimation model so that the estimation accuracy of the second sub-operation subject estimation model based on the estimation result of the second feature quantity estimation model decreases.

[0196] <<Estimation Device 2>> The estimation device 2 acquires the learned main operation estimation model obtained by the learning device 1 and estimates the operation indicated by the input operation data using the learned main operation estimation model.

[0197] FIG. 9 is a diagram showing an example of the hardware configuration of the estimation device 2 in the embodiment. The estimation device 2 includes a control unit 21 including a processor 93 such as a CPU and a memory 94 connected by a bus, and executes a program. The estimation device 2 functions as a device including a control unit 21, an input unit 22, a communication unit 23, a storage unit 24, and an output unit 25 by executing the program.

[0198] More specifically, the processor 93 reads out the program stored in the storage unit 24 and stores the read program in the memory 94. By the processor 93 executing the program stored in the memory 94, the estimation device 2 functions as a device including a control unit 21, an input unit 22, a communication unit 23, a storage unit 24, and an output unit 25.

[0199] The control unit 21 controls the operations of various functional units provided in the estimation device 2. For example, the control unit 21 executes a learned main operation subject estimation model.

[0200] The input unit 22 includes input devices such as a mouse, a keyboard, and a touch panel. The input unit 22 may be configured as an interface for connecting these input devices to the estimation device 2. The input unit 22 receives the input of various information to the estimation device 2. For example, operation data of the estimation target is input to the input unit 22.

[0201] The communication unit 23 includes a communication interface for connecting the estimation device 2 to an external device. The communication unit 23 communicates with the external device via wired or wireless means. The external device is, for example, the device of the transmission source of the estimation target. The communication unit 23 acquires the estimation target by communicating with the transmission source of the estimation target. The external device is, for example, the learning device 1. The communication unit 13 acquires the program of the learned main operation estimation model by communicating with the learning device 1.

[0202] The storage unit 24 is configured using a computer-readable storage medium device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 24 stores various information related to the estimation device 2. The storage unit 24 stores, for example, the information input via the input unit 22 or the communication unit 23. The storage unit 24 stores, for example, various information generated by the processing of estimating the operation indicated by the estimation target.

[0203] The output unit 25 outputs various information. The output unit 25 includes, for example, a display device such as a CRT display, a liquid crystal display, or an organic EL display. The output unit 25 may be configured as an interface for connecting these display devices to the learning device 1. The output unit 25 outputs, for example, the information input to the input unit 22 or the communication unit 23.

[0204] FIG. 10 is a diagram showing an example of the configuration of the control unit 21 included in the estimation device 2 according to the embodiment. The control unit 21 includes an estimation unit 211, an input control unit 212, a communication control unit 213, a memory control unit 214, and an output control unit 215.

[0205] The estimation unit 211 executes a learned main operation estimation model for the estimation target. The estimation unit 211 estimates the operation indicated by the estimation target by executing the learned main operation estimation model for the estimation target.

[0206] The input control unit 212 controls the operation of the input unit 22. The communication control unit 213 controls the operation of the communication unit 23. The memory control unit 214 controls the operation of the memory unit 24. That is, the memory control unit 214 records information in the memory unit 24, for example. The memory control unit 214 reads out information stored in the memory unit 24, for example. The output control unit 215 controls the operation of the output unit 25.

[0207] FIG. 11 is a flowchart showing an example of the processing flow executed by the estimation device 2 according to the embodiment. The estimation unit 211 acquires operation data that is the estimation target (step S301). The estimation target acquired by the estimation unit 211 may be, for example, input to the input unit 22, or may be input to the estimation device 2 from an external device via the communication unit 23.

[0208] Next, the estimation unit 211 estimates the operation indicated by the operation data of the estimation target by executing the learned main operation estimation model (step S302). Next, the output control unit 215 causes the output unit 25 to output the estimation result estimated in step S302 (step S303).

[0209] The learning device 1 configured as described above performs learning to improve the accuracy of estimating the operating entity based on the first type of first feature amount and the second type of second feature amount obtained by the operation information amount attenuation process. Therefore, even when the training data does not contain information indicating the operating entity, the learning device 1 can enhance the accuracy of estimating the operating entity. Since a mathematical model for obtaining information on the operating entity in this way is obtained, even when the training data does not contain information indicating the operating entity, the learning device 1 can also use the information on the estimated operating entity to improve the accuracy of estimating the mathematical model for estimating the operation indicated by the operation data. Therefore, the learning device 1 can suppress the deterioration of the accuracy of operation estimation caused by the lack of information on the operating entity.

[0210] The estimation device 2 configured as described above performs estimation using the mathematical model obtained by the learning device 1, which is a learned mathematical model for estimating the operation indicated by the operation data. Therefore, the estimation device 2 can suppress the deterioration of the accuracy of operation estimation caused by the lack of information on the operating entity.

[0211] The estimation system 100 configured as described above includes the learning device 1. Therefore, the estimation system 100 can suppress the deterioration of the accuracy of operation estimation caused by the lack of information on the operating entity.

[0212] Also, even if any one of the following first condition regarding the training data and the second condition or the third condition is satisfied, the estimation system 100 configured as described above can suppress the deterioration of the accuracy of estimation of the learned main operation estimation model.

[0213] The first condition regarding the training data is the condition that the training data does not contain an operation entity label. That is, it is the condition that none of the training data included in the training data set contains correct answer data. Specifically, the correct answer data is an operation entity label.

[0214] The second condition regarding the training data is the condition that there exists an action class for which action data is missing for each acting entity. The third condition regarding the training data is the condition that data of an unknown action class may be included.

[0215] Even when the first condition regarding the training data is satisfied, the estimation system 100 can suppress deterioration in the estimation accuracy of the learned main action estimation model in order to estimate the acting entity label for the entire training data. Since the estimation system 100 performs learning using the estimated acting entity label, even when the second condition or the third condition regarding the training data is satisfied, the estimation system 100 can suppress deterioration in the estimation accuracy of the learned main action estimation model.

[0216] (Modification example) Note that the estimation unit 211 may execute a learned first-order sub-action estimation model instead of the learned main action estimation model. That is, the estimation unit 211 does not need to use all the mathematical models obtained by learning the main action estimation model, and may execute only the learned first-order sub-action estimation model.

[0217] Note that the main action estimation model may include a third-order sub-action estimation model instead of the first-order sub-action estimation model. The third-order sub-action estimation model is a mathematical model that estimates an action based on action data and information on the acting entity. In such a case, even if action data is input to the main action estimation model and information on the acting entity is not input, the main action entity estimation model outputs information on the acting entity based on the action data. Therefore, the third-order sub-action estimation model can estimate an action based on the action data and the information on the acting entity output by the main action entity estimation model.

[0218] The accuracy of the estimation of the third-order secondary action estimation model is higher as the information of the action subject is more accurate. Therefore, as the accuracy of the estimation of the main action subject estimation model included in the main action estimation model is updated by learning, the accuracy of the estimation of the main action estimation model also improves. The learned main action estimation model configured in this way is executed by, for example, the estimation unit 211. Therefore, the estimation device 2 including the estimation unit 211 that executes the learned main action estimation model configured in this way can suppress the deterioration of the estimation accuracy of the action caused by the lack of information of the action subject.

[0219] Note that each of the learning device 1 and the estimation device 2 may be implemented using a plurality of information processing devices communicably connected via a network. In this case, each functional unit included in each of the learning device 1 and the estimation device 2 may be implemented in a distributed manner in a plurality of information processing devices.

[0220] Note that all or part of each function of the estimation system 100, the learning device 1, and the estimation device 2 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk incorporated in a computer system. The program may be transmitted via an electric communication line.

[0221] Note that the main action subject estimation model is an example of the first mathematical model. The first-order secondary action estimation model is an example of the second mathematical model. The second feature quantity estimation model is an example of the third mathematical model. The second secondary action subject estimation model is an example of the fourth mathematical model. The second-order secondary action estimation model is an example of the fifth mathematical model. The third-order secondary action estimation model is an example of the sixth mathematical model.

[0222] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Explanation of Reference Numerals

[0223] 100…Estimation system, 1…Learning device, 2…Estimation device, 11…Control unit, 12…Input unit, 13…Communication unit, 14…Memory unit, 15…Output unit, 111…Main operation estimation model learning unit, 112…Input control unit, 113…Communication control unit, 114…Memory control unit, 115…Output control unit, 21…Control unit, 22…Input unit, 23…Communication unit, 24…Memory unit, 25…Output unit, 211…Estimation unit, 212…Input control unit, 213…Communication control unit, 214…Memory control unit, 215…Output control unit, 91…Processor, 92…Memory, 93…Processor, 94…Memory

Claims

1. a control unit that executes a first mathematical model for estimating an actor based on action data that is a time series of amounts indicating the action; Equipped with the control unit updates the estimation accuracy of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an execution target, a process of attenuating an amount of information of action types from the execution target, and a process in which an amount of attenuation of the amount of information of the action types possessed by the execution target is greater than an amount of attenuation of the amount of information possessed by the execution target, the amount of attenuation being an amount of information possessed by an action subject that is the subject of the action; the control unit updates the first mathematical model so as to increase a similarity between a feature amount of the action data used in the learning and a time-series feature amount obtained by executing the action information amount attenuation process with the action data as an execution target; the control unit updates the first mathematical model so as not to increase a similarity between a feature amount of the action data and a time-series feature amount obtained by executing the action information amount attenuation process for action data other than the action data. Learning device.

2. The motion information amount attenuation process is a process of randomizing the phase of a sine wave component to be executed.

2. A learning device according to claim 1.

3. The control unit executes a second mathematical model that estimates a movement indicated by the movement data based on the movement data, and the second mathematical model improves the estimation accuracy by learning using a result of the estimation of the first mathematical model. The learning device according to claim 1 or 2.

4. the control unit, in learning the second mathematical model, executes a third mathematical model that estimates a feature amount based on motion data, a fourth mathematical model that estimates an actor based on a result of the estimation of the third mathematical model, and a fifth mathematical model that estimates a motion based on a result of the estimation of the third mathematical model; the control unit updates the fourth mathematical model and the fifth mathematical model based on an estimation result of the third mathematical model, an estimation result of the fourth mathematical model, an estimation result of the fifth mathematical model, and a result of executing the first mathematical model on the motion data so as to increase the estimation accuracy, and updates the third mathematical model so as to decrease the estimation accuracy of the fourth mathematical model based on the estimation result of the third mathematical model. The learning device according to claim 3 .

5. and a control unit that executes a first mathematical model that estimates an actor based on action data that is a time series of amounts indicating an action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of an action type from the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of attenuation of the amount of information possessed by the action target that is an action target that is the subject of the action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of the action type possessed by the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of attenuation of the amount of information possessed by the action target, and an estimation unit that executes a learned second mathematical model obtained by updating the first mathematical model so as to increase a similarity between the feature amount of the motion data and a time series feature amount obtained by executing the motion information amount attenuation process for motion data other than the motion data, and the control unit updates the first mathematical model so as not ... An estimation device comprising:

6. and a control unit that executes a first mathematical model that estimates an actor based on action data that is a time series of amounts indicating an action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of an action type from the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of attenuation of the amount of information possessed by the action target that is an action target that is the subject of the action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of the action type possessed by the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of attenuation of the amount of information possessed by the action target, and an estimation unit that executes a sixth mathematical model that estimates a motion indicated by an estimation target based on a result of estimation of the learned first mathematical model obtained by a learning device and the motion data of the estimation target, the sixth mathematical model updating the first mathematical model so as to increase a similarity between the feature amount of the motion data and a time series feature amount obtained by executing the motion information amount attenuation process for motion data other than the motion data, and the control unit updating the first mathematical model so as not to increase a similarity between the feature amount of the motion data and a time series feature amount obtained by executing the motion information amount attenuation process for motion data other than the motion data; An estimation device comprising:

7. A control step in which a computer executes a first mathematical model that infers an actor based on action data that is a time series of amounts indicating the action; having the control step updates the estimation accuracy of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an execution target, a process of attenuating an amount of information of the action type from the execution target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the execution target is greater than an amount of attenuation of the amount of information possessed by the execution target, the amount of attenuation being an amount of information possessed by an action subject that is the subject of the action; The control step includes updating the first mathematical model so as to increase a similarity between a feature amount of the action data used in the learning and a time-series feature amount obtained by executing the action information amount attenuation process with the action data as an action target; The control step updates the first mathematical model so as not to increase a similarity between a feature amount of the motion data and a time-series feature amount obtained by executing the motion information amount attenuation process for other motion data different from the motion data. How to learn.

8. and a control unit that executes a first mathematical model that estimates an actor based on action data that is a time series of amounts indicating an action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of an action type from the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of information possessed by the action target that is an action target that is the subject of the action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of the action type possessed by the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of attenuation of an amount of information possessed by the action target, and an estimation step of executing a learned second mathematical model obtained by a learning device that updates the first mathematical model so as to increase a similarity between the feature amount of the motion data and a time series feature amount obtained by execution of the motion information amount attenuation process for the motion data to be executed, and the control unit updates the first mathematical model so as not to increase a similarity between the feature amount of the motion data and a time series feature amount obtained by execution of the motion information amount attenuation process for other motion data different from the motion data, and updates a second mathematical model that estimates a motion based on the motion data using a result of estimation of the first mathematical model; The estimation method has the following structure:

9. and a control unit that executes a first mathematical model that estimates an actor based on action data that is a time series of amounts indicating an action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of an action type from the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of attenuation of the amount of information possessed by the action target that is an action target that is the subject of the action, and the control unit updates the accuracy of the estimation of the first mathematical model by learning including execution of an action information amount attenuation process, which is a process of obtaining a time series of action data as an action target, a process of attenuating an amount of information of the action type possessed by the action target, and a process in which an amount of attenuation of the amount of information of the action type possessed by the action target is greater than an amount of attenuation of the amount of information possessed by the action target, and an estimation unit that executes a sixth mathematical model that estimates a motion indicated by an estimation target based on a result of estimation of the learned first mathematical model obtained by a learning device and the motion data of the estimation target, the sixth mathematical model updating the first mathematical model so as to increase a similarity between the feature amount of the motion data and a time series feature amount obtained by executing the motion information amount attenuation process for motion data other than the motion data, and the control unit updating the first mathematical model so as not to increase a similarity between the feature amount of the motion data and a time series feature amount obtained by executing the motion information amount attenuation process for motion data other than the motion data; The estimation method comprises:

10. A program for causing a computer to function as either one of the learning device according to claim 1 or 2 and the estimation device according to claim 5 or 6.

Images