Learning program, information processing device and learning method

Abstract

To provide a learning program for suppressing a load required to generate a learning model, an information processing device and a learning method.SOLUTION: In a method, a computer is caused to execute processing for: learning labeled teacher data including various kinds of data and a corresponding label for each of a plurality of kinds of data about a prescribed object to generate a plurality of first learning models respectively corresponding to the plurality of kinds of data; and learning unlabeled measurement data including the plurality of kinds of data and including no corresponding labels to generate a second learning model including the plurality of first learning models. In the processing for generating the second learning model, the second learning model is generated by learning a parameter in a prescribed layer constituting each of the plurality of first learning models such that the difference of first values respectively outputted from the plurality of first learning models satisfies a prescribed condition by respectively inputting the plurality of kinds of data included in the unlabeled measurement data to the plurality of first learning models.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a learning program, an information processing device, and a learning method. [Background technology]

[0002] For example, a learning model (hereinafter simply referred to as a learning model) is generated that uses measurement results about a person (hereinafter also referred to as a target) as input to estimate the behavior of the person (see Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-168963 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-224122 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, when generating a learning model as described above, it is necessary to prepare a large amount of training data including, for example, data showing measurement results about a person (hereinafter also referred to as measurement data) and data showing the behavior of the person (hereinafter referred to as behavior data). Therefore, generating a learning model as described above may require, for example, a huge load and time.

[0005] Therefore, an object of the present disclosure is to provide a learning program, an information processing device, and a learning method that can reduce the load required to generate a learning model. [Means for solving the problem]

[0006] The learning program in the present disclosure is a learning program that causes a computer to execute a process of: generating a plurality of first learning models corresponding to each of a plurality of types of data for a specified target by learning labeled training data including each type of data and a label corresponding to each type of data; and generating a second learning model including the plurality of first learning models by learning unlabeled measurement data including the plurality of types of data but not including a label corresponding to the plurality of types of data.In the process of generating the second learning model, the second learning model is generated by learning parameters in a specified layer that constitutes each of the plurality of first learning models so that the difference in first values ​​output from each of the plurality of first learning models satisfies a specified condition by inputting the plurality of types of data included in the unlabeled measurement data into each of the plurality of first learning models. [Effects of the Invention]

[0007] According to the learning program, information processing device, and learning method of the present invention, it is possible to reduce the load required to generate a learning model. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing system 10 according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an outline of the first embodiment. [Figure 3] FIG. 3 is a diagram for explaining an outline of the first embodiment. [Figure 4] FIG. 4 is a flowchart illustrating the details of the learning process in the first embodiment. [Figure 5] FIG. 5 is a flowchart illustrating the details of the learning process in the first embodiment. [Figure 6] FIG. 6 is a flowchart illustrating details of the learning process in the first embodiment. [Figure 7] FIG. 7 is a flowchart illustrating details of the learning process in the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating details of the learning process in the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating details of the learning process in the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating details of the learning process in the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating details of the learning process in the first embodiment. [Figure 12] FIG. 12 is a diagram illustrating details of the learning process in the first embodiment. [Figure 13] FIG. 13 is a diagram illustrating details of the learning process in the first embodiment. [Figure 14] FIG. 14 is a flowchart illustrating details of the estimation process according to the first embodiment. [Figure 15] FIG. 15 is a diagram illustrating details of the estimation process in the first embodiment. [Figure 16] FIG. 16 is a diagram illustrating details of the estimation process in the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, such descriptions should not be interpreted in a limiting sense, and do not limit the subject matter described in the claims. Furthermore, various changes, substitutions, and modifications can be made without departing from the spirit and scope of the present disclosure. Furthermore, different embodiments can be combined as appropriate.

[0010] [Configuration Example of Information Processing System 10 in First Embodiment] First, a configuration example of an information processing system 10 according to the first embodiment will be described. Fig. 1 is a diagram showing a configuration example of an information processing device 1 according to the first embodiment.

[0011] As shown in FIG. 1, the information processing system 10 includes, for example, an information processing device 1 and a plurality of measuring devices 2.

[0012] The information processing device 1 is a computer device, such as a general-purpose PC (Personal Computer). The information processing device 1 performs, for example, a process of generating a learning model (hereinafter also referred to as a learning process) and a process of estimating the behavior of a person (not shown) using the learning model (hereinafter also referred to as an estimation process). Hereinafter, the learning process and the estimation process will be collectively referred to simply as the learning process, etc.

[0013] The information processing device 1 has the hardware configuration of a general-purpose computer device, and includes, for example, a CPU 101 which is a processor, a memory 102, a communication device 103, and a storage medium 104 (hereinafter also referred to as the storage unit 104), as shown in Fig. 1. Each unit is connected to each other via a bus 105.

[0014] The storage medium 104 has a program storage area (not shown) that stores, for example, a program 110 for performing learning processing and the like.

[0015] The storage medium 104 also has a storage area (not shown) for storing information used when performing, for example, learning processing, etc. The storage medium 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0016] The CPU 101 performs learning processing and the like by executing a program 110 loaded from the storage medium 104 into the memory 102, for example.

[0017] The communication device 103 communicates with each of the multiple measurement devices 2 attached to the person, for example, by wired communication or wireless communication.

[0018] The plurality of measuring devices 2 may include, for example, measuring devices (e.g., acceleration sensors) that measure acceleration in three orthogonal axes regarding the movement of a person. The plurality of measuring devices 2 may also include, for example, measuring devices that measure the person's heart rate. Furthermore, the plurality of measuring devices 2 may also include, for example, measuring devices that measure vital data other than the heart rate, such as the person's body temperature or blood pressure.

[0019] The following description will be given assuming that the measuring device 2 is attached to a person, but the present invention is not limited to this. Specifically, the measuring device 2 may be attached to an object other than a person (e.g., a robot, etc.). That is, the learning process may be, for example, a process of generating a learning model capable of estimating the behavior of an object other than a person. Furthermore, the estimation process may be, for example, a process of estimating the behavior of an object other than a person by using the learning model generated in the learning process.

[0020] In addition, although the following description will be given of a case where the learning process and the like are performed in a single information processing device 1, the present invention is not limited to this. Specifically, the learning process and the like may be performed in a distributed manner among a plurality of information processing devices 1, for example.

[0021] [Outline of the first embodiment] Next, an outline of the first embodiment will be described. Figures 2 and 3 are diagrams for explaining the outline of the first embodiment. Specifically, Figure 2 is a block diagram for explaining the functions of the information processing device 1. Figure 3 is a diagram for explaining information stored in the storage medium 104.

[0022] 2, the information processing device 1 realizes various functions including, for example, a data acquisition unit 111, a data generation unit 112, a first model generation unit 113, a second model generation unit 114, a behavior estimation unit 115, and a result output unit 116. The data acquisition unit 111, the data generation unit 112, the first model generation unit 113, and the second model generation unit 114 are functions that realize a learning process. The data acquisition unit 111, the data generation unit 112, the second model generation unit 114, the behavior estimation unit 115, and the result output unit 116 are functions that realize an estimation process.

[0023] Furthermore, as shown in FIG. 3, the storage medium 104 stores, for example, measurement data 131, behavioral data 132, labeled training data DT1, unlabeled measurement data DT2, a learning model MD1 (hereinafter also referred to as the first learning model MD1), and a learning model MD2 (hereinafter also referred to as the second learning model MD2).

[0024] First, the function for realizing the learning process will be described.

[0025] The data acquiring unit 111 acquires, for example, each measurement data 131 measured by each measuring device 2 attached to the person from each measuring device 2. The measurement data 131 may include, for example, data indicating the acceleration of the person's movement (hereinafter also referred to as acceleration data 131) and data indicating the person's heart rate (hereinafter also referred to as heart rate data 131). Then, the data acquiring unit 111 stores, for example, each acquired measurement data 131 in the storage medium 104.

[0026] Each of the plurality of measurement devices 2 may, for example, measure the measurement data 131 at the same regular interval. Furthermore, each of the plurality of measurement devices 2 may, for example, voluntarily transmit the measured measurement data 131 to the information processing device 1 when measuring the measurement data 131.

[0027] Furthermore, when the data acquiring unit 111 acquires, for example, acceleration data 131 about the X-axis direction (hereinafter also referred to as X-axis acceleration data 131a), acceleration data 131 about the Y-axis direction (hereinafter also referred to as Y-axis acceleration data 131b), and acceleration data 131 about the Z-axis direction (hereinafter also referred to as Z-axis acceleration data 131c) as measurement data 131, it may also store a value calculated by using the acquired X-axis acceleration data 131a, Y-axis acceleration data 131b, and Z-axis acceleration data 131c in the storage medium 104 as measurement data 131. Specifically, the data acquiring unit 111 may also store, for example, data indicating the square root of the sum of the squares of the X-axis acceleration data 131a, Y-axis acceleration data 131b, and Z-axis acceleration data 131c (hereinafter also referred to as vector data 131d) in the storage medium 104 as measurement data 131.

[0028] The data acquiring unit 111 also acquires, for example, behavioral data 132 indicating the behavior of the person at the time when each measurement data 131 was measured. The behavioral data 132 may include, for example, data indicating that the person was walking, data indicating that the person was working while sitting, data indicating that the person was resting while sitting, data indicating that the person was driving a car, or data indicating that the person was taking a bath. The data acquiring unit 111 then stores the acquired behavioral data 132 in the storage medium 104, for example.

[0029] The behavior data 132 may be data input to the information processing device 1 by a person or an operator of the information processing device 1, for example.

[0030] The data generation unit 112 generates, for example, a plurality of labeled teacher data DT1 each including measurement data 131 stored in the storage medium 104 and behavioral data 132 stored in the storage medium 104. That is, each of the plurality of labeled teacher data DT1 is teacher data including, for example, a so-called correct label. Then, the data generation unit 112 stores, for example, the generated plurality of labeled teacher data DT1 in the storage medium 104.

[0031] Specifically, the data generation unit 112 generates multiple labeled teacher data DT1, for example, for each type of measurement data 131 stored in the storage medium 104, so that the measurement data 131 corresponding to each type and the behavioral data 132 indicating the person's behavior at the time the measurement data 131 was measured are included in the same labeled teacher data DT1.

[0032] More specifically, for example, when five types of measurement data 131, namely, X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in storage medium 104, data generation unit 112 generates a plurality of labeled teacher data DT1 (hereinafter also referred to as a plurality of labeled teacher data DT1a) each including X-axis acceleration data 131a and behavior data 132, and a plurality of labeled teacher data DT1 (hereinafter also referred to as a plurality of labeled teacher data DT1b) each including Y-axis acceleration data 131b and behavior data 132. The system generates a plurality of labeled teacher data DT1 each including Z-axis acceleration data 131c and behavioral data 132 (hereinafter also referred to as a plurality of labeled teacher data DT1c), a plurality of labeled teacher data DT1 each including vector data 131d and behavioral data 132 (hereinafter also referred to as a plurality of labeled teacher data DT1d), and a plurality of labeled teacher data DT1 each including heart rate data 131 (hereinafter also referred to as heart rate data 131e) and behavioral data 132 (hereinafter also referred to as a plurality of labeled teacher data DT1e).

[0033] Furthermore, the data generation unit 112 generates a plurality of pieces of label-free measured data DT2 from each piece of measured data 131 stored in the storage medium 104, for example. That is, each piece of label-free measured data DT2 is data generated by, for example, performing necessary preprocessing on each piece of measured data 131. Then, the data generation unit 112 stores the generated plurality of pieces of label-free measured data DT2 in the storage medium 104, for example.

[0034] Specifically, the data generator 112 generates a plurality of unlabeled measured data DT2 so that, for example, each piece of measured data 131 measured at the same timing by each measuring device 2 is included in the same unlabeled measured data DT2.

[0035] More specifically, for example, if five types of measurement data 131, namely, X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in storage medium 104, data generation unit 112 generates a plurality of unlabeled measurement data DT2 each including X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e.

[0036] The first model generation unit 113 generates a plurality of learning models MD1, for example, by learning a plurality of labeled training data DT1 including measurement data 131 corresponding to each type of measurement data 131 stored in the storage medium 104. Then, the first model generation unit 113 stores the generated plurality of learning models MD1 in the storage medium 104, for example.

[0037] Specifically, for example, when five types of measurement data 131, namely, X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in the storage medium 104, the first model generation unit 113 generates a learning model MD1 (hereinafter also referred to as X-axis learning model MD1a) that has been trained with a plurality of labeled teacher data DT1a including the X-axis acceleration data 131a, and a learning model MD1b that has been trained with a plurality of labeled teacher data DT1b including the Y-axis acceleration data 131b. (hereinafter also referred to as Y-axis learning model MD1b), a learning model MD1 (hereinafter also referred to as Z-axis learning model MD1c) trained with a plurality of labeled teacher data DT1c including Z-axis acceleration data 131c, a learning model MD1 (hereinafter also referred to as vector learning model MD1d) trained with a plurality of labeled teacher data DT1d including vector data 131d, and a learning model MD1 (hereinafter also referred to as heart rate learning model MD1e) trained with a plurality of labeled teacher data DT1e including heart rate data 131e are generated.

[0038] The second model generation unit 114 generates a learning model MD2 including each of the multiple learning models MD1 generated by the first model generation unit 113, for example, by learning multiple unlabeled measurement data DT2 stored in the storage medium 104. Then, the second model generation unit 114 stores the generated learning model MD2 in the storage medium 104, for example. Each of the multiple learning models MD1 and MD2 may be, for example, a neural network.

[0039] Specifically, for example, if five types of measurement data 131, namely X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in the storage medium 104, the second model generation unit 114 inputs the X-axis acceleration data 131a contained in each of the multiple unlabeled measurement data DT2 into the X-axis learning model MD1a, inputs the Y-axis acceleration data 131b contained in each of the multiple unlabeled measurement data DT2 into the Y-axis learning model MD1b, inputs the Z-axis acceleration data 131c contained in each of the multiple unlabeled measurement data DT2 into the Z-axis learning model MD1c, inputs the vector data 131d contained in each of the multiple unlabeled measurement data DT2 into the vector learning model MD1d, and inputs the heart rate data 131e contained in each of the multiple unlabeled measurement data DT2 into the heart rate learning model MD1e.

[0040] In this case, the second model generation unit 114 tunes the parameters of a predetermined layer constituting each of the multiple learning models MD1 included in the learning model MD2, for example, so that the difference between the values ​​output from each of the multiple learning models MD1 included in the learning model MD2 becomes small. The predetermined layer may be, for example, a fully connected layer.

[0041] The learning model MD2 includes, for example, a layer (hereinafter also referred to as a post-processing layer) that aggregates multiple values ​​output from each of the multiple learning models MD1 included in the learning model MD2. Specifically, the post-processing layer identifies, for example, from among the behaviors (person's behaviors) indicated by the values ​​output from each of the multiple learning models MD1 in response to the input of each measurement data 131 included in the unlabeled measurement data DT2, behaviors that can be determined to be highly likely to have been performed by the person at the time each measurement data 131 was measured. Then, the learning model MD2 outputs, for example, behavior data 132 (hereinafter also referred to as behavior data 132a) indicating the behavior identified in the post-processing layer.

[0042] Next, the function for realizing the estimation process will be described.

[0043] The data acquisition unit 111 acquires, for example, each piece of measurement data 131 (new measurement data 131) measured by each measuring device 2 attached to a person.

[0044] The behavior estimation unit 115 inputs, for example, each piece of measurement data 131 acquired by the data acquisition unit 111 into a learning model MD2 stored in the storage medium 104.

[0045] Specifically, the behavior estimation unit 115 inputs, for example, each piece of measurement data 131 acquired by the data acquisition unit 111 to each of the multiple learning models MD1 included in the learning model MD2.

[0046] Then, the behavior estimation unit 115 acquires, for example, behavior data 132a output from the learning model MD2 in association with the input of each measurement data 131.

[0047] Specifically, in the post-processing layer, the learning model MD2 outputs, for example, behavior data 132a indicating behaviors (person's behaviors) indicated by values ​​output from each of the multiple learning models MD1 in response to input of each measurement data 131 that can be determined to be highly likely to have been performed by the person at the time when each measurement data 131 was measured. Then, the behavior estimation unit 115 acquires, for example, the behavior data 132a output from the learning model MD2.

[0048] The result output unit 116 outputs, for example, the behavior data 132a acquired by the behavior estimation unit 115 to an operation terminal (not shown).

[0049] That is, the information processing device 1 in this embodiment generates a plurality of learning models MD1 in advance, for example, by using a plurality of labeled training data DT1 including correct labels (for example, a small amount of labeled training data DT1). Then, the information processing device 1 in this embodiment generates a learning model MD2 including the plurality of learning models MD1 that have already been generated, for example. In other words, the information processing device 1 in this embodiment generates a learning model MD2 by, for example, transfer learning on the plurality of learning models MD1 that have already been generated.

[0050] As a result, the information processing device 1 in this embodiment can, for example, proceed with learning of the learning model MD2 (tuning of parameters of a predetermined layer in each learning model MD1 included in the learning model MD2) while comparing values ​​output from each of the multiple learning models MD1 included in the learning model MD2. Therefore, the information processing device 1 in this embodiment can, for example, perform learning of the learning model MD2 by using multiple unlabeled measurement data DT2 that do not include a correct answer label. Therefore, the information processing device 1 in this embodiment can, for example, eliminate the time required to generate unlabeled measurement data DT2 (the workload associated with adding correct answer labels to training data).

[0051] Furthermore, in the information processing device 1 of this embodiment, for example, by generating a learning model MD2 including multiple learning models MD1 generated in advance, it is possible to reduce the size of the learning model MD2 compared to when multiple learning models MD1 are not generated in advance (when transfer learning of multiple learning models MD1 is not performed). Therefore, in the information processing device 1 of this embodiment, it is possible to install the learning model MD2 in a smaller device compared to when multiple learning models MD1 are not generated in advance.

[0052] [Details of Learning Process in First Embodiment] Next, the learning process in the first embodiment will be described in detail. Figures 4 to 7 are flow charts illustrating the details of the learning process in the first embodiment. Figures 8 to 13 are diagrams illustrating the details of the learning process in the first embodiment.

[0053] [Measurement data storage processing] First, a description will be given of the process of storing the measurement data 131 (hereinafter also referred to as measurement data storage process) in the learning process in the first embodiment. Fig. 4 is a flowchart illustrating the measurement data storage process in the first embodiment.

[0054] As shown in FIG. 4, the data acquisition unit 111 waits until it receives, from each measuring device 2, measurement data 131 measured by each measuring device 2 attached to the person (NO in S1).

[0055] Then, when each measurement data 131 is received from each measurement device 2 (YES in S1), the data acquisition unit 111 stores, for example, each acquired measurement data 131 in the storage medium 104 (S2).

[0056] Specifically, the data acquisition unit 111 stores, in the storage medium 104, for example, the measurement data 131 measured at the same timing by each measurement device 2 in a manner that associates the data with each other.

[0057] [Behavioral Data Amnestic Processing] Next, a description will be given of the process of storing the behavior data 132 (hereinafter also referred to as measurement data storage process) in the learning process in the first embodiment. Fig. 5 is a flowchart illustrating the behavior data storage process in the first embodiment.

[0058] As shown in FIG. 5, the data acquiring unit 111 waits until it receives input of the behavior data 132 (NO in S11), for example.

[0059] Then, when the input of the behavior data 132 is accepted (YES in S11), the data acquiring unit 111 stores the accepted input of the behavior data 132 in the storage medium 104 (S12), for example.

[0060] Specifically, the data acquisition unit 111 stores the behavioral data 132 received as input in the processing of S11 in the storage medium 104 in a manner that associates the behavioral data 132 with each measurement data 131 measured at the same time by each measurement device 2.

[0061] [First model generation process] Next, a process for generating multiple learning models MD1 (hereinafter also referred to as a first model generation process) will be described in the learning process in the first embodiment. Fig. 6 is a flowchart illustrating the first model generation process in the first embodiment.

[0062] As shown in Figures 6 and 8(A), the data generation unit 112 generates, for example, for each type of measurement data 131 stored in the storage medium 104, multiple labeled training data DT1 each including the measurement data 131 stored in the storage medium 104 and the behavioral data 132 stored in the storage medium 104 (S21).

[0063] Specifically, the data generation unit 112 generates multiple labeled training data DT1, for example, for each type of measurement data 131 stored in the storage medium 104, so that the measurement data 131 corresponding to each type and the behavioral data 132 indicating the behavior of a person at the same time as the measurement data 131 were measured are included in the same labeled training data DT1.

[0064] More specifically, for example, if five types of measurement data 131, namely, X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in the storage medium 104, the data generation unit 112 generates a plurality of labeled teacher data DT1a each including the X-axis acceleration data 131a and the behavioral data 132, a plurality of labeled teacher data DT1b each including the Y-axis acceleration data 131b and the behavioral data 132, a plurality of labeled teacher data DT1c each including the Z-axis acceleration data 131c and the behavioral data 132, a plurality of labeled teacher data DT1d each including the vector data 131d and the behavioral data 132, and a plurality of labeled teacher data DT1e each including the heart rate data 131e and the behavioral data 132.

[0065] Then, as shown in Figure 8(B), the first model generation unit 113 generates multiple learning models MD1 by, for example, learning multiple labeled training data DT1 containing measurement data 131 corresponding to each type of measurement data 131 stored in the storage medium 104 (S22).

[0066] Specifically, for example, when five types of measurement data 131, namely, X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in the storage medium 104, the first model generation unit 113 generates an X-axis learning model MD1a trained with a plurality of labeled teacher data DT1a including the X-axis acceleration data 131a, a Y-axis learning model MD1b trained with a plurality of labeled teacher data DT1b including the Y-axis acceleration data 131b, a Z-axis learning model MD1c trained with a plurality of labeled teacher data DT1c including the Z-axis acceleration data 131c, a vector learning model MD1d trained with a plurality of labeled teacher data DT1d including the vector data 131d, and a heart rate learning model MD1e trained with a plurality of labeled teacher data DT1e including the heart rate data 131e.

[0067] Thereafter, the first model generation unit 113 stores, for example, the generated learning models MD1 in the storage medium 104 (S23).

[0068] [Second model generation process] Next, a process for generating a learning model MD2 (hereinafter also referred to as a second model generation process) will be described in the learning process in the second embodiment. Fig. 7 is a flowchart for explaining the second model generation process in the first embodiment.

[0069] As shown in FIGS. 7 and 9(A), the data generating unit 112 generates a plurality of unlabeled measurement data DT2 each including each measurement data 131 stored in the storage medium 104 (S31).

[0070] Specifically, the data generator 112 generates a plurality of unlabeled measured data DT2 so that, for example, each piece of measured data 131 measured at the same timing by each measuring device 2 is included in the same unlabeled measured data DT2.

[0071] More specifically, for example, if five types of measurement data 131, namely, X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in storage medium 104, data generation unit 112 generates a plurality of unlabeled measurement data DT2 each including X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e.

[0072] In addition, the data generation unit 112 may generate multiple unlabeled measurement data DT2 by using, for example, each measurement data 131 stored in the storage medium 104 that has not been used to generate multiple labeled teacher data DT1 in the processing of S21 (for example, each newly measured measurement data 131).

[0073] Then, the second model generation unit 114 generates a learning model MD2 including the multiple learning models MD1 generated in the processing of S22, for example, by training the multiple unlabeled measurement data DT2 stored in the storage medium 104, as shown in Figure 9(B) (S32).

[0074] Specifically, for example, if five types of measurement data 131, namely X-axis acceleration data 131a, Y-axis acceleration data 131b, Z-axis acceleration data 131c, vector data 131d, and heart rate data 131e, are stored in the storage medium 104, the second model generation unit 114 inputs the X-axis acceleration data 131a contained in each of the multiple unlabeled measurement data DT2 into the X-axis learning model MD1a, inputs the Y-axis acceleration data 131b contained in each of the multiple unlabeled measurement data DT2 into the Y-axis learning model MD1b, inputs the Z-axis acceleration data 131c contained in each of the multiple unlabeled measurement data DT2 into the Z-axis learning model MD1c, inputs the vector data 131d contained in each of the multiple unlabeled measurement data DT2 into the vector learning model MD1d, and inputs the heart rate data 131e contained in each of the multiple unlabeled measurement data DT2 into the heart rate learning model MD1e.

[0075] In this case, the second model generation unit 114 tunes the parameters of a predetermined layer (e.g., a fully connected layer) that constitutes the module added to the multiple learning models MD1 included in the learning model MD2, so as to reduce the difference in values ​​output from each of the multiple learning models MD1 included in the learning model MD2.

[0076] Thereafter, the second model generation unit 114 calculates a loss value by inputting values ​​output from each of the multiple learning models MD1 included in the learning model MD2 into a loss function (hereinafter also referred to as a predetermined function) (S33). Note that the loss function used in the processing of S33 may be, for example, a combination of one or more loss functions.

[0077] That is, in this case, the second model generation unit 114 calculates, for example, a loss value indicating the magnitude of the difference between the values ​​output from each of the multiple learning models MD1 included in the learning model MD2.

[0078] As a result, if the calculated loss value does not satisfy the predetermined condition (NO in S33), the second model generation unit 114 performs, for example, the processes from S32 onwards again. Specifically, if, for example, the calculated loss is less than a predetermined threshold value (hereinafter also simply referred to as the predetermined threshold value), the second model generation unit 114 performs, for example, the processes from S32 onwards again. That is, in this case, the second model generation unit 114 continues learning the learning model MD2, for example.

[0079] On the other hand, if the calculated loss value satisfies the predetermined condition (YES in S33), the second model generation unit 114 stores, for example, the learning model MD2 generated in the processing of S32 in the storage medium 104 (S34). Specifically, for example, the second model generation unit 114 performs the processing of S34 when the calculated loss value is equal to or greater than a threshold. That is, in this case, the second model generation unit 114 ends the learning of the learning model MD2, for example.

[0080] [Specific example of learning model MD1] Next, a specific example of the learning model MD1 will be described. Fig. 10 is a diagram for explaining a specific example of the learning model MD1.

[0081] As shown in FIG. 10, the learning model MD1 has, for example, from the input layer (not shown) to the output layer (not shown), a first convolution layer (hereinafter referred to as Conv), a first Maxpooling layer (hereinafter also referred to as Maxpooling), a second convolution layer, a second Maxpooling layer, an LSTM layer (hereinafter also referred to as LSTM), a Dropout layer (hereinafter also referred to as Dropout), a first Dense layer (hereinafter also referred to as Dense), and a second Dense layer in this order.

[0082] The learning model MD1 may be, for example, a learning model having layers other than those shown in FIG.

[0083] [Specific example of learning model MD2] Next, a specific example of the learning model MD2 will be described. Fig. 11 is a diagram for explaining a specific example of the learning model MD2.

[0084] The learning model MD2 has, for example, a plurality of learning models MD1 arranged in parallel and an additional module, as shown in Fig. 11. Then, each measurement data 131 included in the unlabeled measurement data DT2 input to the learning model MD2 is input to, for example, the first convolution layer in each of the plurality of learning models MD1, as shown in Fig. 11.

[0085] In addition, the learning model MD2 has a concatenation layer (hereinafter also referred to as Concatenate) that concatenates values ​​output from the Dropout layer in each of the multiple learning models MD1 included in the learning model MD2, for the purpose of improving the generalization performance of the learning model MD2, and inputs the concatenated values ​​to the first Dense layer in each of the multiple learning models MD1.

[0086] The learning model MD2 further includes a post-processing layer (hereinafter also referred to as "post-processing") that aggregates values ​​output from the second dense layers of each of the multiple learning models MD1. Specifically, the learning model MD2 identifies, in the post-processing layer, actions (person's actions) indicated by the values ​​output from the second dense layers of each of the multiple learning models MD1 that can be determined to be highly likely to have been performed by the person at the time when each measurement data 131 included in the unlabeled measurement data DT2 input to the learning model MD2 was measured. The learning model MD2 then outputs, for example, behavior data 132a indicating the actions identified in the post-processing layer.

[0087] In the learning process, the second model generation unit 114 may, for example, tune the parameters of the first dense layer and the second dense layer included in each of the multiple learning models MD1.

[0088] [Example of loss function (1)] Next, a first specific example of the loss function will be described. Fig. 12 is a diagram for explaining a specific example of the loss function. The following description will be given assuming that X-axis acceleration data 131a, Y-axis acceleration data 131b, and Z-axis acceleration data 131c are stored in storage medium 104 as measurement data 131. The following description will also be given assuming that X-axis learning model MD1a, Y-axis learning model MD1b, and Z-axis learning model MD1c are generated in the learning process. The following description will also be given assuming that the type of person's behavior is one of "walking," "working," and "resting."

[0089] In the process of S33, the second model generation unit 114 uses, for example, a loss function that outputs a smaller loss value as the difference between the values ​​output from two learning models MD1 out of the multiple learning models MD1 becomes smaller.

[0090] Specifically, the second model generation unit 114 calculates a value (Loss1) indicating the difference between the value output from the X-axis learning model MD1a and the value output from the Y-axis learning model MD1b by using, for example, the following formulas 1 and 2: XY ) is calculated.

[0091]

number

[0092]

number

[0093] In the above formulas 1 and 2, N indicates the number of unlabeled measurement data DT2 (i.e., the number of samples), C indicates the number (maximum number) of types of behavior that a person may perform, and α indicates a predetermined constant. x is a predicted probability of a person's behavior output from the X-axis learning model MD1a, as shown in FIG. 12, and indicates a predicted probability for each unlabeled measurement data DT2 (X-axis acceleration data 131a) input to the X-axis learning model MD1a and for each type of person's behavior, and p y is the predicted probability of the person's behavior output from the Y-axis learning model MD1b, and indicates the predicted probability for each unlabeled measurement data DT2 (Y-axis acceleration data 131b) input to the Y-axis learning model MD1b and for each type of person's behavior. x As shown in Figure 12, x indicates the value after OneHot encoding for each predicted probability included in yp y is p y The values ​​after OneHot encoding are shown for each predicted probability included in the above formulas 1 and 2. x As shown in Figure 12, x The values ​​after smoothing for each value included in are shown.

[0094] In addition, P shown in Figure 12 x In the above, the value of each row indicates, for example, the predicted probability for each unlabeled measurement data DT2 (X-axis acceleration data 131a) input to the X-axis learning model MD1a. xIn the above, the values ​​in the first row indicate the predicted probability when, for example, the first unlabeled measurement data DT2 (first X-axis acceleration data 131a) is input to the X-axis learning model MD1a, the values ​​in the second row indicate the predicted probability when, for example, the second unlabeled measurement data DT2 (second X-axis acceleration data 131a) is input to the X-axis learning model MD1a, and the values ​​in the third row indicate the predicted probability when, for example, the third unlabeled measurement data DT2 (third X-axis acceleration data 131a) is input to the X-axis learning model MD1a. Also, the P shown in FIG. x In the table, the values ​​in each column indicate, for example, the predicted probability for each type of person's behavior. x In the above, the values ​​in the first column indicate, for example, the predicted probability that a person's behavior is "walking" among the predicted probabilities when unlabeled measurement data DT2 (X-axis acceleration data 131a) is input to the X-axis learning model MD1a; the values ​​in the second column indicate, for example, the predicted probability that a person's behavior is "working" among the predicted probabilities when unlabeled measurement data DT2 (X-axis acceleration data 131a) is input to the X-axis learning model MD1a; and the values ​​in the third column indicate, for example, the predicted probability that a person's behavior is "resting" among the predicted probabilities when unlabeled measurement data DT2 (X-axis acceleration data 131a) is input to the X-axis learning model MD1a.

[0095] That is, P shown in FIG. x In the example, the value "0.8" in the first row and first column indicates the predicted probability that the person's behavior is "walking" among the predicted probabilities when the first unlabeled measurement data DT2 (the first X-axis acceleration data 131a) is input to the X-axis learning model MD1a. x In this example, the value "0.05" in the second column of the fifth row indicates, for example, the predicted probability that the person's behavior is "work" when the fifth unlabeled measurement data DT2 (the fifth X-axis acceleration data 131a) is input to the X-axis learning model MD1a.

[0096] In this case, the second model generation unit 114 calculates, for example, a value (Loss1) indicating the difference between the value output from the X-axis learning model MD1a and the value output from the Z-axis learning model MD1c. ZX ) and the value indicating the difference between the value output from the Y-axis learning model MD1b and the value output from the Z-axis learning model MD1c (Loss1 YZ ) is also calculated.

[0097] Thereafter, the second model generation unit 114 calculates the loss value (Loss1) in the process of S33 by using the following Equation 3.

[0098]

number

[0099] Then, in the process of S33, the second model generation unit 114 determines, for example, whether or not the calculated loss value is less than a threshold value.

[0100] [Example of loss function (2)] Next, a second specific example of the loss function will be described.

[0101] In the processing of S33, the second model generation unit 114 uses, for example, a loss function that outputs a smaller value the closer the number of types of behavior indicated by the predicted probabilities output from each of the multiple learning models MD1 (the number of types of behavior that can be determined to have been performed by a person) is to the number (maximum number) of types of behavior that a person may possibly perform.

[0102] Specifically, the second model generation unit 114 calculates the loss value (Loss2) in the process of S33 by using, for example, the following Equation 4.

[0103]

number

[0104] In the above formula 4, β represents a predetermined constant.X As shown in Figure 13, M indicates the number of types of behavior indicated by the predicted probability output from the X-axis learning model MD1a. Y As shown in Figure 13, Y-axis indicates the number of types of behaviors indicated by the predicted probability output from the learning model MD1b, and M Z As shown in FIG. 13, indicates the number of types of behavior indicated by the predicted probability output from the Z-axis learning model MD1c.

[0105] Thereafter, in the process of S33, the second model generation unit 114 determines, for example, whether or not the calculated loss value (Loss2) is less than a threshold value.

[0106] [Example of loss function (3)] Next, a third specific example of the loss function will be described.

[0107] In the process of S33, the second model generation unit 114 uses, for example, a function that outputs a smaller value as the difference between the predicted probability output from each learning model MD1 and the value determined by clustering the predicted probabilities output from the LSTM layer in each learning model MD1 becomes smaller. Note that the value determined by clustering the predicted probabilities output from the LSTM layer in each learning model MD1 is, for example, the value indicated by the largest cluster when clustering is performed on the predicted probabilities output from each learning model MD1 (a value indicating the center of gravity of the predicted probabilities of the values ​​classified into the largest cluster).

[0108] Specifically, the second model generation unit 114 uses, for example, the following formula 5 to calculate a loss value (Loss3) indicating the difference between the predicted probability output from the X-axis learning model MD1a and the value determined by clustering for the predicted probability output from the X-axis learning model MD1a. X ) is calculated.

[0109]

number

[0110] In the above formula 5, p l indicates the value determined by clustering for the predicted probability output from the LSTM layer in the X-axis learning model MD1a.

[0111] In this case, the second model generation unit 114 calculates a loss value (Loss3) indicating the difference between the predicted probability output from the Y-axis learning model MD1b and the value determined by clustering for the predicted probability output from the Y-axis learning model MD1b. Y In this case, the second model generation unit 114 also calculates a loss value (Loss3) that indicates the difference between the predicted probability output from the Z-axis learning model MD1c and the value determined by clustering for the predicted probability output from the Z-axis learning model MD1c. Z ) is also calculated.

[0112] Thereafter, the second model generation unit 114 calculates the loss value (Loss3) in the process of S33 by using the following equation 6.

[0113]

number

[0114] Then, the second model generating unit 114 determines, for example, whether or not the calculated loss value (Loss3) is less than a threshold value.

[0115] The second model generation unit 114 may use the sum of two or more values ​​of Loss1, Loss2, and Loss3 as the loss value (Loss), for example, as shown in the following Equations 7 to 9.

[0116]

number

[0117]

number

[0118]

number

[0119] That is, the second model generation unit 114 can prevent the results of behavior estimation by each learning model MD1 from being biased toward a specific type (a small number of types) of behavior, compared to, for example, when using the sum of Loss1 and Loss2 as the loss value, when using the sum of Loss1 and Loss3 as the loss value, when using the sum of Loss1, Loss2, and Loss3 as the loss value, or when using Loss1 as the loss value. In other words, in this case, the second model generation unit 114 can prevent, for example, each learning model MD1 from overfitting to a specific type of behavior.

[0120] [Details of Estimation Processing in the First Embodiment] Next, details of the estimation process in the first embodiment will be described. Fig. 14 is a flowchart illustrating details of the estimation process in the first embodiment. Fig. 15 and Fig. 16 are diagrams illustrating details of the estimation process in the first embodiment.

[0121] As shown in FIG. 14, the data acquiring unit 111 waits until it receives, from each measuring device 2, measurement data 131 (new measurement data 131) measured by each measuring device 2 attached to the person (NO in S41).

[0122] Then, when each measurement data 131 is received from each measurement device 2 (YES in S41), the behavior estimation unit 115 inputs, for example, each measurement data 131 acquired by the processing of S41 into each learning model MD2 stored in the storage medium 104 (S42), as shown in FIG. 15.

[0123] Specifically, the behavior estimation unit 115 inputs, for example, each piece of measurement data 131 acquired in the process of S41 to each of the multiple learning models MD1 included in the learning model MD2 stored in the storage medium 104.

[0124] Thereafter, as shown in FIG. 15, the behavior estimation unit 115 acquires, for example, behavior data 132a output from the learning model MD2 in conjunction with the input of each measurement data 131 in the process of S42 (S43).

[0125] Then, the result output unit 116 outputs, for example, the behavior data 132a acquired in the process of S43 (S44).

[0126] In addition, as shown in FIG. 16, the post-processing layer of the learning model MD2 may, for example, calculate the sum of the predicted probability output from the X-axis learning model MD1a, the predicted probability output from the Y-axis learning model MD1b, and the predicted probability output from the Z-axis learning model MD1c for each type of behavior of a person.Then, the post-processing layer may, for example, identify the behavior with the largest sum of calculated predicted probabilities among the types of behavior of the person.Then, the learning model MD2 may, for example, output behavior data 132a indicating the behavior identified in the post-processing layer (the behavior with the largest sum of predicted probabilities).

[0127] In this way, the information processing device 1 in this embodiment has, for example, a first model generation unit 113 that generates multiple learning models MD1 corresponding to each of multiple types of measurement data 131 for a specified object by learning labeled training data DT1 including each type of measurement data 131 and behavioral data 132 (correct label) corresponding to each type of measurement data 131, and a second model generation unit 114 that generates a second learning model MD2 including multiple learning models MD1 by learning unlabeled measurement data DT2 including the multiple types of measurement data 131.

[0128] Specifically, the second model generation unit 114 generates a learning model MD2 by, for example, inputting multiple types of measurement data 131 contained in the unlabeled measurement data DT2 into each of multiple learning models MD1, and learning parameters in a specified layer that constitutes each of the multiple learning models MD1 so that the difference in predicted probabilities output from each of the multiple learning models MD1 satisfies a specified condition.

[0129] That is, the information processing device 1 in this embodiment generates a plurality of learning models MD1 in advance by using, for example, a plurality of labeled training data DT1 including correct labels (for example, a small amount of labeled training data DT1). Then, the information processing device 1 in this embodiment generates, for example, a learning model MD2 including the plurality of generated learning models MD1.

[0130] As a result, in the information processing device 1 of this embodiment, for example, it is possible to proceed with the learning of the learning model MD2 (tuning of parameters of a predetermined layer in each learning model MD1 included in the learning model MD2) while comparing values ​​output from each of the multiple learning models MD1 included in the learning model MD2. Therefore, in the information processing device 1 of this embodiment, for example, it is possible to perform the learning of the learning model MD2 by using multiple unlabeled measurement data DT2 that do not include a correct answer label.

[0131] During the execution of the estimation process, the data generation unit 112 may generate new unlabeled measurement data (not shown) including each piece of measurement data 131 (new measurement data 131) acquired by the data acquisition unit 111. The second model generation unit 114 may further perform learning on the learning model MD2 by using the new unlabeled measurement data generated by the data generation unit 112, for example.

[0132] Specifically, in this case, the second data generation unit 112 may generate new unlabeled measurement data DT2 for each person from the measurement data 131 (new measurement data 131) measured from each person. The second model generation unit 114 may generate a learning model MD2 suitable for executing an estimation process for each person by performing learning using the new unlabeled measurement data DT2 corresponding to each person.

[0133] This makes it possible for the information processing device 1 in this embodiment to further improve the determination accuracy of the learning model MD2 even during the execution of the estimation process, for example. [Explanation of symbols]

[0134] 1: Information processing device 2: Measuring equipment 101:CPU 102: Memory 103: Communication equipment 104:Storage medium 105: Bus 111: Data acquisition section 112: Data generation unit 113: First model generation unit 114: Second model generation unit 115: Behavior estimation section 116: Result output section 131: Measurement data 132: Behavioral data DT1: Labeled training data DT2: Unlabeled measurement data MD1: Learning model MD2: Learning Model

Claims

1. generating a plurality of first learning models corresponding to each of a plurality of types of data for a predetermined target by learning labeled training data including each type of data and a label corresponding to each type of data; A learning program that causes a computer to execute a process of generating a second learning model including the plurality of first learning models by learning unlabeled measurement data that includes the plurality of types of data and does not include labels corresponding to the plurality of types of data, In the process of generating the second learning model, a learning program generates the second learning model by inputting the multiple types of data contained in the unlabeled measurement data into each of the multiple first learning models, and learning parameters in a specified layer that constitutes each of the multiple first learning models so that the difference in first values ​​output from each of the multiple first learning models satisfies a specified condition.

2. The learning program according to claim 1 , wherein the predetermined layer is one or more fully connected layers.

3. The learning program of claim 1, wherein the difference is a value calculated by inputting the plurality of types of data contained in the unlabeled measurement data into each of the plurality of first learning models and inputting the first value output from each of the plurality of first learning models into a predetermined function.

4. The learning program according to claim 3 , wherein the predetermined function includes a function that outputs a smaller value as the difference between the first values ​​output from each of the plurality of first learning models becomes smaller.

5. the predetermined object is a person, The learning program according to claim 1 , wherein each of the plurality of types of data is data measured from the person at a predetermined timing.

6. Furthermore, by inputting a plurality of types of new data about the predetermined target into each of the plurality of first learning models included in the second learning model, a plurality of types of values ​​output from the second learning model are obtained; Identifying new labels corresponding to the plurality of types of new data from the plurality of types of values ​​acquired; outputting the identified new label; 2. The learning program according to claim 1, which causes a computer to execute processing.

7. a first model generation unit that generates a plurality of first learning models corresponding to each of a plurality of types of data for a predetermined target by learning labeled training data including each type of data and a label corresponding to each type of data; a second model generation unit that generates a second learning model including the plurality of first learning models by learning unlabeled measurement data that includes the plurality of types of data and does not include labels corresponding to the plurality of types of data, The second model generation unit generates the second learning model by inputting the multiple types of data included in the unlabeled measurement data into each of the multiple first learning models, and learning parameters in a predetermined layer that constitutes each of the multiple first learning models so that the difference in first values ​​output from each of the multiple first learning models satisfies a predetermined condition.

8. generating a plurality of first learning models corresponding to each of a plurality of types of data for a predetermined target by learning labeled training data including each type of data and a label corresponding to each type of data; A learning method in which a computer executes a process of generating a second learning model including the plurality of first learning models by learning unlabeled measurement data that includes the plurality of types of data and does not include labels corresponding to the plurality of types of data, A learning method in which, in the process of generating the second learning model, the second learning model is generated by inputting the multiple types of data contained in the unlabeled measurement data into each of the multiple first learning models, and learning parameters in a specified layer that constitutes each of the multiple first learning models so that the difference in first values ​​output from each of the multiple first learning models satisfies a specified condition.

Images