Prediction apparatus with noise removal, prediction method with noise removal, and program

Abstract

To provide a technique for removing noise included in an explanatory variable which is input to a prediction model.SOLUTION: A prediction apparatus with noise removal includes: a first data acquisition unit which acquires data comprising one or more explanatory variables and objective variable; a learning data generation unit which generates learning data by adding noise to values of at least part of the explanatory variables out of the one or more explanatory variables included in the data; a training unit which trains a denoising autoencoder which uses the data as training data and receives as inputs at least the values of the one or more explanatory variables included in the learning data; a second data acquisition unit which acquires prediction target data comprising the one or more explanatory variables; a noise removal unit which removes noise from the values of the one or more explanatory variables included in the prediction target data by the trained denoising autoencoder; and a prediction unit which predicts a value of the objective variable by means of a prediction model, using the denoised prediction target data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a prediction device with noise removal, a prediction method with noise removal, and a program. [Background technology]

[0002] Data used for prediction and learning of AI (artificial intelligence) and the like may contain information unrelated to the prediction, which is called noise. When noise is present in the data used for prediction and learning of AI and the like, the prediction model may learn information unrelated to the prediction, which may result in a decrease in prediction accuracy. In response to this, there is known a conventional technology that can robustly train a model in the presence of label noise (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-548952 Summary of the Invention [Problem to be solved by the invention]

[0004] However, for example, the conventional technique described in Patent Document 1 does not take into consideration the case where noise is present in explanatory variables other than the label (that is, the objective variable).

[0005] The present disclosure has been made in consideration of the above points, and provides a technology that can remove noise present in explanatory variables input to a prediction model. [Means for solving the problem]

[0006] A prediction device with noise removal according to one embodiment of the present disclosure includes a first data acquisition unit that acquires data consisting of one or more explanatory variables and a dependent variable; a learning data creation unit that creates learning data by adding noise to values ​​of at least some of the one or more explanatory variables included in the data; a learning unit that uses the data as teacher data and trains a denoising autoencoder that has as input at least the values ​​of the one or more explanatory variables included in the learning data; a second data acquisition unit that acquires prediction target data consisting of the one or more explanatory variables; a noise removal unit that removes noise from values ​​of the one or more explanatory variables included in the prediction target data by the trained denoising autoencoder; and a prediction unit that predicts the value of the dependent variable by a prediction model using the prediction target data after noise removal. Effect of the Invention

[0007] A technique is provided that can remove noise present in explanatory variables input to a prediction model. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating an example of a hardware configuration of a prediction device with noise removal according to the present embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of a prediction device with noise removal according to the present embodiment. [Diagram 3] FIG. 1 is a diagram showing an example (part 1) of a noise removal model. [Figure 4] FIG. 13 is a diagram showing an example (part 2) of a noise removal model. [Diagram 5] 10 is a flowchart illustrating an example of a noise removal model learning process according to the present embodiment. [Figure 6] 5 is a flowchart illustrating an example of a noise removal process according to the present embodiment. [Figure 7] FIG. 13 is a diagram showing an example of data distribution in “A) No DAE performed.” [Figure 8]FIG. 13 is a diagram showing an example of data distribution in “C) DAE performed (proposed method)”. [Figure 9] FIG. 13 is a diagram showing an example of comparing data distributions in “A) without DAE” and “C) with DAE (proposed method).” DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment of the present invention will be described below. In the following embodiment, a prediction device with noise removal function 10 will be described, which can remove noise present in explanatory variables input to a prediction model and then perform prediction using the prediction model. Hereinafter, the prediction model will be referred to as f.

[0010] <Hardware Configuration Example of the Noise Removal Prediction Device 10> An example of the hardware configuration of a prediction device with noise removal 10 according to this embodiment is shown in Fig. 1. As shown in Fig. 1, the prediction device with noise removal 10 according to this embodiment includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a random access memory (RAM) 105, a read only memory (ROM) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware are connected to each other via a bus 109 so as to be able to communicate with each other.

[0011] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the prediction device with noise removal 10 does not necessarily have to include at least one of the input device 101 and the display device 102.

[0012] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0013] The communication I / F 104 is an interface for connecting the prediction device with noise removal 10 to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The processor 108 is various types of arithmetic device such as a central processing unit (CPU) or a general purpose computing with graphic processing unit (GPGPU).

[0014] Note that the hardware configuration shown in FIG. 1 is just an example, and the prediction device with noise removal 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various types of hardware other than the hardware shown in the figure.

[0015] <Example of Functional Configuration of Noise Removal Prediction Device 10> An example of the functional configuration of the prediction device with noise removal 10 according to this embodiment is shown in Fig. 2. As shown in Fig. 2, the prediction device with noise removal 10 according to this embodiment has a first data acquisition unit 201, a noise creation unit 202, a learning data creation unit 203, a learning unit 204, a second data acquisition unit 205, a noise removal unit 206, and a prediction unit 207. Each of these units is realized by, for example, processing in which one or more programs installed in the prediction device with noise removal 10 are executed by a processor 108 or the like.

[0016] The first data acquisition unit 201 acquires a first data set provided to the prediction device with noise removal 10. The first data set is a data set used for learning a noise removal model for removing noise present in explanatory variables input to a prediction model. Hereinafter, the first data set will be referred to as D 1 ={d (i) |i=1, ,N}, where d (i) =(x 1 (i) ,x n (i) ,y (i) ) is the first data set D 1 N is the i-th data in the first data set D 1 represents the number of data included in x 1 (i) ,x n (i) is the explanatory variable x of the i-th data 1 ,x n and y (i) represents the value of the objective variable y of the i-th data.

[0017] Here, the noise removal model is assumed to be a model realized by DAE (Denoising Autoencoder). DAE is a model that trains by adding noise to the input of an autoencoder to remove noise. In the following, DAE(x;θ)=D(E(x;θ) E );θ D) where D(·) is the decoder, E(·) is the encoder, θ is the learnable parameter of the DAE, and θ D are the learnable parameters of the decoder, θ E represents the learnable parameters of the encoder. However, the notation of each learning parameter may be omitted. For details of DAE, see, for example, Reference 1.

[0018] The noise generator 202 generates a first data set D 1 Each data included in (i) Each explanatory variable value x 1 (i) ,x n (i) Noise ε to be added to 1 (i) , ,ε n (i) The noise generator 202 generates the noise ε 1 (i) , ,ε n (i) For example, we can create a noise ε j' (i) (j'∈{1, ,n}), or we can create a j (i) Let us consider a certain noise ε j' (i) (j'∈{1, ,n}), or for some explanatory variable value x j (i) (i=1, ,N) statistics (e.g., median, mean, maximum, minimum, etc.) are computed with some noise ε j' (i) (j'∈{1, ,n}). Note that ε 1 (i) , ,ε n (i) Some of the noise in the data may be zero (i.e., there may be explanatory variables that are not assigned noise). (i) Every noise ε 1 (i) , ,ε n (i)may be created and all data d (i) The common noise ε 1 , ,ε n may be created.

[0019] The learning data creation unit 203 creates a first data set D 1 and noise ε 1 (i) , ,ε n (i) (i=1, ,N) for each data d (i) Each explanatory variable value x 1 (i) ,x n (i) For noise ε 1 (i) , ,ε n (i) That is, the learning data creation unit 203 creates a learning data set D by assigning x j '(i) ←x j (i) +ε j (j=1, ,n) and noise are added to create a training dataset D={d' (i) |i=1, ,N}, where d' (i) =(x 1 '(i) ,x n '(i) ,y (i) ).

[0020] The learning unit 204 uses a first data set D 1 and the learning data set D. Here, the learning unit 204 learns the noise removal model DAE(·) by the following learning method 1 or 2.

[0021] Learning Method 1 (x 1 '(i) ,x n '(i) ,dy (i) ) is the input data of the noise removal model DAE(·), (x1 (i) ,x n (i) ,y (i) ) is used as training data to train a noise removal model DAE(·). Here, dy is a dummy objective variable (hereinafter referred to as a dummy objective variable). As the value of the dummy objective variable dy, for example, the value of an explanatory variable that is considered to have a high correlation with the objective variable y may be used as is, or a predicted value of the objective variable using a prediction model created using a physical model, simple regression, multiple regression, or other algorithm may be used.

[0022] As a specific example, suppose the objective variable y represents "electricity demand" and a certain explanatory variable x j If "temperature" is represented, "electricity demand" and "temperature" are considered to be correlated, so the dummy response variable value dy (i) As the explanatory variable value x j (i) or x j '(i) It is possible to use the following.

[0023] In addition, for example, a prediction model created using a physical model, simple regression, multiple regression, or other algorithms can be used to calculate y=g(x 1 ,x n ), the dummy objective variable value dy (i) As g(x 1 (i) ,x n (i) ) or g(x 1 '(i) ,x n '(i) ) can be used.

[0024] The output data of the encoder E(·) is (z 1 ,...,z m), when learning method 1 is used, the noise removal model DAE(·) is given by the DAE shown in Figure 3. Note that the number of dimensions of the output data m is a hyperparameter that can be set by the user, etc. In addition to this, for example, the activation function of the noise removal model DAE(·), the number of mini-batches during learning, the number of epochs, etc. are also hyperparameters that can be set by the user, etc.

[0025] Learning Method 2 (x 1 '(i) ,···,x n '(i) ) is the input data of the noise removal model DAE(·), (x 1 (i) ,···,x n (i) ,y (i) ) is used as training data to train the noise removal model DAE(·).

[0026] The output data of the encoder E(·) is (z 1 ,...,z m ), when learning method 2 is used, the noise removal model DAE(·) is given by the DAE shown in Figure 4. Note that the number of dimensions of the output data m is a hyperparameter that can be set by the user, etc. In addition to this, for example, the activation function of the noise removal model DAE(·), the number of mini-batches during learning, the number of epochs, etc. are also hyperparameters that can be set by the user, etc.

[0027] Both of the above learning methods 1 and 2 use the objective variable y as training data. Therefore, according to the above learning methods 1 and 2, it is possible to obtain a noise removal model DAE(·) that also takes into account the objective variable y.

[0028] The second data acquisition unit 205 acquires a second data set or prediction target data provided to the prediction device with noise removal 10. Here, the second data set is provided at the time of learning the prediction model f, and the prediction target data is provided at the time of prediction (inference) of the prediction model f. The second data set is a data set used for learning the prediction model f. Hereinafter, the second data set is referred to as D2 ={d (k) |k=1, ,K}, where d (k) =(x 1 (k) ,x n (k) ,y (k) ) is the second data set D 2 where K is the kth data in the second data set D 2 represents the number of data included in x 1 (k) ,x n (k) is the explanatory variable x of the kth data 1 ,x n and y (k) represents the value of the objective variable y of the kth data. On the other hand, the data to be predicted is (p) =(x 1 (p) ,x n (p) ) where x 1 (p) ,x n (p) is the data to be predicted d (p) Explanatory variables x 1 ,x n respectively.

[0029] When learning the prediction model f, the noise removal unit 206 uses the noise removal model DAE(·) to remove the second data set D 2 Each data included in (k) Each explanatory variable value x 1 (k) ,x n (k) That is, when the noise removal model DAE(·) is trained by the above-mentioned training method 1, the noise removal unit 206 removes noise from each data d (k) Each explanatory variable value x 1 (k) ,x n (k) and the dummy response variable value dy (k) and are input to the noise removal model DAE(·), and each of these explanatory variable values ​​x 1(k) ,x n (k) On the other hand, when the noise removal model DAE(·) is trained by the above-mentioned training method 2, the noise removal unit 206 performs the noise removal process for each data d (k) Each explanatory variable value x 1 (k) ,x n (k) are input to the noise removal model DAE(·), and each of these explanatory variable values ​​x 1 (k) ,x n (k) Remove noise from the signal.

[0030] When predicting the prediction model f, the noise removal unit 206 uses the noise removal model DAE(·) to remove the prediction target data d (p) Each explanatory variable value x 1 (p) ,x n (p) That is, when the noise removal model DAE(·) is trained by the above-mentioned training method 1, the noise removal unit 206 removes noise from the prediction target data d (p) Each explanatory variable value x 1 (p) ,x n (p) and the dummy response variable value dy (p) and are input to the noise removal model DAE(·), and each of these explanatory variable values ​​x 1 (p) ,x n (p) On the other hand, when the noise removal model DAE(·) is trained by the above-mentioned training method 2, the noise removal unit 206 removes noise from the prediction target data d (p) Each explanatory variable value x 1 (p) ,x n (p) are input to the noise removal model DAE(·), and each of these explanatory variable values ​​x 1 (p) ,x n (p) Remove noise from the signal.

[0031] When the prediction model f is being trained, the prediction unit 207 calculates the second data set D after noise removal. 2 For example, the prediction unit 207 uses each data d after noise removal to learn a prediction model f. (k) Each explanatory variable value x 1 (k) ,x n (k) is input to the prediction model f, and the objective variable value y (k) The prediction model f is trained using the training data.

[0032] When making a prediction using the prediction model f, the prediction unit 207 calculates the prediction target data d after removing noise. (p) Each explanatory variable value x 1 (p) ,x n (p) is input into the prediction model f to predict the value of the dependent variable y.

[0033] In the above, the prediction model f takes explanatory variable values ​​after noise removal as input, but this is just an example, and the input of the prediction model f is not limited to this. For example, the prediction model f may take both explanatory variable values ​​before noise removal and explanatory variable values ​​after noise removal as input. In addition, for example, the prediction model f may take the hidden state (i.e., z 1 ,...,z m The method may take as input the value of

[0034] <Noise removal model learning process> An example of the noise removal model learning process according to this embodiment will be described below with reference to Fig. 5. Note that, although the learning of the prediction model f and the prediction (inference) will be described below with reference to the same flowchart, the learning of the prediction model f is executed before the prediction.

[0035] First, the first data acquisition unit 201 acquires a first data set D 1 (Step S101).

[0036] Next, the noise generator 202 generates a first data set D 1 Each data included in (i) Each explanatory variable value x 1 (i) ,x n (i) Noise ε to be added to 1 (i) , ,ε n (i) is created (step S102).

[0037] Next, the learning data creation unit 203 creates a first data set D 1 and noise ε 1 (i) , ,ε n (i) (i=1, ,N) for each data d (i) Each explanatory variable value x 1 (i) ,x n (i) For noise ε 1 (i) , ,ε n (i) Thus, the learning data set D is created (step S103). (i) |i=1,···,N}, d' (i) =(x 1 '(i) ,x n '(i) ,y (i) ) is obtained.

[0038] Then, the learning unit 204 uses the first data set D 1 Using the learning dataset D, the noise removal model DAE(·) is learned by the above learning method 1 or 2 (step S104). Note that a known supervised learning method may be used to learn the noise removal model DAE(·), and a known method may also be used as the optimization method. For example, Adam or the like may be used as the optimization method.

[0039] <Noise removal processing> An example of the noise removal process according to this embodiment will be described below with reference to FIG.

[0040] First, the second data acquisition unit 205 acquires the second data set D when the prediction model f is being trained. 2 , when prediction model f is used for prediction (inference), the prediction target data d (p) is obtained (step S201).

[0041] Next, the noise removal unit 206 performs the noise removal model DAE(·) on the second data set D 2 Each data d included in (k) Each explanatory variable value x 1 (k) ,···,x n (k) When the prediction model f is used for prediction (inference), the noise is removed and the prediction target data d is (p) Each explanatory variable value x 1 (p) ,···,x n (p) Then, noise is removed (step S202).

[0042] Then, the prediction unit 207 uses the second data set D after noise removal when learning the prediction model f. 2 A prediction model f is trained using the above, and when the prediction model f is used for prediction (inference), the prediction target data d after noise removal is (p) The value of the objective variable y is predicted using (step S203).

[0043] <Examples and their evaluation> In this example, we used a data set showing minute-by-minute changes in power consumption in a certain household, which is described in Reference 2. From this data set, we calculated hourly average values ​​from January 1, 2007 to December 31, 2009, and used these as learning and validation data for prediction model f. The data items are shown in Table 1 below.

[0044] [Table 1] Here, t represents the current time, and ti represents i hours ago. The objective variable is Global_active_power (the minute-by-minute average of active power for the entire household) one hour later.

[0045] Moreover, data from January 1, 2007 to December 31, 2008 was used as training data for prediction model f, and data from January 1, 2009 to December 31, 2009 was used as validation data for prediction model f. At this time, the training data and validation data were normalized to a minimum value of 0 and a maximum value of 1 before use. Furthermore, gradient boosting was used for prediction model f.

[0046] For comparison, A) no DAE was performed, B) DAE was performed (however, the training data did not include a target variable), and C) DAE was performed (proposed method). Note that C is the method proposed in this embodiment.

[0047] When training the DAE, noise sampled from a uniform distribution ranging from -0.05 to 0.05 was added in A) and C), and the current Global_active_power was used as a dummy objective variable in C).The number of input dimensions of the DAE was 108 in A) and 109 in C), the number of dimensions of the intermediate layer was 75, the activation function was relu, the number of epochs was 50, the number of mini-batches was 80, and Adam was used as the optimization algorithm for training.

[0048] Under the above conditions, prediction model f was trained using the training data for A, B, and C, and then the prediction accuracy of prediction model f was calculated using the validation data. The mean absolute error was used as the index value for prediction accuracy. In this case, the results were 0.373 for A), 0.388 for B), and 0.365 for C), with C achieving the best prediction accuracy.

[0049] Additionally, to confirm the effect of C), the distribution of voltage and the objective variable is shown in Figs. 7 to 9 as an example of data distribution. Fig. 7 shows A) the data distribution without DAE, Fig. 8 shows C) the data distribution with DAE (proposed method), and Fig. 9 shows the data distribution without DAE A) overlaid on the data distribution with DAE C) (proposed method). As shown in Figs. 7 to 9, it can be seen that the variability in the data is reduced in C) with DAE (proposed method).

[0050] <Summary> As described above, the prediction device 10 with noise removal according to this embodiment learns the DAE as a noise removal model using training data including a response variable. As a result, the prediction device 10 with noise removal according to this embodiment obtains a noise removal model that can perform noise removal while also taking into account the relationship with the response variable, and as a result, it becomes possible to realize highly accurate prediction of the prediction model f by noise removal using this noise removal model.

[0051] The present invention is not limited to the above-described embodiments specifically disclosed, and various modifications, changes, combinations with known technologies, etc. are possible without departing from the scope of the claims.

[0052] [References] Reference 1: Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol, "Extracting and Composing Robust Features with Denoising Autoencoders". 2008. Reference 2: Georges Hebrail, Alice Berard, "Individual household electric power consumption Data Set", 2012-08-30. [Explanation of symbols]

[0053] 10 Predictor with noise removal 101 Input Device 102 Display device 103 External I / F 103a Recording media 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage 108 processors 109 Bus 201 First Data Acquisition Unit 202 Noise Creation Section 203 Learning Data Creation Department 204 Learning Department 205 Second Data Acquisition Unit 206 Noise Removal Section 207 Prediction Department

Claims

1. A first data acquisition unit that acquires data consisting of one or more explanatory variables and a response variable; a learning data creation unit that creates learning data by adding noise to values ​​of at least some of the one or more explanatory variables included in the data; A learning unit that uses the data as training data and learns a denoising autoencoder that uses as input at least values ​​of one or more explanatory variables included in the learning data; A second data acquisition unit that acquires prediction target data configured with the one or more explanatory variables; a noise removal unit that removes noise from values ​​of one or more explanatory variables included in the prediction target data by the denoising autoencoder after the learning; a prediction unit that predicts a value of the objective variable by a prediction model using the prediction target data after the noise has been removed; A prediction device with noise removal comprising:

2. The denoising autoencoder 2. The prediction device with noise removal according to claim 1, wherein values ​​of the one or more explanatory variables and a value of a dummy dependent variable representing a variable correlated with the dependent variable are input.

3. The dummy response variable is 3. The prediction device with noise removal according to claim 2, wherein the explanatory variable is any one of the one or more explanatory variables, or a variable representing a predicted value of a model that predicts the value of the objective variable using values ​​of the one or more explanatory variables as input.

4. 4. The prediction device with noise removal function according to claim 3, wherein the model is any one of a physical model, a simple regression model, a multiple regression model, and a model realized by a predetermined prediction algorithm.

5. a first data acquisition step of acquiring data consisting of one or more explanatory variables and a response variable; a learning data creation step of creating learning data by adding noise to values ​​of at least some of the one or more explanatory variables included in the data; A learning procedure for learning a denoising autoencoder using the data as training data and inputting at least values ​​of one or more explanatory variables included in the learning data; A second data acquisition step of acquiring prediction target data composed of the one or more explanatory variables; a denoising step of removing noise from values ​​of one or more explanatory variables included in the prediction target data by the trained denoising autoencoder; a prediction step of predicting a value of the objective variable by a prediction model using the prediction target data after the noise removal; A prediction method with noise removal performed by a computer.

6. a first data acquisition step of acquiring data consisting of one or more explanatory variables and a response variable; a learning data creation step of creating learning data by adding noise to values ​​of at least some of the one or more explanatory variables included in the data; A learning procedure for learning a denoising autoencoder using the data as training data and inputting at least values ​​of one or more explanatory variables included in the learning data; A second data acquisition step of acquiring prediction target data composed of the one or more explanatory variables; a denoising step of removing noise from values ​​of one or more explanatory variables included in the prediction target data by the trained denoising autoencoder; a prediction step of predicting a value of the objective variable by a prediction model using the prediction target data after the noise removal; A program that causes a computer to execute the following.

Images