Machine learning program, machine learning method, and information processing apparatus

Abstract

To improve the stability of a generating model during machine learning.SOLUTION: A machine learning program of an embodiment causes a computer to execute processing for acquiring third data, acquiring fourth data, and executing training of an encoder and a decoder. In the processing of acquiring the third data, first data is inputted to the encoder, and the third data obtained by adding noise to second data outputted by the encoder is acquired. In the processing of acquiring the fourth data, the third data is input to a decoder corresponding to the inverse calculation of the encoder, and the fourth data output by the decoder is acquired. The processing to be executed is to train the encoder and the decoder based on a loss function that includes the third data and an error between the first and fourth data.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] An embodiment of the present invention relates to a machine learning program, a machine learning method, and an information processing device. [Background technology]

[0002] Conventionally, in fields such as image and natural language processing, generative deep learning models (hereafter referred to as generative models or models) are used to generate latent representations (also called latent variables or latent spaces) that capture the characteristics of data. These generative models are trained by machine learning based on a large amount of unlabeled data. Generative models are also called autoencoders.

[0003] The generative model has an encoder and a decoder. By inputting input data to the encoder, a latent representation with a lower dimension and a smaller amount of data than the input data is generated. Also, by inputting the latent representation to the decoder, output data that is data that restores the input data is generated.

[0004] The encoder and decoder are trained using machine learning to reduce the reconstruction error between input data and output data. By inputting input data into the encoder of a trained generative model, a latent representation that captures the characteristics of the input data is obtained. In other words, a trained generative model has the role of quantitatively determining the characteristics of the input data, such as sampling the input data, generating features, generating explanatory variables, and acquiring the data distribution. Therefore, generative models can be used for prediction and classification based on input data. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2022-20138 [Patent Document 2] JP 2022-15573 A [Patent Document 3] Special Publication No. 2022-533264 [Patent Document 4] US Patent Application Publication No. 2019 / 0327501 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the above conventional technology, there is a problem that the convergence of the parameters of the encoder and the decoder deteriorates due to fluctuations in the reconstruction error during machine learning, and the machine learning is likely to become unstable. For example, when the number of parameters of the encoder and the decoder increases and the model size is large, it becomes difficult to converge the parameters.

[0007] In one aspect, an object of the present invention is to provide a machine learning program, a machine learning method, and an information processing device that can improve the stability during machine learning of a generative model. [Means for solving the problem]

[0008] In one proposal, the machine learning program causes a computer to execute a process of acquiring third data, a process of acquiring fourth data, and a process of executing. The process of acquiring third data includes inputting the first data to an encoder, and acquiring third data by adding noise to the second data output by the encoder. The process of acquiring fourth data includes inputting the third data to a decoder equivalent to the inverse calculation of the encoder, and acquiring fourth data output by the decoder. The process of executing executes training of the encoder and the decoder based on the third data and a loss function including the error between the first data and the fourth data. Effect of the Invention

[0009] It can improve the stability of generative models during machine learning. [Brief description of the drawings]

[0010] [Figure 1]FIG. 1 is an explanatory diagram for explaining a conventional model. [Diagram 2] FIG. 2 is an explanatory diagram for explaining the conventional model. [Diagram 3] FIG. 3 is an explanatory diagram for explaining a conventional model. [Figure 4] FIG. 4 is an explanatory diagram for explaining an outline of the model according to the embodiment. [Diagram 5] FIG. 5 is a block diagram illustrating an example of a functional configuration of the information processing device according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an operation example of the information processing device according to the embodiment. [Figure 7] FIG. 7 is an explanatory diagram illustrating an overview of a model according to a modified example. [Figure 8] FIG. 8 is a flowchart showing an example of the operation of the information processing device according to the modified example. [Figure 9] FIG. 9 is an explanatory diagram illustrating an example of a computer configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a machine learning program, a machine learning method, and an information processing device according to the embodiments will be described with reference to the drawings. Configurations having the same functions in the embodiments are given the same reference numerals, and duplicated descriptions will be omitted. Note that the machine learning program, the machine learning method, and the information processing device described in the following embodiments are merely examples, and do not limit the embodiments. In addition, the following embodiments may be appropriately combined within a range that does not contradict each other.

[0012] (Overview of the embodiment) First, an overview of the model according to the embodiment will be described while comparing it with a conventional model with reference to Figures 1 to 4. Figures 1 to 3 are explanatory diagrams for explaining the conventional model. Figure 4 is an explanatory diagram for explaining the overview of the model according to the embodiment.

[0013] As shown in Figure 1, a conventional model M101 is a Variational Auto Encoder (VAE). This model M101 is an encoder (f φ (x)) and a decoder (g θ (z)).

[0014] In model M101, the encoder (f φ (x) is the function that calculates f based on the parameter (φ) when input data (x) is entered. φ Calculate (x). For example, the encoder (f φ (x)) is f φ Based on the calculation result of (x), μ and σ are output. μ is the average of the calculation result. σ is the standard deviation of the calculation result.

[0015] In the model M101, the latent variable z is obtained by adding ε (noise ε) that follows a normal distribution of N(0,σI) to μ (I is a unit matrix). θ (z) is the function g based on the parameter θ when the latent variable z is input. θ (z) is calculated. θ (z)) is g θ The output data (x (hat)) which is the calculation result of (z) is output.

[0016] In the model M101, during machine learning using input data (x), the recovery error (D(x, x(hat))) between the input data (x) and the output data (x(hat)) and the amount of encoded information (R) are calculated.

[0017] The reconstruction error D is the distance between the input data (x) and the output data (x(hat)), and is calculated based on, for example, the cross entropy or the sum of squared differences.

[0018] The amount of coding information R is, for example, the Kullback-Leibler divergence (D KL ) and the probability distribution of the latent variable z (p φ (z)) and its prior distribution (q θ(z|x) is a quantification of the difference between

[0019] During machine learning, a conventional information processing device calculates the value of a loss function L of E including a recovery error D and an amount of encoded information R, and updates the parameters (θ, φ) of the encoder and decoder so as to reduce the value of the loss function L, as shown in the following formula (1). Here, β is a correction coefficient for the amount of encoded information R, and is a preset value.

[0020]

number

[0021] In this way, in the model M101, noise is added to the latent variables and optimization is performed using the recovery error D and the amount of encoded information R, so that clusters of information are formed in the latent space, forming a latent space with high interpretability.

[0022] As shown in FIG. 2, the conventional model M102 has an encoder (f φ (x)) and a decoder (g θ (z)) and an encoder (f φ The output of (x) may be used as the latent variable z. In the model M102, the amount of coded information R is calculated by the probability distribution (p φ (z)), R = -log(p φ In the model M102, the latent variable z is added with noise ε of N(0,(β / 2)I) and the data is calculated as θ (z)).

[0023] During machine learning of this model M102, a conventional information processing device calculates the value of a loss function L of E including a recovery error D and an amount of encoded information R, and adjusts the probability distribution (θ, φ, p φ ) to update the

[0024]

number

[0025] In this way, the model M102 can obtain a highly interpretable latent space (formation of information clusters, acquisition of explanatory variables, etc.) by performing "rate-distortion optimization" that reduces the amount of encoded information R at the same time as the reconstruction error D. However, in the above conventional models M101 and M102, the convergence of the encoder and decoder parameters (θ, φ) deteriorates due to fluctuations in the reconstruction error D, making machine learning prone to instability.

[0026] As shown in FIG. 3, the conventional model M103 includes a flow-based model. Specifically, in the model M103, the decoder is replaced by the encoder (f φ (x)) inverse function (f φ -1 (z)), and the encoder and decoder parameters φ(m,n) are shared.

[0027] During machine learning of this model M103, a conventional information processing device performs optimization so that the value of the loss function L becomes small, as shown in the following formula (3), and the encoder (f φ In such a flow-based model (model M103), the lossless encoder directly maximizes the likelihood of the input distribution.

[0028]

number

[0029] As a result, the flow-based model can obtain highly accurate data distribution and perform highly accurate sampling. However, since the flow-based model does not add noise to the latent variable z, it is difficult for information clusters to form in the latent space, resulting in a latent space with low interpretability.

[0030] Optimization of the conventional models M101 and M102 requires updating the encoder and decoder parameters (θ, φ) to reduce both the recovery error D and the amount of encoded information R in a balanced manner, which can easily make machine learning unstable (poor parameter convergence).

[0031] Here, the recovery error D for the conventional models M101 and M102 can be approximately expanded to a form including a term A (D(x, x(Floebe))) and a term B (D(x(Floebe), x(Hat))) as in the following formula (4). x(Floebe) in formula (4) refers to the case where noise is not added to the latent variable z.

[0032]

number

[0033] In the expansion of the above equation (4), the term A is g θ (x)=f φ -1 A is a term that promotes (x), and convergence becomes difficult as the model size increases. B is a term that preserves the characteristics of the input data (x), and converges early in the learning process. In other words, it was found that the convergence of A has a significant impact on the performance (convergence) of the conventional models M101 and M102.

[0034] The term A will be identically 0 if the encoder is a flow-based model (model M103). In other words, to stabilize learning, it is sufficient to incorporate a flow-based model, but since the latent space has low interpretability, there is no positive reason to use it in an autoencoder for a generative model.

[0035] As shown in FIG. 4, the model M1 according to the embodiment is an autoencoder capable of reversible calculation, similar to the flow-based model. The model M1 is an encoder (f φ Noise is added to the output of (x) to obtain a latent variable z, and the encoder (decoder) parameters (φ) are updated based on the amount of encoded information based on the latent variable z and the reconstruction error D.

[0036] Specifically, in model M1, the encoder (f φ (x) is the function that calculates f based on the parameter (φ) when input data (x) is entered. φ Calculate (x). For example, the encoder (f φ (x)) is f φ Based on the calculation result of (x), μ and σ are output. μ is the average of the calculation result. σ is the standard deviation of the calculation result.

[0037] Next, in model M1, we obtain the latent variable z by adding ε (noise ε) that follows a normal distribution of N(0,σI) to μ. In model M1, the decoder calculates f φ -1 (z) and the encoder (f φ This decoder is equivalent to the inverse calculation (inverse function) of f(x) based on the parameter φ when the latent variable z is input. φ -1 (z). The decoder calculates f φ -1 The output data (x (hat)) which is the calculation result of (z) is output.

[0038] The information processing device of the embodiment calculates the recovery error (D(x, x(hat))) between the input data (x) and the output data (x(hat)) and the amount of encoded information (R) during machine learning of a model M1 using input data (x).

[0039] The recovery error D for the model M1 is the distance between the input data (x) and the output data (x(hat)), similar to the models M101 and M102, and is calculated based on, for example, the cross entropy or the sum of squared differences.

[0040] The coding information amount R for model M1 is the same as for models M101 and M102, for example, the Kullback-Leibler divergence (D KL ) That is, the amount of coded information R for the model M1 is the probability distribution (p φ (z)) and its prior distribution (q θ(z|x) is a quantification of the difference between

[0041] The information processing device of the embodiment calculates the value of a loss function L of E including a recovery error D and an amount of encoded information R, and updates a parameter (φ) of the encoder (decoder) so that the value of the loss function L becomes smaller, as shown in the following equation (5).

[0042]

number

[0043] Therefore, in the machine learning of the model M1, the information processing device of the embodiment can realize highly accurate sampling, and since the term A in the recovery error D is identically 0, the convergence can be improved (stable learning can be performed). φ By adding noise to the output of (x) to create the latent variable z, clusters of information are formed in the latent space, making it possible to form a latent space with high interpretability.

[0044] (Embodiment) 5 is a block diagram showing an example of a functional configuration of an information processing device according to an embodiment. As shown in FIG. 5, the information processing device 1 includes a communication unit 10, an input unit 20, a display unit 30, a storage unit 40, and a control unit 50.

[0045] The communication unit 10 receives various data from an external device via a network. The communication unit 10 is an example of a communication device. For example, the communication unit 10 may receive model information 41, training data 42, and the like from the external device.

[0046] The input unit 20 is an input device that inputs various information to the control unit 50 of the information processing device 1. The input unit 20 corresponds to a keyboard, a mouse, a touch panel, etc. The display unit 30 is a display device that displays information output from the control unit 50.

[0047] The storage unit 40 stores data such as model information 41 and training data 42. The storage unit 40 corresponds to a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk drive (HDD).

[0048] The model information 41 includes parameters φ related to the above model M1, etc. The training data 42 is data used for machine learning of the model M1, such as image data and text data used for image and natural language processing, etc.

[0049] The control unit 50 includes a model construction unit 51, a model input unit 52, a recovery error calculation unit 53, a loss function calculation unit 54, and a model learning unit 55. The control unit 50 is realized by, for example, a central processing unit (CPU), a graphics processing unit (GPU), or hardwired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0050] The model construction unit 51 is a processing unit that constructs the model M1 according to the embodiment. Specifically, the model construction unit 51 constructs the encoder (f φ (x)) and the decoder (f φ -1 (z)) and

[0051] The model input unit 52 is a processing unit that inputs data to the model M1 constructed by the model construction unit 51. Specifically, the model input unit 52 reads out the training data 42 from the storage unit 40 during machine learning, and inputs the encoder (f φ Next, the model input unit 52 inputs the input data (x) to the encoder (f φThe model input unit 52 adds noise to the output (x') of the decoder (f φ -1 The input data is input to the decoder (z) and the output data (x) is obtained.

[0052] The recovery error calculation unit 53 is a processing unit that calculates a recovery error D(x, x(hat)) between the input data (x) and the output data (x(hat)). Specifically, the recovery error calculation unit 53 calculates the distance between the input data (x) and the output data (x(hat)) based on the cross entropy, the sum of squared differences, etc.

[0053] The loss function calculation unit 54 is a processing unit that calculates a loss function L of E, which includes a latent variable z and a recovery error D. Specifically, the loss function calculation unit 54 calculates the probability distribution (p φ (z)) and its prior distribution (q θ (z|x)) and the Kullback-Leibler divergence (D KL ) is calculated as the amount of encoded information R. Next, the loss function calculation unit 54 calculates the reconstruction error D and the Kullback-Leibler divergence (D KL ) and calculate the loss function L according to equation (5).

[0054] The model learning unit 55 is a processing unit that executes training (machine learning) of the encoder and decoder in the model M1 based on the calculation result of the loss function L. Specifically, the model learning unit 55 updates the parameters (φ) of the encoder (decoder) so as to reduce the value of the loss function L, as shown in formula (5). Then, the model learning unit 55 stores the updated parameter values ​​in the model information 41.

[0055] 6 is a flowchart showing an example of the operation of the information processing device 1 according to the embodiment. It is assumed that the model M1 has already been constructed by the model construction unit 51 before the start of the process shown in FIG.

[0056] As shown in FIG. 6, when the process starts, the model input unit 52 reads the training data 42 and inputs the encoder (f φ (x)). As a result, the information processing device 1 inputs the encoder f φ The latent variable (μ) and the standard deviation σ of the noise for the input (x) are obtained by (S1). Next, the model input unit 52 adds the noise ε to the latent variable μ to obtain the latent variable z (S2).

[0057] Next, the model input unit 52 inputs the latent variable z to a decoder (f φ -1 The input is input to the decoder (z) and converted from z to the output (x(hat)) (S3).

[0058] Next, the restoration error calculation unit 53 calculates the restoration error D(x, x(hat)) between the input (x) and the output (x(hat)) (S4).

[0059] After S2, the loss function calculation unit 54 calculates the amount of encoded information R (S5). Also, after S4, the loss function calculation unit 54 calculates the loss function L based on the recovery error D and the amount of encoded information R. φ Calculate (x) = D + βR (S6).

[0060] Next, the model learning unit 55 calculates the loss function L φ Based on the calculation result of (x), the loss function L φ The parameter (φ) of the encoder (decoder) is updated so that the value of (x) becomes smaller (S7).

[0061] Next, the model learning unit 55 determines whether or not the process is to be terminated based on whether or not the termination condition is satisfied (S8). Here, the termination condition indicates a condition for terminating the machine learning regarding the model M1. For example, the termination condition may be φ This includes cases where the value of (x) is less than a predetermined threshold value, where the processes of S1 to S7 are repeated a predetermined number of times, and so on.

[0062] If the processing is to be ended (S8: Yes), the model learning unit 55 stores the updated parameter values ​​in the model information 41 and ends the processing. If the processing is not to be ended (S8: No), the model learning unit 55 returns the processing to S1.

[0063] (Modification) Next, a modified example of the above embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is an explanatory diagram for explaining an outline of a model according to the modified example. Fig. 8 is a flowchart showing an operation example of the information processing device 1 according to the modified example.

[0064] As shown in FIG. 7, in the model M2 according to the modified example, the input x is multi-channelized (M21) as an input (x'), and the encoder (f φ Specifically, the model input unit 52 copies the input x by the number of channels to make it multi-channel, and inputs it to the encoder. For example, if the input x is an image and the number of channels is N, an input (x') in which N images are arranged in the channel direction is input to the encoder. This results in a multi-channel latent variable z.

[0065] Next, the model input unit 52 outputs the multi-channel latent variable z to a decoder (f φ -1 The model input unit 52 converts this output (x'(hat)) into a single channel (M22) to obtain the output (x(hat)). Note that the conversion into a single channel may involve arbitrarily selecting one channel from the multi-channel output (x'(hat)), or may involve finding the average value of all channels and integrating them into a single channel.

[0066] As in FIG. 6, it is assumed that the model M1 has already been constructed by the model construction unit 51 before the processing shown in FIG.

[0067] 8, when the process starts, the model input unit 52 reads the training data 42 and sets it as input data (x). Next, the model input unit 52 multi-channelizes the input (x) to arrange N pieces in the channel direction to obtain input data (x') (S11).

[0068] Next, the model input unit 52 inputs the input data (x') to the encoder (f φ (x)). As a result, the information processing device 1 inputs the encoder f φ The latent variable (μ) and the standard deviation σ of the noise for the input (x′) are obtained by (S12). Next, the model input unit 52 adds the noise ε to the latent variable μ to obtain the latent variable z (S13).

[0069] Next, the model input unit 52 inputs the latent variable z to a decoder (f φ -1 The input is input to a decoder (z) and converted from z to the output (x' (hat)) (S14).

[0070] Next, the model input unit 52 extracts a single-channel output (x(hat)) from the multi-channel output (x'(hat)) by converting it into a single channel. Next, the restoration error calculation unit 53 calculates the restoration error D(x, x(hat)) between the input (x) and the output (x(hat)) (S15).

[0071] After S13, the loss function calculation unit 54 calculates the amount of encoded information R (S16). Also, after S15, the loss function calculation unit 54 calculates the loss function L based on the recovery error D and the amount of encoded information R. φ Calculate (x) = D + βR (S17).

[0072] Next, the model learning unit 55 calculates the loss function L φ Based on the calculation result of (x), the loss function L φ The parameter (φ) of the encoder (decoder) is updated so that the value of (x) becomes smaller (S18).

[0073] Next, similarly to S8, the model learning unit 55 determines whether or not the process is to be terminated based on whether or not the termination condition is satisfied (S19). If the process is to be terminated (S19: Yes), the model learning unit 55 stores the updated parameter values ​​in the model information 41 and terminates the process. If the process is not to be terminated (S19: No), the model learning unit 55 returns the process to S11.

[0074] As described above, the information processing device 1 inputs the input data (x) to the encoder of the model (M1) and obtains data (z) by adding noise to the data (x') output by the encoder. The information processing device 1 inputs the data (z) to a decoder that corresponds to the inverse calculation of the encoder and obtains the output data (x(hat)) output by the decoder. The information processing device 1 trains the encoder and decoder in the model (M1) based on the data (z) and a loss function including the error between the input data (x) and the output data (x(hat)).

[0075] Therefore, in the machine learning of the model (M1), the information processing device 1 can improve convergence (can learn stably) because the term A in the recovery error D in formula (4) is identically 0. In addition, in the model (M1), data (latent variable z) is generated by adding noise to the output of the encoder, so that a cluster of information is formed in the latent space, and a latent space with high interpretability can be formed.

[0076] Furthermore, the information processing device 1 multi-channelizes the input data (x) and inputs it to an encoder to obtain multi-channelized data (z). The information processing device 1 inputs the multi-channelized data (z) to a decoder, converts the output of the decoder into a single channel, and obtains output data (x (hat)). In this way, the information processing device 1 can efficiently advance machine learning by multi-channelizing the input data and performing machine learning.

[0077] Note that each component of each device shown in the figure does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0078] Moreover, various processing functions of the model construction unit 51, the model input unit 52, the recovery error calculation unit 53, the loss function calculation unit 54, and the model learning unit 55 performed in the control unit 50 of the information processing device 1 may be executed in whole or in part on a CPU (or a microcomputer such as an MPU or an MCU (Micro Controller Unit)). Needless to say, various processing functions may be executed in whole or in part on a program analyzed and executed by the CPU (or a microcomputer such as an MPU or an MCU), or on hardware based on wired logic. Various processing functions performed in the information processing device 1 may be executed by multiple computers working together through cloud computing.

[0079] Incidentally, the various processes described in the above embodiment can be realized by executing a program prepared in advance on a computer. Therefore, an example of a computer configuration (hardware) that executes a program having the same functions as those of the above embodiment will be described below. Figure 9 is an explanatory diagram for explaining an example of a computer configuration.

[0080] 9, the computer 200 includes a CPU 201 that executes various arithmetic processes, an input device 202 that accepts data input, a monitor 203, and a speaker 204. The computer 200 also includes a medium reading device 205 that reads programs and the like from a storage medium, an interface device 206 for connecting to various devices, and a communication device 207 for connecting to external devices by wire or wireless communication. The information processing device 1 also includes a RAM 208 that temporarily stores various information, and a hard disk device 209. Each unit (201 to 209) in the computer 200 is connected to a bus 210.

[0081] The hard disk drive 209 stores a program 211 for executing various processes in the functional configurations (e.g., the model construction unit 51, the model input unit 52, the restoration error calculation unit 53, the loss function calculation unit 54, and the model learning unit 55) described in the above embodiment. The hard disk drive 209 also stores various data 212 referenced by the program 211. The input device 202, for example, accepts input of operation information from an operator. The monitor 203, for example, displays various screens operated by the operator. The interface device 206 is connected to, for example, a printing device. The communication device 207 is connected to a communication network such as a LAN (Local Area Network), and exchanges various information with external devices via the communication network.

[0082] The CPU 201 reads out the program 211 stored in the hard disk device 209, expands it in the RAM 208, and executes it to perform various processes related to the above-mentioned functional configuration (for example, the model construction unit 51, the model input unit 52, the recovery error calculation unit 53, the loss function calculation unit 54, and the model learning unit 55). The program 211 does not have to be stored in the hard disk device 209. For example, the program 211 stored in a storage medium readable by the computer 200 may be read out and executed. The storage medium readable by the computer 200 corresponds to, for example, a portable recording medium such as a CD-ROM or a DVD disk, a USB (Universal Serial Bus) memory, a semiconductor memory such as a flash memory, a hard disk drive, etc. Also, the program 211 may be stored in a device connected to a public line, the Internet, a LAN, etc., and the computer 200 may read out and execute the program 211 from the device.

[0083] Regarding the above embodiment, the following supplementary notes are further disclosed.

[0084] (Note 1) First data is input to an encoder, and third data is obtained by adding noise to second data output by the encoder; inputting the third data into a decoder corresponding to the inverse calculation of the encoder, and obtaining fourth data output by the decoder; training the encoder and the decoder based on a loss function including the third data and an error between the first data and the fourth data; A machine learning program that causes a computer to execute processing.

[0085] (Note 2) The process of acquiring the third data includes acquiring the third data by multi-channelizing the first data and inputting the multi-channelized third data to the encoder; The process of acquiring the fourth data includes inputting the multi-channel third data to the decoder, and acquiring the fourth data by converting an output of the decoder into a single channel. 2. The machine learning program according to claim 1,

[0086] (Additional Note 3) The loss function adds a value obtained by multiplying the coding information of the probability distribution and the prior distribution of the third data by a correction coefficient to the error. 2. The machine learning program according to claim 1,

[0087] (Appendix 4) Inputting the first data into an encoder, and adding noise to the second data output by the encoder to obtain third data; inputting the third data into a decoder corresponding to the inverse calculation of the encoder, and obtaining fourth data output by the decoder; training the encoder and the decoder based on a loss function including the third data and an error between the first data and the fourth data; A machine learning method characterized in that processing is executed by a computer.

[0088] (Additional Note 5) The process of acquiring the third data includes acquiring the third data by multi-channelizing the first data and inputting the multi-channelized third data to the encoder; The process of acquiring the fourth data includes inputting the multi-channel third data to the decoder, and acquiring the fourth data by converting an output of the decoder into a single channel. 5. The machine learning method according to claim 4,

[0089] (Additional Note 6) The loss function adds a value obtained by multiplying the encoded information of the probability distribution and the prior distribution of the third data by a correction coefficient to the error. 5. The machine learning method according to claim 4,

[0090] (Appendix 7) Inputting first data into an encoder, and adding noise to second data output by the encoder to obtain third data; inputting the third data into a decoder corresponding to the inverse calculation of the encoder, and obtaining fourth data output by the decoder; training the encoder and the decoder based on a loss function including the third data and an error between the first data and the fourth data; 1. An information processing device comprising: a control unit that executes processing.

[0091] (Appendix 8) The process of acquiring the third data includes acquiring the third data by multi-channelizing the first data and inputting the multi-channelized third data to the encoder; The process of acquiring the fourth data includes inputting the multi-channel third data to the decoder, and acquiring the fourth data by converting an output of the decoder into a single channel. 8. The information processing device according to claim 7,

[0092] (Additional Note 9) The loss function adds a value obtained by multiplying the encoded information of the probability distribution and the prior distribution of the third data by a correction coefficient to the error. 8. The information processing device according to claim 7, [Explanation of symbols]

[0093] 1...Information processing device 10. Communications Department 20...Input section 30...Display section 40...Storage section 41…Model information 42…Training data 50...Control unit 51…Model Construction Section 52…Model input section 53...Restoration error calculation unit 54...Loss function calculation section 55…Model Learning Section 200…Computer 201…CPU 202...Input device 203…Monitor 204…Speaker 205... Media reader 206...Interface device 207...Communication equipment 208…RAM 209...Hard disk drive 210…Bus 211…Program 212...Various data M1, M2, M101~M103…Models M21…Multi-channel M22…Single channel

Claims

1. inputting the first data into an encoder, and obtaining third data by adding noise to the second data output by the encoder; inputting the third data into a decoder corresponding to an inverse calculation of the encoder, and obtaining fourth data output by the decoder; training the encoder and the decoder based on a loss function including the third data and an error between the first data and the fourth data; A machine learning program that causes a computer to execute processing.

2. The process of acquiring the third data includes converting the first data into a multi-channel data and inputting the multi-channel data into the encoder to acquire the multi-channel third data; The process of acquiring the fourth data includes inputting the multi-channel third data to the decoder, and acquiring the fourth data by converting an output of the decoder into a single channel.

2. The machine learning program according to claim 1 .

3. The loss function adds a value obtained by multiplying encoded information of the probability distribution and the prior distribution of the third data by a correction coefficient to the error.

2. The machine learning program according to claim 1 .

4. inputting the first data into an encoder, and obtaining third data by adding noise to the second data output by the encoder; inputting the third data into a decoder corresponding to an inverse calculation of the encoder, and obtaining fourth data output by the decoder; training the encoder and the decoder based on a loss function including the third data and an error between the first data and the fourth data; A machine learning method characterized in that processing is executed by a computer.

5. inputting the first data into an encoder, and obtaining third data by adding noise to the second data output by the encoder; inputting the third data into a decoder corresponding to an inverse calculation of the encoder, and obtaining fourth data output by the decoder; training the encoder and the decoder based on a loss function including the third data and an error between the first data and the fourth data; 1. An information processing device comprising: a control unit that executes processing.

Images