Training device, training method, inference device, inference method, detection device, detection method, and program

Abstract

An information processing device (10) according to the present invention includes a creation unit (152), an update unit (153), and a detection unit (154). The creation unit (152) creates an objective function that decreases as the amount of frequency components of a signal input to a state space model that is at least one layer of the deep learning model increases. The update unit (153) updates the parameters of the deep learning model so that the objective function is optimized. On the basis of the amount of frequency components of the signal input to the state space model, the detection unit (154) detects that the training data of the deep learning model is insufficient or the distribution of the data input to the deep learning model differs from the distribution of the training data used for training the deep learning model.

Description

Learning device, learning method, inference device, inference method, detection device, detection method, and program

[0001] The present invention relates to a learning device, a learning method, an inference device, an inference method, a detection device, a detection method, and a program.

[0002] Deep learning models (e.g., deep neural networks) have achieved great success in image recognition, speech recognition, and the like (see, for example, Non-Patent Document 1).

[0003] For example, a technology is known in which, when voice data of a short command is input into a deep learning model containing a large number of nonlinear functions, the deep learning model outputs an identification result as to which command the voice represents.

[0004] In particular, deep learning models incorporating a state space model can directly handle raw speech data with large sequence lengths (see, for example, Non-Patent Document 2 or Non-Patent Document 3).

[0005] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.p22-p26Albert Gu, Karan Goel, and Christopher R´e. ``Efficiently modeling long sequences with structured state spaces.'' arXiv preprint arXiv:2111.00396 (2021).Jimmy TH Smith, Andrew Warrington, and Scott W. Linderman. “Simplified state space layers for sequence modeling.” arXiv preprint arXiv:2208.04933 (2022).

[0006] However, conventional techniques have the problem that the accuracy of deep learning models that incorporate state space models may not be sufficient.

[0007] State space models are used as intermediate layers in deep learning models. Deep learning models with state space models can accurately learn the relationship between past inputs and current outputs in long-term time series data.

[0008] However, with conventional techniques, it is difficult to evaluate whether the input signals input to the state space model of each layer of a deep learning model are appropriate. As a result, with conventional techniques, appropriate input signals may not be input to the state space model, resulting in a decrease in the accuracy of the entire deep learning model.

[0009] In order to solve the above-mentioned problems and achieve the objectives, the learning device is characterized by having a creation unit that creates an objective function that becomes smaller as the amount of frequency components of a signal input to a state space model, which is at least one layer of a deep learning model, increases, and an update unit that updates the parameters of the deep learning model so that the objective function is optimized.

[0010] The inference device is characterized in that it performs inference using a deep learning model in which at least one layer is a state space model, and in which parameters are advanced so that an objective function that becomes smaller as the amount of frequency components of the signal input to the state space model increases is optimized.

[0011] The detection device is characterized by having a detection unit that detects, based on the amount of frequency components of a signal input to a state space model, which is at least one layer of the deep learning model, that there is a lack of training data for the deep learning model, or that the distribution of the data input to the deep learning model is different from the distribution of the training data used to learn the deep learning model.

[0012] According to the present invention, it is possible to improve the accuracy of a deep learning model that has a state space model built therein.

[0013] FIG. 1 is a diagram illustrating an example of the configuration of an information processing apparatus according to a first embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a deep learning model. FIG. 3 is a flowchart illustrating the flow of a learning process. FIG. 4 is a flowchart illustrating the flow of a process for creating a loss function. FIG. 5 is a flowchart illustrating the flow of a process for inspecting training data. FIG. 6 is a flowchart illustrating the flow of a process for inspecting training data. FIG. 7 is a flowchart illustrating the flow of a process for inspecting inference data. FIG. 8 is a diagram illustrating an example of a computer that executes a program.

[0014] Hereinafter, embodiments of a learning device, a learning method, an inference device, an inference method, a detection device, a detection method, and a program according to the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0015] [Configuration of First Embodiment] The configuration of an information processing device according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing device according to the first embodiment. An information processing device 10 functions as a learning device, an inference device, or a detection device.

[0016] The information processing device 10 performs a process (learning process) to optimize a deep learning model using training data (hereinafter referred to as training data). In this case, the information processing device 10 functions as a learning device. For example, the training data is data in which explanatory variables and objective variables are associated with each other. The deep learning model estimates the objective variable based on the input explanatory variables.

[0017] The information processing device 10 obtains an inference result by inputting inference data into a trained deep learning model (inference processing). In this case, the information processing device 10 functions as an inference device. For example, the training data is an explanatory variable whose corresponding objective variable is unknown.

[0018] The information processing device 10 detects that the training data in the learning process is insufficient and that the inference data is outside the distribution of the training data used in the learning. In this case, the information processing device 10 functions as a detection device. For example, the detection results by the detection device are fed back to the learning process and the inference process and used to adjust the training data and the inference data. In this way, the detection device contributes to improving the accuracy of the deep learning model.

[0019] As shown in FIG. 1, the information processing device 10 includes a communication unit 11 , an input unit 12 , an output unit 13 , a storage unit 14 , and a control unit 15 .

[0020] The communication unit 11 performs data communication with other devices via a network. For example, the communication unit 11 is a network interface card (NIC).

[0021] The input unit 12 receives input of data. For example, the input unit 12 is an interface connected to input devices such as a mouse and a keyboard.

[0022] The output unit 13 outputs data. The output unit 13 is an interface connected to an output device such as a display and a speaker.

[0023] The storage unit 14 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. Note that the storage unit 14 may also be a data-rewritable semiconductor memory such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM). The storage unit 14 stores an operating system (OS) and various programs executed by the information processing device 10.

[0024] The storage unit 14 stores model information 141. The model information 141 is parameters for constructing a model. For example, the model information 141 is parameters such as weights and biases of each layer of a deep learning model.

[0025] The control unit 15 controls the entire information processing device 10. The control unit 15 is, for example, an electronic circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0026] The control unit 15 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory. The control unit 15 also functions as various processing units when various programs are run. For example, the control unit 15 functions as a calculation unit 151, a creation unit 152, an update unit 153, and a detection unit 154.

[0027] The calculation unit 151 inputs data into a deep learning model constructed based on the model information 141 and calculates the output of the deep learning model. In the inference process, the output of the deep learning model calculated by the calculation unit 151 becomes the inference result.

[0028] The creation unit 152 creates a loss function for the learning process. The creation unit 152 can create the loss function using the output of the deep learning model and the correct answer (target label) included in the training data.

[0029] The update unit 153 updates the parameters of the deep learning model (model information 141) so that the loss function created by the creation unit 152 is optimized.

[0030] The following describes in detail the processing performed by the information processing device 10. First, the configuration of the deep learning model will be described with reference to Fig. 2. Fig. 2 is a diagram illustrating an example of the configuration of the deep learning model.

[0031] Data input to each layer of a deep learning model is called a signal. As shown in Figure 2, a deep learning model has an input layer where a signal is first input, intermediate layers (1st to Lth layers) that convert the signal output from the input layer, and a final layer that converts the signal output from the last intermediate layer (Lth layer) into an output such as a probability. Here, L is an integer greater than or equal to 1. When L = 1, the only intermediate layer is the first layer.

[0032] 3 is a flowchart showing the flow of the learning process. As shown in FIG. 3, first, the calculation unit 151 applies data points randomly selected from a data set to a deep learning model (step S101).

[0033] Here, the dataset is training data, which is a set of combinations of data points and target labels. The data points may be referred to as data points, samples, explanatory variables, etc. The target labels may be referred to as correct labels, objective variables, etc. Applying data points to a deep learning model means inputting the data points as signals to the input layer.

[0034] Next, the creation unit 152 calculates a loss function using the output of the deep learning model and the target label of the dataset (step S102). Then, the update unit 153 updates the parameters of the deep learning model using the gradient of the loss function (step S103). The update unit 153 can update the parameters so as to reduce the loss function by a known method such as backpropagation.

[0035] If the evaluation criteria are met (Yes in step S104), the information processing device 10 ends the learning process. On the other hand, if the evaluation criteria are not met (No in step S104), the information processing device 10 returns to step S101 and repeats the process.

[0036] The evaluation criteria may be based on the response of the deep learning model with updated parameters to a pre-prepared test dataset, or may be whether the parameter updates have been performed a specified number of times, whether the parameter updates have converged, etc.

[0037] Here, the deep learning model in this embodiment includes a state space model. Furthermore, in order to improve the accuracy of the deep learning model including the state space model, the information processing device 10 creates a loss function using a method different from conventional methods.

[0038] The learning process in this embodiment will be described in detail below. In the following description, capital bold letters represent matrices, and lowercase bold letters represent column vectors. Row vectors are expressed using transpose.

[0039] This embodiment can be generally used for prediction, classification, etc. using a deep learning model that has an internal state space model, regardless of the data and task. For example, the tasks of the deep learning model to which this embodiment is applied are not limited to image recognition and speech recognition.

[0040] [Regarding State Space Models and PE Properties] By using a state space model as an intermediate layer of a deep learning model, it is possible to accurately learn the relationship between past inputs and current outputs in long-term time-series data.

[0041] The time to be processed in the time series data is assumed to be t, and the signal input to this state space model is assumed to be u ｔ ∈R ｄｉｎ Then, the state space model is expressed by equations (1) and (2).

[0042]

[0043]

[0044] Here, x ｔ ∈R ｎ is the state (input signal) of the state space model. ｔ ∈R ｄｏｕｔ is the output (output signal) of the state space model. ｔ is input to the next layer. ｎ×ｎ , B ｎ×ｄｉｎ , C ｄｏｕｔ×ｎ , D ｄｏｕｔ×ｄｉｎ are model parameters. That is, A, B, C, and D are included in the model information 141 and are updated by the update unit 153.

[0045] To identify a state-space model, the input signal must be able to sufficiently excite the system modes. This is evaluated by the persistent excitation (PE) of the input signal. PE is determined by the rank of the autocorrelation matrix consisting of the autocorrelation function of the input signal. A single frequency is a second-order PE-like signal. Depending on the dimensions of the states and input / output of the state-space model, a signal with appropriate PE is required.

[0046] [Learning Time-Series Data by Deep Learning] The flow of the learning process is generally as described in Fig. 3. Here, the flow of the learning process will be described in detail.

[0047] A deep learning model consists of multiple layers consisting of linear and nonlinear transformations, and calculations are performed in each layer. Now, let us consider the internal state of the lth layer of the intermediate layer as x∈R. ｄ ｌ Then, the state x of the l+1th layer ｌ＋１ is as shown in equation (3).

[0048]

[0049] where θ ｌ is a parameter of the l-th layer of the deep learning model. When dealing with time series data, the input signal and state depend on time, so the state x at time t of the l+1-th layer is ｌ＋１，ｔ is as shown in equation (4).

[0050]

[0051] If the l-th layer is a state space model, then from equation (1), the l-th layer has (5) internally for the state at time t. Also, the input signal input to the l+1-th layer is as shown in equation (6).

[0052]

[0053]

[0054] In addition, the parameter θ of the deep learning model is １ Ｔ , …, θ Ｌ Ｔ ] Ｔis optimized by learning from training data. Note that the superscript T represents transposition. The creation unit 152 can create the loss function L(θ) as shown in equation (7).

[0055]

[0056] The loss function is, for example, cross-entropy, negative log-likelihood, mean square error, etc. The creation unit 152 further creates a loss function that takes PE properties into consideration based on the loss function of equation (7).

[0057] [Learning Process Considering PE Property] As described above, the input signal needs to have sufficient PE property in the state space model. Ｔ = [u １ , u ２ , ..., u Ｔ ] PE property of φ(U Ｔ ) and φ(U Ｔ ) and a function that has a positive correlation with Ｔ ) ψ(U Ｔ ) is large enough to accurately estimate the parameters of the state-space model.

[0058] The flow of processing for creating a loss function will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of processing for creating a loss function.

[0059] First, the calculation unit 151 selects a data point (step S201), inputs the selected data point into the deep learning model, and generates an input signal X Ｔ (Step S202). For example, X Ｔ is the input signal input to the state space model of the l-th layer of the intermediate layer of the deep learning model, and X Ｔ = [x １，ｌ , x ２，ｌ , ..., x Ｔ，ｌ The creation unit 152 creates X Ｔ ψ(X Ｔ ) is calculated (step S203).

[0060] Then, the creation unit 152 applies ψ(X Ｔ ) input g(ψ(X Ｔ)) is obtained (step S204). The creation unit 152 adds a regularization term λ(g(ψ(X Ｔ ))) and obtain the sum as the loss function (Equation (8)) (step S205).

[0061]

[0062] where λ∈R is a hyperparameter that balances the original loss function (Equation (7)) and the evaluation of PE-likelihood. Furthermore, when the update unit 153 performs optimization based on the gradient, g(·) and ψ(·) are differentiable functions. As described above, according to this embodiment, it is possible to increase the PE-likelihood of signals input to the state space model of the intermediate layer.

[0063] [Example Using Discrete Fourier Transform] Various examples based on this embodiment can be considered depending on how g(·) and ψ(·) are chosen. Here, a method using a Discrete Fourier Transform (DFT) will be described, taking advantage of the fact that the number of frequency components contained in the input signal correlates with the magnitude of the PE characteristic.

[0064] Here, a function for evaluating the PE property for each element of the input signal is considered based on the index of the PE property of a one-input one-output system. ｔ，ｌ The i-th component of x ｔ，ｌ ｉ First, the creation unit 152 normalizes the input signal with the L2 norm as shown in equation (9).

[0065]

[0066] However, X Ｔ，ｌ ｉ = [x １，ｌ ｉ , x ２，ｌ ｉ , ..., x Ｔ，ｌ ｉ Next, the creation unit 152 obtains equation (10) by DFT, where s=1, . . . , T.

[0067]

[0068] In this embodiment, the creation unit 152 focuses on the number of frequency components as an index of PE characteristics, and therefore creates a function ψ(·) that has a positive correlation with PE characteristics by focusing on the sum of amplitudes as in equation (11).

[0069]

[0070] Here, the number of frequency components is the number of frequencies that have an amplitude greater than 0 when DFT is performed. The larger the number of frequency components, the larger the sum of the amplitudes. The function ψ(·) can be said to be a function that represents the component quantities, such as the number of frequency components and the sum of the amplitudes.

[0071] In order to reduce the amount of calculation, the creation unit 152 calculates |x from s to T in equation (11). ｓ，ｌ ｉ Instead of taking the sum of all |x ｓ，ｌ ｉ | in descending order ｌ The sum of the selected numbers may be taken.

[0072] Furthermore, the creation unit 152 creates a differentiable and monotonically decreasing function g(·) by －ｃｘ Then, equation (12) is obtained.

[0073]

[0074] Here, cεR is a parameter that adjusts the strength of the penalty (regularization).

[0075] From the above, in this embodiment, the creation unit 152 creates a loss function such as that shown in equation (13).

[0076]

[0077] In this way, the creating unit 152 creates an objective function that decreases as the sum of amplitudes obtained by performing a discrete Fourier transform on the signals input to the state space model increases.

[0078] A state-space model can model linear systems. For example, if the system being modeled has a high order and complex frequency characteristics, the system can be modeled with higher accuracy if the signal input to the state-space model reflects information on multiple frequencies corresponding to the order. As a result, the state-space model allows deep learning models to perform more accurate inferences (predictions, classifications, etc.).

[0079] As mentioned above, the better the PE property of the input signal input to the state-space model, the more it can excite the system modes, resulting in better representation of multidimensional signals in the state-space model and improved accuracy of the deep learning model.

[0080] Therefore, the creation unit 152 creates an objective function that decreases as the amount of frequency components of a signal input to a state space model, which is at least one layer of a deep learning model, increases.

[0081] The amount of frequency components may be represented by the number of frequencies contained in the signal or by the sum of amplitudes. For example, the greater the number or sum of non-zero terms obtained by performing a discrete Fourier transform on the input signal, the greater the amount of frequency components in the input signal. The greater the amount of frequency components in the input signal, the better the PE performance.

[0082] According to the objective functions shown in equations (8) and (13), the information processing device 10 can obtain an input signal with sufficient PE properties to train a deep learning model that has a state space model internally, thereby improving the accuracy of the deep learning model.

[0083] [Detection process] The PE property of an input signal is related to the identifiability of a state-space model. The number of frequencies contained in the signal also affects the PE property. Furthermore, when training data is input to a deep learning model, the frequencies contained in the input signal of the intermediate layer are related to the frequency characteristics of the learned state-space model.

[0084] Therefore, the information processing device 10 (detection unit 154) detects, based on the amount of frequency components of the signal input to the state space model, which is at least one layer of the deep learning model, that there is a shortage of training data for the deep learning model, or that the distribution of the data input to the deep learning model differs from the distribution of the training data used to learn the deep learning model.

[0085] [Detection of insufficient training data] Fig. 5 is a flowchart showing the flow of processing for inspecting training data. As shown in Fig. 5, first, the detection unit 154 determines N, which is the required PE property, from the dimensions of the state space model included in the deep learning model and the dimensions of the input and output (step S301).

[0086] Next, when the training data points are input, the detection unit 154 calculates the autocorrelation function for the input signal input to the state space model of the l-th layer, and obtains an autocorrelation matrix (step S302). Subsequently, the detection unit 154 performs singular value decomposition on the autocorrelation matrix, and sorts the N-th singular value σ Ｎ is calculated (step S303).

[0087] In addition, the detection unit 154 detects σ Ｎ For example, the magnitude of σ in step S305 is calculated for the entire training data (step S304). Ｎ may be the sum of singular values ​​calculated by the method of step S303 for all state space models included in the deep learning model.

[0088] Here, the detection unit 154 detects σ Ｎ If the threshold value is less than the threshold value (Yes at step S305), the detecting unit 154 determines that there is insufficient training data (step S306).

[0089] On the other hand, the detection unit 154 detects σ Ｎ If the threshold value is equal to or greater than the threshold value (No at step S305), it is determined that there is sufficient training data (step S307). In this case, the detection unit 154 does not detect a shortage of training data.

[0090] In reality, singular value decomposition involves errors, making it difficult to accurately calculate ranks, and it may be difficult to evaluate the PE nature as shown in Fig. 5. Therefore, the detection unit 154 may inspect the training data using the method shown in Fig. 6.

[0091] 6 is a flowchart showing the flow of the process of inspecting training data. As shown in Fig. 6, first, the detection unit 154 determines N, which is the required PE property, from the dimensions of the state space model included in the deep learning model and the dimensions of the input and output (step S401).

[0092] Next, the detection unit 154 performs a Fourier transform on the input signal input to the state space model of the first layer when a training data point is input during learning (step S402). Subsequently, the detection unit 154 performs a Fourier transform on the N / 2-th value F Ｎ／２ is calculated (step S403).

[0093] In addition, the detection unit 154 detects F Ｎ／２ For example, the magnitude of F in step S405 is calculated for the entire training data (step S404). Ｎ／２ is the F calculated by the method of step S403 for all state space models included in the deep learning model. Ｎ／２ It may be the sum of

[0094] Here, the detection unit 154 detects F Ｎ／２ If the threshold value is less than the threshold value (Yes at step S405), the detecting unit 154 determines that there is insufficient training data (step S406).

[0095] On the other hand, the detection unit 154 detects F Ｎ／２ If the threshold value is equal to or greater than the threshold value (No at step S405), it is determined that there is sufficient training data (step S407). In this case, the detection unit 154 does not detect a shortage of training data.

[0096] 7 is a flowchart showing the flow of processing for inspecting inference data. First, when a data point is input during learning, the detection unit 154 performs a Fourier transform on the input signal of sequence length T input to the state space model of the first layer (step S501).

[0097] Next, the detection unit 154 calculates the amplitude F=[F １ , ..., F Ｔ ] is averaged over the entire training data to calculate ^F (^ just above F) (step S502).

[0098] Furthermore, when a data point is input during inference, the detection unit 154 performs a Fourier transform on the input signal of sequence length T input to the state space model of the first layer (step S503).

[0099] Next, the detection unit 154 calculates the amplitude Ftest=[F １ , ..., F Ｔ ] is averaged over the entire training data. ｔｅｓｔ is calculated (step S504).

[0100] Then, the detection unit 154 detects ^F and ^F ｔｅｓｔ The similarity r(^F, ^F ｔｅｓｔ ) is calculated (step S505). The similarity r may be a cosine similarity or a square error that measures distance.

[0101] Here, the detection unit 154 detects r(^F, ^F ｔｅｓｔ If the difference (%) is less than the threshold (Yes at step S506), the data point at the time of inference is determined to be out of the distribution of the training data (step S507). In this case, the detection unit 154 detects that the distribution of the inference data is different from the distribution of the training data.

[0102] On the other hand, the detection unit 154 detects r(^F, ^F ｔｅｓｔ If the difference (%) is equal to or greater than the threshold (step S506, No), the data point at the time of inference is determined to be data within the distribution of the training data (step S508). In this case, the detection unit 154 does not detect that the distribution of the inference data is different from the distribution of the training data.

[0103] The detection unit 154 may perform the process of FIG. 7 for each state space model included in the deep learning model.

[0104] For example, if the detection unit 154 detects that there is a lack of training data or that the distribution of the inference data is different from the distribution of the training data, re-learning or additional learning of the deep learning model is performed.

[0105] [System Configuration, etc.] The components of each device shown in the figures are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0106] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0107] [Program] In one embodiment, the information processing device 10 can be implemented by installing a program that executes the above-described learning process as package software or online software on a desired computer. For example, by executing the above-described program on an information processing device, the information processing device can function as the information processing device 10. The information processing device referred to here includes desktop and notebook personal computers. Furthermore, other information processing devices also include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).

[0108] The information processing device 10 may also be implemented as a server device that provides services related to the above processing to a client terminal device used by a user. For example, the server device may be implemented as a server device that receives parameters and input signals of a model before updating and outputs parameters of a model after updating. In this case, the server device may be implemented as a web server or as a cloud that provides services related to the above processing by outsourcing.

[0109] 8 is a diagram showing an example of a computer that executes a program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0110] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0111] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define each process of the information processing device 10 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configuration of the information processing device 10 are stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0112] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0113] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0114] REFERENCE SIGNS LIST 10 Information processing device 11 Communication unit 12 Input unit 13 Output unit 14 Storage unit 15 Control unit 141 Model information 151 Calculation unit 152 Creation unit 153 Update unit 154 Detection unit

Claims

1. A learning device comprising: a creation unit that creates an objective function that becomes smaller as the amount of frequency components of a signal input to a state space model, which is at least one layer of a deep learning model, increases; and an update unit that updates parameters of the deep learning model so as to optimize the objective function.

2. The learning device according to claim 1, characterized in that the creation unit creates the objective function such that the larger the sum of amplitudes obtained by performing a discrete Fourier transform on the signals input to the state space model, the smaller the objective function becomes.

3. A learning method executed by a learning device, comprising: a creation step of creating an objective function that becomes smaller as the amount of frequency components of a signal input to a state space model, which is at least one layer of a deep learning model, increases; and an update step of updating parameters of the deep learning model so that the objective function is optimized.

4. An inference device that performs inference using a deep learning model in which at least one layer is a state space model, and in which parameters are advanced so as to optimize an objective function that becomes smaller as the amount of frequency components of the signal input to the state space model increases.

5. An inference method executed by an inference device, characterized in that inference is performed using a deep learning model in which at least one layer is a state space model, and parameters are advanced so as to optimize an objective function that becomes smaller as the amount of frequency components of a signal input to the state space model increases.

6. A detection device having a detection unit that detects, based on the amount of frequency components of a signal input to a state space model, which is at least one layer of a deep learning model, that the training data of the deep learning model is insufficient, or that the distribution of the data input to the deep learning model is different from the distribution of the training data used to learn the deep learning model.

7. A detection method executed by a detection device, comprising: detecting, based on the amount of frequency components of a signal input to a state space model, which is at least one layer of a deep learning model, that there is a shortage of training data for the deep learning model, or that the distribution of the data input to the deep learning model is different from the distribution of the training data used to train the deep learning model.

8. A program for causing a computer to function as the learning device according to claim 1, the inference device according to claim 4, or the detection device according to claim 6.