System and method for deep neural network based on infinite impulse response layer

Abstract

According to an infinite impulse response layer-based deep neural network system and method proposed in the present invention, the system includes at least one IIR layer, which is a layer for learning IIR filter coefficients corresponding to filter coefficients of an IIR filter among digital filters, so that superior performance can be achieved, compared to an existing convolutional layer-based deep neural network, through an IIR layer capable of modeling a more complex function by means of fewer parameters than an existing convolutional layer interpreted as a layer for learning FIR filter coefficients.

Description

Deep neural network system and method based on infinite impulse response layers

[0001] The present invention relates to a deep neural network system and method, and more particularly, to a deep neural network system and method based on an infinite impulse response layer.

[0002] The content described in this section merely provides background information for one embodiment of the present invention and does not constitute prior art.

[0003]

[0004] Deep neural networks, a type of machine learning technology, are being utilized in a wide range of fields. Learning using these deep neural networks is called deep learning, and deep learning is currently the most actively developing technology. Recently, it's become so widespread that it's difficult to find a field where deep learning isn't being utilized.

[0005]

[0006] Deep learning is being actively used in areas such as image processing, speech processing, and natural language processing (NLP). According to IMARC, the global deep learning market size reached $17.2 billion in 2022 and is expected to grow at a compound annual growth rate of 38.2%, reaching $113 billion by 2028 (IMARC, “Deep Learning Market: Global Industry Trends, Share, Size, Growth, Opportunity, and Forecast 2023-2028,” February 2023).

[0007]

[0008] Deep neural networks are multilayered neural network structures. For example, convolutional neural networks (CNNs), a type of deep neural network, consist of three types of layers: convolutional layers, pooling layers, and fully connected layers. Various layers for constructing deep neural networks have been studied and proposed.

[0009]

[0010] Meanwhile, as a prior art related to the present invention, Patent Publication No. 10-2021-0022455 (Title of invention: Deep neural network learning device and method thereof, Publication date: March 3, 2021) has been disclosed.

[0011]

[0012] The background technology described above is technical information that the inventor possessed for the purpose of deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be said to be publicly known technology disclosed to the general public prior to the application for the present invention.

[0013] The present invention has been proposed to solve the above-mentioned problems of the existing proposed methods, and the purpose of the present invention is to provide a deep neural network system and method based on an infinite impulse response layer, which can achieve better performance than a deep neural network based on an existing convolutional layer, through an IIR layer that can model a more complex function with fewer parameters than a conventional convolutional layer interpreted as a layer that learns the filter coefficients of a finite impulse response (FIR) filter by including at least one IIR layer, which is a layer that learns the filter coefficients of an infinite impulse response (IIR) filter among digital filters.

[0014]

[0015] In addition, the present invention provides a deep neural network system and method based on an infinite impulse response layer, which are easy to apply to various industries because they can replace convolutional layers in various deep neural network systems with the proposed IIR layers by presenting both the forward function and the backward function of the IIR layer to enable learning and inference of the IIR layer, and because they can easily replace part or all of the deep neural network with the IIR layer for learning in numerous fields that use deep learning. Another purpose of the present invention is to provide a deep neural network system and method based on an infinite impulse response layer, which are easy to apply to various industries because they can easily replace part or all of the deep neural network with the IIR layer for learning.

[0016]

[0017] In addition, another object of the present invention is to provide a deep neural network system and method based on an infinite impulse response layer, which can improve the final performance of a deep neural network system by applying an IIR layer capable of modeling more complex functions with fewer parameters to the deep neural network.

[0018]

[0019] However, the technical problem to be achieved by the present invention is not limited to the technical problem described above, and other technical problems may exist, and even if not explicitly mentioned, the purpose or effect that can be understood from the solution or embodiment of the problem is also included.

[0020] In order to achieve the above-mentioned purpose, a deep neural network system based on an infinite impulse response layer according to the features of the present invention is provided.

[0021] As a deep neural network system,

[0022] Its configuration is characterized by including at least one IIR layer, which is a layer that learns using IIR filter coefficients, which are filter coefficients of an infinite impulse response (IIR) filter among digital filters, as weights.

[0023]

[0024] Preferably, the IIR layer comprises:

[0025] The first weight (a) applied to the input and the second weight (b) applied to the output of the previous time component can be learned.

[0026]

[0027] More preferably, the forward function of the IIR layer is

[0028] A function that computes the output y given the input x, for n=0, …, T-1. can be defined as

[0029] Here, x is the input of the IIR layer. The size is C×T, x[i,n] (i=0, …, C-1, n=0, …, T-1) is the ith channel, nth time component, a and b are the weights to be learned. The sizes are C×K1 and C×K2, respectively. y is the output of the forward function of the IIR layer. The size is C×T, y[j,n] (j=0, …, C-1, n=0, …, T-1) is the jth channel, nth time component.

[0030]

[0031] Even more preferably, the backward function of the IIR layer is

[0032] If given , and It is a function that calculates , and can be defined by the mathematical formula below for n=T-1, …, 0.

[0033] [Mathematical formula]

[0034]

[0035] Here, L is the loss function.

[0036]

[0037] In order to achieve the above-mentioned purpose, a deep neural network method based on an infinite impulse response layer according to the characteristics of the present invention is provided.

[0038] A deep neural network learning method performed on a computer,

[0039] Its structural feature is that it learns a deep neural network that includes at least one IIR layer, which is a layer that learns using the IIR filter coefficients, which are filter coefficients of an infinite impulse response (IIR) filter among digital filters, as weights.

[0040]

[0041] Preferably, the IIR layer comprises:

[0042] The first weight (a) applied to the input and the second weight (b) applied to the output of the previous time component can be learned.

[0043]

[0044] More preferably, the forward function of the IIR layer is

[0045] A function that computes the output y given the input x, for n=0, …, T-1. can be defined as

[0046] Here, x is the input of the IIR layer. The size is C×T, x[i,n] (i=0, …, C-1, n=0, …, T-1) is the ith channel, nth time component, a and b are the weights to be learned. The sizes are C×K1 and C×K2, respectively. y is the output of the forward function of the IIR layer. The size is C×T, y[j,n] (j=0, …, C-1, n=0, …, T-1) is the jth channel, nth time component.

[0047]

[0048] Even more preferably, the backward function of the IIR layer is

[0049] If given , and It is a function that calculates , and can be defined by the mathematical formula below for n=T-1, …, 0.

[0050] [Mathematical formula]

[0051]

[0052] Here, L is the loss function.

[0053]

[0054] The present invention is characterized in that it is a computer program stored in a computer-readable recording medium for executing a deep neural network method based on an infinite impulse response layer according to the characteristics of the present invention to achieve the above-mentioned purpose on a computer.

[0055] According to the deep neural network system and method based on the infinite impulse response layer proposed in the present invention, by including at least one IIR layer, which is a layer that learns IIR filter coefficients, which are filter coefficients of an IIR filter among digital filters, the IIR layer can model more complex functions with fewer parameters than the existing convolutional layer interpreted as a layer that learns FIR filter coefficients, thereby achieving better performance than the existing convolutional layer-based deep neural network.

[0056]

[0057] In addition, according to the deep neural network system and method based on the infinite impulse response layer proposed in the present invention, by presenting both the forward function and the backward function of the IIR layer, learning and inference of the IIR layer are possible, so that convolutional layers in various deep neural network systems can be replaced with the proposed IIR layer, and since part or all of the deep neural network can be easily replaced with the IIR layer for learning in numerous fields that use deep learning, it is easy to apply to various industries.

[0058]

[0059] In addition, according to the deep neural network system and method based on the infinite impulse response layer proposed in the present invention, the final performance of the deep neural network system can be improved by applying an IIR layer capable of modeling more complex functions with fewer parameters to the deep neural network.

[0060]

[0061] In addition, the various advantageous advantages and effects of the present invention are not limited to the above-described contents, and will be more easily understood in the process of explaining specific embodiments of the present invention.

[0062] Figure 1 is a diagram illustrating the structure of a deep neural network.

[0063] Figure 2 is a diagram illustrating the FIR filtering process.

[0064] Figure 3 is a diagram illustrating the IIR filtering process.

[0065] FIG. 4 is a diagram illustrating the configuration of a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention.

[0066] FIG. 5 is a diagram illustrating an inference process of an IIR layer of a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention.

[0067] FIG. 6 is a diagram illustrating a backpropagation process of an IIR layer of a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention.

[0068] FIG. 7 is a diagram illustrating a method for configuring a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention.

[0069] <Explanation of symbols>

[0070] 100: IIR layer

[0071] 110: Forward function of the IIR layer

[0072] 120: Backward function of the IIR layer

[0073] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement them. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar reference numerals have been used throughout the specification to indicate similar parts.

[0074]

[0075] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "indirectly connected" with another element in between. Furthermore, when a part is said to "include" a component, this should be understood to mean that, unless specifically stated to the contrary, it may include other components rather than excluding them, and does not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0076]

[0077] The following examples are provided as detailed explanations to aid understanding of the present invention and do not limit the scope of the invention. Therefore, inventions with the same functions and scope as the present invention are also within the scope of the present invention.

[0078]

[0079] In addition, each configuration, process, procedure or method included in each embodiment of the present invention may be shared within a scope that is not technically inconsistent with each other.

[0080]

[0081] Additionally, some of the operations or functions described as being performed by a terminal, apparatus, or device in the present invention may instead be performed by a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described as being performed by a server may also be performed by a terminal, apparatus, or device connected to the server.

[0082]

[0083] In particular, a means for executing a system according to each embodiment of the present invention may be an application or a web server, and a terminal that is a means for reading a recording medium recording the application or web server may include not only a general PC such as a general desktop or laptop, but also a mobile terminal such as a smart phone or tablet PC.

[0084]

[0085] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0086]

[0087] To understand the present invention, a basic understanding of deep neural network inference, learning, forward and backward functions of artificial neural network layers, filtering, and convolutional layers is essential. Therefore, sections 1) through 3) provide technical details for understanding prior knowledge, while section 4) presents information on the problems of prior art and the IIR layer, which can improve upon them.

[0088]

[0089] 1) Deep neural network

[0090] Artificial intelligence is a technology that artificially implements human abilities such as learning, reasoning, and perception. Deep learning is a branch of artificial intelligence that trains deep neural networks. A deep neural network is a model that artificially mimics the human neural network by stacking artificial neural networks (layers) deeply. Figure 1 is a diagram illustrating the structure of a deep neural network. In Figure 1, the leftmost circle (x) represents the input, the rightmost circle (y) represents the final output of the model, the circle (h) in between represents the hidden layer output, and the arrow (W) represents the weights or parameters that the deep neural network will learn. Training a deep neural network requires a large number of (input, output) data pairs. The goal of deep learning is to effectively train the model weights of a deep neural network using a dataset. Once training is complete, the corresponding output can be produced when given any input. For example, if deep learning is applied to voice enhancement, if a noisy voice is given as input to the model, it can output a clean voice with the noise removed.

[0091]

[0092] 1-1) Forward function

[0093] The deep neural network in Figure 1 consists of a fully connected layer, which is the most basic layer. Each layer consists of input / output represented by circles and weights represented by arrows. Specifically, the kth layer of the deep neural network has an input of h k-1 , the output is h k , the weight is W k and h i k-1 , h j k , W i,j k are connected. The input and output of the kth fully connected layer are mathematically expressed as the following mathematical expression 1.

[0094]

[0095]

[0096]

[0097] At this time, σ(·) is a separately defined nonlinear function, and functions such as ReLU and sigmoid are commonly used. In addition, is the forward function of the kth layer. Now, input x 1:L When it comes in, h is calculated according to the above formula in the first layer. 1:M 1 can be calculated. And in the second layer h 1:M 2 We can calculate y and repeat this until the last layer. 1:p can be calculated. When an input is input to a deep neural network, the process of giving an output is called inference, and for inference, the forward function of the layer used in the deep neural network must be defined.

[0098]

[0099] 1-2) Backward function

[0100] The purpose of deep learning is to give the model an input x and then output it. When , the correct output y and the model output The goal is to update the model's weights W so that the loss L, which is the difference between the two, is minimized. To do this, we need to know the model's backward operation, and the specific learning algorithm is as follows.

[0101]

[0102] 1. Initialize the weights W of the deep neural network to random values.

[0103] 2. Select (x, y) from the dataset. When x is given as input to the model, the deep neural network performs forward propagation to produce some output. will give.

[0104] 3. Output of deep neural network Calculate L, which is the difference between the actual desired output y and the desired output y.

[0105] 4. Differentiation of L with respect to W Calculate.

[0106] 5. For a sufficiently small value of ε, adjust W as in Equation 2 below. Then, under certain conditions, It is known that L, the difference between y and y, is statistically reduced.

[0107]

[0108]

[0109]

[0110] 6. Repeat steps 2 to 5 as many times as necessary. Then, the output of the deep neural network for any input x in the dataset is obtained. The difference L between the actual desired output y and the actual desired output y will become very small.

[0111]

[0112] The key to deep learning is How to calculate it. If is given, and the backward function of the layer and If you know and can be obtained using the mathematical formula 3 below.

[0113]

[0114]

[0115]

[0116] Using this, first L is the output of the deep neural network Differentiated value for , and in the Kth layer and can be calculated, and in the K-1th layer and If we can get , and repeat this up to the first layer, In order to train a deep neural network like this, you need to define not only the forward function of the layer used, but also the backward function.

[0117]

[0118] 2) Finite impulse response (FIR) and infinite impulse response (IIR) filters

[0119] A filter's role is to pass only signals with desired characteristics from the input signal. Among them, digital filters are broadly divided into two types: FIR filters and IIR filters.

[0120]

[0121] Figure 2 is a diagram illustrating the FIR filtering process. The process of obtaining the output signal y by applying the FIR filter coefficient h[k] (k=0, 1, ,…,N-1) of length N to the input signal x is expressed in a formula as shown in Mathematical Formula 4 below.

[0122]

[0123]

[0124]

[0125] Figure 3 is a diagram illustrating the IIR filtering process. Unlike the FIR filter, the IIR filter uses not only x[n], x[n-1], … but also y[n-1], y[n-2], … to create the output y[n]. The IIR filtering process can be expressed as a formula in Mathematical Expression 5 below.

[0126]

[0127]

[0128]

[0129] Here, a[k] (k=0, 1, ,…,N-1) and b[k] (k=0, 1, ,…,M-1) are IIR filter coefficients. IIR filters can model more complex functions using fewer filter coefficients than FIR filters.

[0130]

[0131] 3) Convolutional layer

[0132] Deep neural networks are structures that stack a large number of artificial neural network layers. There are several types of artificial neural network layers, including the fully connected layer shown in Figure 1, and one of the most commonly used layers is the convolutional layer. Convolutional layers can be one-dimensional, two-dimensional, or three-dimensional, but only the one-dimensional convolutional layer will be explained here. The input of the convolution is a multi-channel sequence, and the channel size is C. in When x[i,n] (i=0, …, C in -1, n=0,1,2,…). The convolutional layer has input x and weights W[j,i,k] (j=0, …, C out -1, i=0, …, C in -1, k=0,…,K-1) and output y[k,n] (j=0, …, C out -1, n=0,1,2,…) is generated, and the formula is as follows: Mathematical Formula 6.

[0133]

[0134]

[0135]

[0136] 4) IIR layer

[0137] Looking at Equations 4 and 6 above, the convolutional layer can be interpreted as multi-channel FIR filtering. In other words, the convolutional layer can be viewed as a layer that learns FIR filter coefficients.

[0138]

[0139] However, as previously explained in comparing FIR and IIR filters in 2), IIR filters can model more complex functions than FIR filters using fewer filter coefficients. Therefore, the present invention proposes a new IIR layer that learns IIR filter coefficients. Compared to convolutional layers frequently used in existing deep neural networks, IIR layers can achieve better performance with fewer parameters.

[0140]

[0141] FIG. 4 is a diagram illustrating a configuration of a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention. As illustrated in FIG. 4, the deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention may include at least one IIR layer (100), which is a layer that learns by using IIR filter coefficients, which are filter coefficients of an infinite impulse response filter among digital filters, as weights.

[0142]

[0143] In addition, a deep neural network learning method including an infinite impulse response layer according to an embodiment of the present invention is a deep neural network learning method performed on a computer, and can be implemented by learning a deep neural network including at least one IIR layer (100), which is a layer that learns using IIR filter coefficients, which are filter coefficients of an infinite impulse response filter among digital filters, as weights.

[0144]

[0145] That is, the present invention relates to a deep neural network system comprising an IIR layer (100) and a learning method thereof. Unlike a convolutional layer that learns FIR filter coefficients, the IIR layer (100) is characterized by learning IIR filter coefficients. More specifically, the IIR layer (100) can learn a first weight (a) applied to an input and a second weight (b) applied to the output of a previous time component.

[0146]

[0147] Below, the forward function and backward function of the IIR layer (100) will be described.

[0148]

[0149] First, the input and output of the forward and backward functions are defined in detail.

[0150]

[0151] x: Input of the IIR layer (100). Size is C×T. x[i,n] (i=0, …, C-1, n=0, …, T-1) is the i-th channel, n-th time component.

[0152] a, b: Weights to be learned. Their sizes are C×K1 and C×K2, respectively.

[0153] y: Output of the forward function of the IIR layer (100). Size is C×T. y[j,n] (j=0, …, C-1, n=0, …, T-1) is the jth channel, nth time component.

[0154] : Loss L is differentiated with respect to the forward function output y, and its size is C×T.

[0155] , : Loss L is differentiated with respect to weights a and b, and their sizes are C×K1 and C×K2, respectively.

[0156] : Loss L is differentiated with respect to the forward function input x, and its size is C×T.

[0157]

[0158] FIG. 5 is a diagram illustrating an inference process of an IIR layer (100) of a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention. As illustrated in FIG. 5, the forward function (110) of the IIR layer (100) of the deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention is a function that calculates y when x is given. Through this, the inference of the deep neural network can be performed so that a desired output is produced when an arbitrary input is input.

[0159]

[0160] More specifically, the forward function (110) of the IIR layer (100) is as shown in the following mathematical expression 7 for n=0, …, T-1.

[0161]

[0162]

[0163]

[0164] FIG. 6 is a diagram illustrating a backpropagation process of an IIR layer (100) of a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention. As illustrated in FIG. 6, the backward function (120) of the IIR layer (100) of the deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention is If given (in other words, , )and Calculate. and The parameters a and b of the IIR layer (100) can be learned using It can be used to learn the previous layer, i.e., to learn the deep neural network.

[0165]

[0166] The backward function (120) of the IIR layer (100) is as shown in mathematical expression 8 below for n=T-1, …, 0.

[0167]

[0168]

[0169]

[0170] Conventional convolutional layers can be interpreted as layers that learn FIR filter coefficients. However, FIR filters have lower expressive power than IIR filters. In other words, IIR filters can model more complex functions with fewer parameters than FIR filters. Therefore, the present invention proposes a new IIR layer (100) that learns IIR filter coefficients, and presents both the forward function (110) and the backward function (120) of the IIR layer (100), thereby enabling learning and inference of the IIR layer (100).

[0171]

[0172] FIG. 7 is a diagram illustrating a method for constructing a deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention. As illustrated in FIG. 7, the deep neural network system based on an infinite impulse response layer according to an embodiment of the present invention can be implemented by replacing a convolutional layer with an IIR layer (100) in various deep neural network systems, thereby improving the final performance of the system.

[0173]

[0174] The IIR layer (100) proposed in the present invention is readily applicable to industry, as it can be easily replaced with a portion or all of a deep neural network in numerous fields utilizing deep learning. Furthermore, it exhibits excellent business potential, as it can improve the performance of deep neural networks, which are currently being used in a wide variety of services.

[0175]

[0176] An embodiment of the present invention can also be implemented in the form of a computer program stored in a computer-readable recording medium to execute a deep neural network learning method including an infinite impulse response layer according to an embodiment of the present invention as described above on a computer.

[0177]

[0178] Here, computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism, and includes any information delivery media.

[0179]

[0180] As described above, according to the deep neural network system and method based on the infinite impulse response layer proposed in the present invention, by including at least one IIR layer (100), which is a layer that learns the IIR filter coefficients, which are filter coefficients of the infinite impulse response filter among digital filters, the IIR layer (100) can model a more complex function with fewer parameters than the existing convolutional layer interpreted as the layer that learns the FIR filter coefficients, thereby achieving better performance than the existing convolutional layer-based deep neural network.

[0181]

[0182] In addition, according to the deep neural network system and method based on the infinite impulse response layer proposed in the present invention, by presenting both the forward function (110) and the backward function (120) of the IIR layer (100), learning and inference of the IIR layer (100) are possible, so that the convolutional layer in various deep neural network systems can be replaced with the proposed IIR layer (100), and since part or all of the deep neural network can be easily replaced with the IIR layer (100) for learning in numerous fields that use deep learning, it is easy to apply to various industries.

[0183]

[0184] In addition, according to the deep neural network system and method based on the infinite impulse response layer proposed in the present invention, the final performance of the deep neural network system can be improved by applying an IIR layer (100) that can model more complex functions with fewer parameters to the deep neural network.

[0185]

[0186] The foregoing description of the present invention is provided for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0187]

[0188] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. As a deep neural network system, A deep neural network system including an infinite impulse response layer, characterized in that it includes at least one IIR layer, which is a layer that learns using IIR filter coefficients, which are filter coefficients of an infinite impulse response (IIR) filter among digital filters, as weights.

2. In the first paragraph, the IIR layer, A deep neural network system including an infinite impulse response layer, characterized in that it learns a first weight (a) applied to an input and a second weight (b) applied to the output of a previous time component.

3. In the second paragraph, the forward function of the IIR layer is A function that computes the output y given the input x, for n=0, …, T-1. A deep neural network system comprising an infinite impulse response layer, characterized by being defined as follows. Here, x is the input of the IIR layer. The size is C×T, x[i,n] (i=0, …, C-1, n=0, …, T-1) is the i-th channel, the n-th time component, and a and b are the weights to be learned. The size is C×K, respectively. 1 , C×K 2 , y is the output of the IIR layer forward function, the size of which is C×T, and y[j,n] (j=0, …, C-1, n=0, …, T-1) is the jth channel, nth time component.

4. In the third paragraph, the backward function of the IIR layer is If given , and A deep neural network system comprising an infinite impulse response layer, characterized in that the function for calculating , is defined by the following mathematical equation for n=T-1, …, 0. [Mathematical formula] Here, L is the loss function.

5. A method for learning a deep neural network performed on a computer, A method for learning a deep neural network including an infinite impulse response layer, characterized by learning a deep neural network including at least one IIR layer, which is a layer that learns using IIR filter coefficients, which are filter coefficients of an infinite impulse response (IIR) filter among digital filters, as weights.

6. In the fifth paragraph, the IIR layer, A deep neural network learning method including an infinite impulse response layer, characterized by learning a first weight (a) applied to an input and a second weight (b) applied to the output of a previous time component.

7. In the 6th paragraph, the forward function of the IIR layer is, A function that computes the output y given the input x, for n=0, …, T-1. A method for learning a deep neural network including an infinite impulse response layer, characterized in that it is defined as follows. Here, x is the input of the IIR layer. The size is C×T, x[i,n] (i=0, …, C-1, n=0, …, T-1) is the i-th channel, the n-th time component, and a and b are the weights to be learned. The size is C×K, respectively. 1 , C×K 2 , y is the output of the IIR layer forward function, the size of which is C×T, and y[j,n] (j=0, …, C-1, n=0, …, T-1) is the jth channel, nth time component.

8. In the 7th paragraph, the backward function of the IIR layer is If given , and A deep neural network learning method including an infinite impulse response layer, characterized in that the function for calculating is defined by the following mathematical formula for n=T-1, …, 0. [Mathematical formula] Here, L is the loss function.

9. A computer program stored in a computer-readable recording medium for executing a deep neural network learning method including an infinite impulse response layer according to any one of claims 5 to 8 on a computer.

Images