Method for providing time series prediction deep learning neural network and its system

Abstract

To provide a deep learning neural network for improving the deep learning prediction performance of a time series database based on learning by temporal relationship and temporal influence.SOLUTION: The method for providing time series prediction deep learning neural network and its system are methods for providing a time series prediction deep learning neural network including operations in which a time series prediction application executed by at least one processor of a terminal provides a time series prediction deep learning neural network, which make a time relation-effect layer module trained, input the first hour point time series data to the time relation-effect layer module, acquire the second hour point time series data based on the first time point time series data from the time relation-effect layer module, and provide the second hour point time series data, and in which the time relation-effect layer module includes a time decomposition gate module, a time relationship gate module, and a time influence degree gate module.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a method and system for providing a deep learning neural network for time series prediction, more particularly to a method and system for providing a deep learning neural network that improves deep learning prediction performance based on time series data based on learning based on temporal relation and temporal effect, and a method for recognizing infant voice using the same. [Background technology]

[0002] In business systems in various fields, such as the manufacturing and / or financial industries, various types of time series data are generated, such as system resource performance (e.g., CPU usage, memory usage, etc.) and business processing volume (e.g., transactions per unit time, etc.).

[0003] For stable system operation, it is often necessary to monitor time series data on system resource performance and business processing volume according to preset criteria, detect or predict changes in the time series data while monitoring, and make appropriate decisions.

[0004] For example, professional investors at securities firms monitor changes in market values, such as exchange rates and / or interest rates, to predict the timing and amount of investments, while factory equipment operators investigate temperature, pressure, and / or flow rate information, etc., to predict equipment conditions and perform optimal control.

[0005] In this way, the problem of time series analysis is one of predicting future situations using past information that changes over time, and time series forecasting algorithms for this purpose are used extensively in various business decision-making processes.

[0006] However, because the analysis of time-series data involves multiple factors, it is difficult to accurately determine which factors influence each other and in what relationship.

[0007] In other words, time series data collected on business systems in various fields contains many different types of latent factors, and the reality is that there are limitations to taking all of these into account when conducting time series analysis.

[0008] As a result, there has been an increase in research into time series analysis techniques that incorporate artificial intelligence network structures, which show superior predictive performance in time series analysis compared to traditional statistical analysis.

[0009] One example is a time series analysis technique that utilizes deep learning network algorithms such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory models (LSTMs), and / or Gated Recurrent Units (GRUs), etc.

[0010] However, even in the case of such conventional AI-based time series analysis methods, it is necessary to define all factors that affect the time series analysis, and even if all the factors are defined, there is still a problem that the time at which each factor affects the time series analysis is very different. In fact, there are many limitations in applying this method to business systems in various fields.

[0011] To solve these problems, a new method has emerged in recent years that uses signal decomposition techniques such as Variational Mode Decomposition (VMD) to decompose time series data into multiple different time series patterns, and then performs time series analysis using a model that learns the characteristics of the decomposed time series patterns.

[0012] However, when applying the conventional variational mode decomposition method to layers (CNN, RNN, LSTM, and / or GRU) used in existing artificial intelligence models, there is a problem in that the input time series data cannot be decomposed into similar time series patterns within the layer.

[0013] In addition, when input time series data is decomposed into different time series patterns and learned according to the conventional technology, even if correlations exist between the individual time series patterns, the patterns are learned through different models, and the relationship between the time series patterns cannot be taken into consideration.

[0014] Therefore, there is a need for an improved artificial intelligence network that can learn and analyze various factors in time series data based on their time-dependent relationships and influences.

[0015] Meanwhile, intelligent personal assistants can recognize the voices spoken by users using voice recognition technology. However, most existing voice recognition technologies were trained based on adult voices, so there was a problem that the recognition rate was low when applying the voices of infants to the existing voice recognition models.

[0016] In addition, there is a drawback in that it is difficult to obtain the initial training data required to train an acoustic model in order to recognize children's voices, and because of the mischievous and distracted speech characteristic of children, it is very difficult to extract voices suitable for training an acoustic model from children's voice data. [Prior art documents] [Patent documents]

[0017] [Patent Document 1] Korean Patent No. 10-2011689 [Patent Document 2] Korean Patent Publication No. 2016-0071111 Summary of the Invention [Problem to be solved by the invention]

[0018] An object of one embodiment of the present invention is to provide a deep learning neural network that improves deep learning prediction performance based on time series data based on learning based on temporal relation and temporal effect.

[0019] In addition, an object of one embodiment of the present invention is to provide a method for automatically analyzing differences in infant voices and learning features using a deep learning neural network that improves deep learning prediction performance based on time series data, and recognizing infant voices based on the analysis and prediction of infant voices.

[0020] Additionally, one embodiment of the present invention provides a method for recognizing infant voice based on a time series prediction deep learning neural network that updates infant voice with feedback input including a caregiver's voice.

[0021] However, the technical problems that the present invention and the embodiments of the present invention aim to solve are not limited to the above-mentioned technical problems, and other technical problems may exist. [Means for solving the problem]

[0022] A method and system for providing a time series prediction deep learning neural network according to one embodiment of the present invention is a method for a time series prediction application executed by at least one processor of a terminal to provide a time series prediction deep learning neural network, the method including the steps of training a temporal relation-effect layer module, inputting time series data at a first time point into the temporal relation-effect layer module, obtaining time series data at a second time point based on the time series data at the first time point from the temporal relation-effect layer module, and providing the time series data at the second time point, wherein the temporal relation-effect layer module includes a temporal decomposition gate module, a temporal relation gate module, and a temporal effect gate module.

[0023] In another aspect, the temporal decomposition gate module is a deep learning module that receives the time series data at the first time point as input data and receives time series pattern decomposition data as output data, the time series data being decomposed into a plurality of mutually different time series patterns via a variational mode decomposition method.

[0024] In another aspect, training the temporal relationship-effect layer module includes training the temporal decomposition gate module based on a training data set comprising a plurality of time series data and time series pattern decomposition data corresponding to each of the plurality of time series data.

[0025] In another aspect, the temporal relationship gate module is a deep learning module that receives as input data concealment state matrix data having k (k>0) pieces of concealment state data and the time series pattern decomposition data, and outputs as output data temporal relationship analysis data that is data obtained by analyzing a multivariable temporal relation between the concealment state matrix data and the time series pattern decomposition data.

[0026] In another aspect, training the temporal relationship-effect layer module includes training the temporal relationship gate module based on a training data set comprising a plurality of hidden state matrix data and a plurality of time series pattern decomposition data.

[0027] In another aspect, the temporal influence gate module is a deep learning module that receives as input data concealment state matrix data having k (k>0) pieces of concealment state data and the time series pattern decomposition data, and outputs as output data temporal influence analysis data that is data obtained by analyzing a multivariable temporal effect between the concealment state matrix data and the time series pattern decomposition data.

[0028] In another aspect, training the temporal relationship-effect layer module includes training the temporal influence gating module based on a training data set comprising a plurality of hidden state matrix data and a plurality of time series pattern decomposition data.

[0029] In another aspect, the concealment state data is data learned by accumulating past information of the first time point time series data with respect to a predetermined time point as a reference.

[0030] In another aspect, the step of acquiring the second time series data includes a step of updating the concealment state data at a predetermined time t through pattern learning based on the input data at time t and the concealment state data at time t-1, and a step of updating the concealment state data at time t+1 through pattern learning based on the input data at time t+1 and the concealment state data at time t.

[0031] In another aspect, the step of acquiring the second time point time series data includes a step of acquiring the second time point time series data based on time series pattern decomposition data which is output data of the temporal decomposition gate module, temporal relationship analysis data which is output data of the temporal relationship gate module, and temporal influence analysis data which is output data of the temporal influence gate module.

[0032] In another aspect, providing the second time point time series data includes providing the second time point time series data based on a predetermined application service.

[0033] Meanwhile, a system for providing a time series prediction deep learning neural network according to an embodiment of the present invention includes at least one memory storing a time series prediction application, and at least one processor reading the time series prediction application stored in the memory to provide a time series prediction deep learning neural network, wherein the time series prediction application includes instructions for training a temporal relation-effect layer module, inputting time series data at a first time point into the temporal relation-effect layer module, obtaining time series data at a second time point according to the time series data at the first time point from the temporal relation-effect layer module, and providing the time series data at the second time point, and the temporal relation-effect layer module includes a temporal decomposition gate module, a temporal relation gate module, and a temporal effect gate module.

[0034] In addition, a method for recognizing infant voice based on a time series prediction deep learning neural network according to an embodiment of the present invention is a method in which an infant education application executed by at least one processor of a terminal provides an infant education platform service, and includes the steps of training a temporal relation-effect sequence-to-sequence layer module, acquiring real-time voice, detecting infant voice data from the acquired real-time voice, inputting the detected infant voice data into the trained temporal relation-effect sequence-to-sequence layer module, generating answer data based on question data output from the temporal relation-effect sequence-to-sequence layer module, and providing an infant education platform service based on the generated answer data. Effect of the Invention

[0035] The method and system for providing a time series prediction deep learning neural network according to an embodiment of the present invention provides a deep learning neural network that improves deep learning prediction performance based on time series data based on learning based on temporal relation and temporal effect, thereby taking into account all of the numerous latent factors that may exist in time series data, regardless of their form, and enabling time series prediction deep learning based on the time series data with high accuracy.

[0036] In addition, the method and system for providing a time series prediction deep learning neural network according to one embodiment of the present invention provides a deep learning neural network that is structured to self-learn the relationships and influences between temporal factors contained in specific time series data at the deep learning neural network layer level, thereby achieving time series prediction deep learning with high accuracy (recognition rate), and as a result, it has the effect of improving the performance and quality of various application services that utilize data.

[0037] In addition, the method for recognizing infant voice based on an infant voice recognition network according to one embodiment of the present invention provides an infant education platform service based on a deep learning network structure that improves deep learning prediction performance based on voice data based on learning based on temporal relation and temporal effect, thereby having the effect of being able to perform service task processing using recognized voice data with higher accuracy.

[0038] As a result, the method for recognizing an infant voice based on an infant voice recognition network according to an embodiment of the present invention has the effect of further improving the performance and usability of an infant education platform service.

[0039] In addition, the method for recognizing infant voices based on a time series prediction deep learning neural network according to an embodiment of the present invention has the effect of improving the infant voice recognition rate by utilizing a deep learning network structure that automatically analyzes infant voice differences and performs feature learning.

[0040] In addition, the method for recognizing infant voice based on a time series prediction deep learning neural network according to an embodiment of the present invention has the effect of updating the infant voice through feedback input including parental voice, thereby correcting the infant voice data so that it can be used based on the input feedback even if the infant voice data is not suitable for training a deep learning model, thereby increasing the utility of the infant voice data.

[0041] However, the effects that can be obtained in the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood from the following description. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 is a conceptual diagram of a system for providing a time series prediction deep learning neural network according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an internal block diagram of a terminal according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a conceptual diagram illustrating a temporal relationship-effect layer module according to an embodiment of the present invention. [Figure 4] 1 is an example of input and output data of a temporal relationship-effect layer module according to an embodiment of the present invention; [Diagram 5] 1 is an example of a diagram illustrating output data of a temporally resolved gate module according to an embodiment of the present invention; [Figure 6] 1 is an example of a drawing for explaining concealment state matrix data according to an embodiment of the present invention; [Figure 7] 1 is a flowchart illustrating a method for providing a time series prediction deep learning neural network according to an embodiment of the present invention. [Figure 8] FIG. 2 is a conceptual diagram for explaining a temporal relationship-effect sequence-to-sequence layer module TSSL according to an embodiment of the present invention. [Figure 9]2 is a flowchart illustrating a method for implementing an early childhood education platform providing service according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] The present invention can be modified in various ways and can have various embodiments. A specific embodiment will be illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and the method of achieving them will become clear with reference to the embodiments described in detail below with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be realized in various forms. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one component from another component, not in a limiting sense. In addition, singular expressions include plural expressions unless otherwise clearly different in the context. In addition, terms such as include or have mean that a feature or component described in the specification exists, and do not exclude in advance the possibility that one or more other features or components are added. In addition, in the drawings, the size of the components may be increased or decreased for the convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for the convenience of explanation, so the present invention is not necessarily limited to what is shown in the drawings.

[0044] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. When describing with reference to the drawings, identical or corresponding components are denoted by the same reference numerals and duplicate descriptions thereof will be omitted.

[0045] FIG. 1 is a conceptual diagram of a system for providing a time series prediction deep learning neural network according to one embodiment of the present invention.

[0046] As shown in FIG. 1, a time series prediction deep learning neural network providing system 1000 (hereinafter, referred to as a time series prediction system) according to an embodiment of the present invention can realize a time series prediction service that provides a deep learning neural network that improves deep learning prediction performance based on time series data based on learning based on temporal relation and temporal effect.

[0047] In the embodiment, a time series prediction system 1000 that realizes a time series prediction service can include a terminal 100, a service providing server 200, and a network 300 (Network).

[0048] In this case, the terminal 100 and / or the service providing server 200 may be connected via a network 300 .

[0049] Here, the network 300 according to the embodiment refers to a connection structure that allows information exchange between each node, such as the terminal 100 and / or the service providing server 200.

[0050] Examples of network 300 include, but are not limited to, a 3GPP (registered trademark) (3rd Generation Partnership Project) network, a LTE (Long Term Evolution) network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth (registered trademark) network, a satellite broadcast network, an analog broadcast network, and / or a DMB (Digital Multimedia Broadcasting) network.

[0051] Hereinafter, the terminal 100 and the service providing server 200 which realize the time series prediction system 1000 will be described in detail with reference to the attached drawings.

[0052] <Terminal 100> The terminal 100 according to the embodiment of the present invention may be a predetermined computing device provided with a time series prediction application (hereinafter, application) that provides a time series prediction service.

[0053] Specifically, in terms of hardware, the terminal 100 may include a mobile type computing device 100-1 and / or a desktop type computing device 100-2 on which an application is installed.

[0054] Here, the mobile type computing device 100-1 can be a mobile device provided with an application.

[0055] For example, the mobile type computing device 100-1 may comprise a smart phone, a mobile phone, a digital broadcasting device, a personal digital assistant (PDA), a portable multimedia player (PMP), and / or a tablet PC.

[0056] Furthermore, the desktop type computing device 100-2 may be a wired / wireless communication based device on which applications are installed.

[0057] For example, desktop-type computing device 100-2 may comprise a personal computer, such as a fixed desktop PC, a laptop computer, and / or an ultrabook.

[0058] Depending on the embodiment, the terminal 100 may further include a server computing device that provides a time series prediction service environment.

[0059] FIG. 2 is an internal block diagram of a terminal 100 according to an embodiment of the present invention.

[0060] 2, from a functional standpoint, the terminal 100 may include a memory 110, a processor assembly 120, a communication processor 130, an interface unit 140, an input system 150, a sensor system 160, and a display system 170. In an embodiment, the terminal 100 may include the above components within a housing.

[0061] Specifically, the memory 110 may store an application 111 .

[0062] At this time, the application 111 may store at least one of various application programs, data, and commands for providing a time series prediction service environment.

[0063] That is, the memory 110 may store instructions, data, etc. that may be used to generate a time series prediction service environment.

[0064] The memory 110 may also include a program area and a data area.

[0065] Here, the program area according to the embodiment can be connected to an operating system (OS) for booting the terminal 100 and functional elements, etc.

[0066] In addition, in the data area according to the embodiment, data generated by the use of the terminal 100 can be stored.

[0067] Additionally, memory 110 may comprise at least one or more non-transitory computer-readable storage media and transitory computer-readable storage media.

[0068] For example, the memory 110 can be a variety of storage devices such as a ROM, an EPROM, a flash drive, a hard drive, and the like, and can include web storage that provides storage functionality for the memory 110 over the internet.

[0069] The processor assembly 120 may include at least one or more processors capable of executing instructions of the application 111 stored in the memory 110 to perform various tasks for generating the time series prediction service environment.

[0070] In an embodiment, the processor assembly 120 may control the overall operation of the components via the application 111 in the memory 110 to provide time series prediction services.

[0071] Specifically, the processor assembly 120 may be a system-on-chip SOC suitable for the terminal 100 including a central processing unit (CPU) and / or a graphics processing unit (GPU).

[0072] In addition, the processor assembly 120 can execute an operating system (OS) and / or application programs stored in the memory 110 .

[0073] The processor assembly 120 can also control each component installed in the terminal 100 .

[0074] Furthermore, the processor assembly 120 can internally communicate with each component via a system bus, and can include one or more predetermined bus structures such as a local bus.

[0075] In addition, the processor assembly 120 may be implemented with at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0076] The communications processor 130 may include one or more devices for communicating with external devices, such that the communications processor 130 may communicate over a wireless network.

[0077] Specifically, the communication processor 130 can communicate with a predetermined device that stores a content source for realizing a time series prediction service environment.

[0078] The communications processor 130 may also be in communication with various user input components, such as a controller that receives user input.

[0079] In an embodiment, the communication processor 130 may transmit and receive various data related to the time series prediction service to and from other terminals 100 and / or external servers.

[0080] Such a communication processor 130 can wirelessly transmit and receive data with at least one of a base station, an external terminal 100, and an arbitrary server over a mobile communication network constructed via a communication device capable of implementing a technical standard or communication method for mobile communication (e.g., LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), WIFI), or a short-range communication method, etc.

[0081] The sensor system 160 may include various sensors such as an image sensor 161, a position sensor (IMU) 163, an audio sensor 165, a distance sensor, a proximity sensor, and a contact sensor.

[0082] Here, the image sensor 161 can capture images (images and / or videos, etc.) of the physical space around the terminal 100.

[0083] Specifically, the image sensor 161 can capture an image of a predetermined physical space through a camera disposed facing the outside of the terminal 100 .

[0084] In an embodiment, the image sensor 161 is disposed on the front and / or rear surface of the terminal 100 and can capture an image of the physical space in the direction in which the image sensor 161 is disposed.

[0085] Such an image sensor 161 may comprise an image sensor unit and an image processing module.

[0086] Specifically, the image sensor 161 can process still or moving images acquired by an image sensor device (eg, CMOS or CCD).

[0087] In addition, the image sensor 161 can use an image processing module to process still images or moving images acquired via the image sensor device to extract necessary information, and transmit the extracted information to the processor.

[0088] Such an image sensor 161 can be a camera assembly comprising at least one or more cameras.

[0089] Here, the camera assembly may include a general camera that captures visible light, and may further include a special camera such as an infrared camera or a stereo camera.

[0090] In addition, the above-mentioned image sensor 161 may be provided and operated in the terminal 100 depending on the embodiment, or may be provided in an external device (e.g., an external server, etc.) and operate through interlocking based on the above-mentioned communication processor 130 and / or interface unit 140.

[0091] The position sensor (IMU) 163 can sense at least one of the motion and acceleration of the terminal 100. For example, the position sensor (IMU) 163 can be composed of a combination of various sensors such as an accelerometer, a gyroscope, and / or a magnetometer.

[0092] In addition, the position sensor (IMU) 163 can recognize spatial information for the physical space around the terminal 100 in conjunction with the position communication processor 130 such as a GPS of the communication processor 130.

[0093] The audio sensor 165 can recognize sounds around the terminal 100 .

[0094] Specifically, the audio sensor 165 may include a microphone capable of sensing voice input from a user using the terminal 100 .

[0095] In an embodiment, the audio sensor 165 can receive audio data from a user required for the time series prediction service.

[0096] The interface unit 140 may communicatively couple the terminal 100 to one or more other devices.

[0097] Specifically, the interface unit 140 may include wired and / or wireless communication devices compatible with one or more different communication protocols.

[0098] Through the interface unit 140, the terminal 100 can be connected to various input / output devices.

[0099] For example, the interface unit 140 can be connected to an audio output device such as a headset port or a speaker to output audio.

[0100] Although the audio output device has been described as being connected via the interface unit 140 by way of example, an embodiment in which the audio output device is provided within the terminal 100 may also be included.

[0101] For example, the interface unit 140 may also be connected to an input device such as a keyboard and / or a mouse to obtain user input.

[0102] Such an interface unit 140 may be configured to include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, an earphone port, a power amplifier, an RF circuit, a transceiver, and other communication circuits.

[0103] The input system 150 can sense user input (eg, gestures, voice commands, button activations, or other types of input) associated with the time series prediction service.

[0104] Specifically, the input system 150 may include a predetermined button, a touch sensor, an image sensor 161 for sensing a user motion input, and / or an audio sensor 165 for sensing a user voice input.

[0105] In addition, the input system 150 is connected to an external controller via the interface unit 140 to receive user input.

[0106] The display system 170 can output various information related to the time series prediction service as graphic images.

[0107] In some embodiments, display system 170 may display various user interfaces, time series data, images, and / or text for the time series forecasting service.

[0108] Such a display may include at least one of, but is not limited to, a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and / or an e-ink display.

[0109] In addition, according to an embodiment, the display system 170 may include a display 171 that outputs an image, and a touch sensor 173 that senses a touch input by a user.

[0110] For example, the display 171 may be implemented as a touch screen by being mutually layered with the touch sensor 173 or being integrally formed therewith.

[0111] Such a touch screen can function as a user input unit that provides an input interface between the terminal 100 and a user, and can also provide an output interface between the terminal 100 and a user.

[0112] Meanwhile, the terminal 100 according to an embodiment of the present invention may perform deep learning related to a time series prediction service based on a predetermined deep-learning neural network.

[0113] Here, the deep learning neural network according to the embodiment is Custom Diffusion Deep-learning Neural Networks, Stable Diffusion Deep-learning Neural Networks, CLIP (Contrastive Language-Image Pre-training Encoder), VGG-19, GPT-3, GPT-3.5, EfficientNet, ResNet, ARIMA (Autoregressive Integrated Moving Average), VAR (Vector Auto Regression), RNN (Recurrent Neural Networks), LSTM (Long Short-Term Memory models), GRU (Gated Recurrent Unit), GAN (Generative Adversarial Networks), DualStyleGAN, StyleGAN, Graph Convolution Network (GCN), CNN (Convolution Neural Network, CNN), DPSNet (Deep Plane Sweep Network, DPSNet), AGN (Attention Guided Network, AGN), R-CNN (Regions with CNN features), Fast R-CNN, Faster The neural network may include, but is not limited to, R-CNN, Mask R-CNN, and / or U-Net network.

[0114] Specifically, in an embodiment, the terminal 100 may perform deep learning required for a time series prediction service in conjunction with at least one deep learning neural network capable of implementing a temporal relation-effect layer module according to an embodiment of the present invention.

[0115] On the other hand, depending on the embodiment, the terminal 100 may further perform at least a part of the functional operations performed by the service providing server 200, which will be described later.

[0116] <Service Provider Server 200>

[0117] Meanwhile, the service providing server 200 according to the embodiment of the present invention can perform a series of processes for providing a time series prediction service.

[0118] Specifically, in an embodiment, the service providing server 200 can provide a time series prediction service by exchanging data required for driving a time series prediction service process with an external device such as the terminal 100 .

[0119] More specifically, in an embodiment, the service providing server 200 can provide an environment in which the application 111 can operate on an external device (in an embodiment, the mobile type computing device 100-1 and / or the desktop type computing device 100-2, etc.).

[0120] To this end, the service providing server 200 may include application programs, data, and / or commands for the application 111 to operate, and may transmit and receive various data based on the application programs, data, and / or commands to and from the external device.

[0121] In addition, in an embodiment, the service providing server 200 may train a temporal relationship-effect layer module according to an embodiment of the present invention.

[0122] Here, the temporal relationship-effect layer module according to the embodiment may be a deep learning module that receives time series data at a predetermined first time point as input data and receives time series data at a predetermined second time point as output data, the time series data being predicted based on the input time series data at the first time point.

[0123] In addition, in an embodiment, the service providing server 200 can obtain time-series target data.

[0124] Here, the time series target data in the embodiment may refer to time series data to be input to the temporal relationship-effect layer module (i.e., time series data at a predetermined first time point for which time series prediction is to be performed via the temporal relationship-effect layer module).

[0125] In addition, in an embodiment, the service providing server 200 can input the acquired time-series target data into a temporal relationship-effect layer module.

[0126] In addition, in an embodiment, the service providing server 200 may obtain time-series prediction data in conjunction with the temporal relationship-effect layer module that receives the time-series target data.

[0127] Here, the time series prediction data according to the embodiment may refer to output data of the temporal relationship-effect layer module that receives the time series target data (i.e., time series data at a predetermined second point in time obtained through time series prediction deep learning based on the temporal relationship-effect layer module).

[0128] In addition, in the embodiment, the service providing server 200 can provide the acquired time-series prediction data in various ways.

[0129] In an embodiment, the service providing server 200 can provide time series prediction data based on a predetermined application service (eg, a voice recognition service and / or an automated trading service) that utilizes time series data.

[0130] In addition, in the embodiment, the service providing server 200 can perform deep learning required for the time series prediction service based on a predetermined deep-learning neural network.

[0131] Specifically, in an embodiment, the service providing server 200 can perform deep learning required for a time series prediction service in conjunction with at least one deep learning neural network capable of implementing a temporal relation-effect layer module according to an embodiment of the present invention.

[0132] More specifically, in an embodiment, the service providing server 200 can read from the memory module 230 a predetermined deep learning neural network driving program configured to perform the deep learning.

[0133] Then, the service providing server 200 can perform deep learning required for the time series prediction service according to the read-out predetermined deep learning neural network system.

[0134] Here, the deep learning neural network according to the embodiment includes Custom Diffusion Deep-learning Neural Networks, Stable Diffusion Deep-learning Neural Networks, CLIP (Contrastive Language-Image Pre-training Encoder), VGG-19, GPT-3, GPT-3.5, EfficientNet, ResNet, ARIMA (Autoregressive Integrated Moving Average), VAR (Vector Auto Regression), RNN (Recurrent Neural Networks), LSTM (Long Short-Term Memory models), GRU (Gated Recurrent Unit), GAN (Generative Adversarial Networks), DualStyleGAN, StyleGAN, Graph Convolution Network (GCN), CNN (Convolution Neural Network, CNN), DPSNet (Deep Plane Sweep Network, DPSNet), AGN (Attention Guided Network, AGN), R-CNN (Regions with CNN features), Fast R-CNN, Faster R-CNN, Mask The network may include, but is not limited to, R-CNN, and / or U-Net network.

[0135] In this case, depending on the embodiment, the deep learning neural network may be directly provided in the service providing server 200 or may be realized as a device and / or server separate from the service providing server 200.

[0136] In the following description, the deep learning neural network is provided in the service providing server 200, but is not limited to this.

[0137] In addition, in the embodiment, the service providing server 200 may store and manage various application programs, commands, and / or data for implementing the time series prediction service.

[0138] In an embodiment, the service providing server 200 can store and manage at least one or more time series data, data processing algorithms, deep learning algorithms, and / or user interfaces, etc.

[0139] In addition, in the embodiment, the service providing server 200 may transmit and receive various application programs, commands, and / or data for implementing a time series prediction service to and from the external device.

[0140] However, in the embodiment of the present invention, the functions and operations that the service providing server 200 can perform are not limited to those described above, and other functions and operations can also be performed.

[0141] Meanwhile, as further shown in FIG. 1, in an embodiment, the service providing server 200 may be realized by a predetermined computing device including at least one processor module 210 for data processing, at least one communication module 220 for data exchange with external devices, and at least one memory module 230 for storing various application programs, data, and / or commands for providing a time series prediction service.

[0142] Here, the memory module 230 may store at least one of an operating system (OS), various application programs, data, and commands for providing a time series prediction service.

[0143] The memory module 230 may also include a program area and a data area.

[0144] In this case, the program area according to the embodiment can be connected between an operating system (OS) that boots the server and the functional elements.

[0145] In addition, the data area according to the embodiment can store data generated by the use of the server.

[0146] Furthermore, the memory module 230 can be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and can also be a web storage that performs the storage function of the memory module 230 over the Internet.

[0147] Furthermore, the memory module 230 can be a removable recording medium on the server.

[0148] Meanwhile, the processor module 210 can control the overall operation of each of the above-mentioned units to realize a time series prediction service.

[0149] Specifically, the processor module 210 may be a system-on-chip SOC suitable for a server equipped with a central processing unit (CPU) and / or a graphics processing unit (GPU).

[0150] In addition, the processor module 210 can execute an operating system OS and / or application programs stored in the memory module 230 .

[0151] Moreover, the processor module 210 can control each component installed in the server.

[0152] Furthermore, the processor module 210 can communicate with each component internally via a system bus, and can include one or more predetermined bus structures such as a local bus.

[0153] In addition, the processor module 210 may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0154] In the above explanation, it has been described that the service providing server 200 according to an embodiment of the present invention performs the functional operations described above. However, various embodiments are possible, such as at least a part of the functional operations performed by the service providing server 200 being performed by an external device (e.g., the terminal 100, etc.) depending on the embodiment, and at least a part of the functional operations performed by the external device being further performed by the service providing server 200.

[0155] <Temporal Relation-Effect Layer Module (TREL)> FIG. 3 is a conceptual diagram for explaining the temporal relation-effect layer module TREL according to an embodiment of the present invention.

[0156] As shown in FIG. 3, the temporal relationship-effect layer module TREL according to an embodiment of the present invention may be a deep learning module that performs deep learning (hereinafter, time series prediction deep learning) using a predetermined first time point time series data D1 as input data and a predetermined second time point time series data D2 predicted based on the input first time point time series data D1 as output data.

[0157] FIG. 4 is an example of input and output data of the temporal relation-effect layer module TREL according to an embodiment of the present invention.

[0158] As shown in FIG. 4, for example, the temporal relationship-effect layer module TREL can take data at a first time point as input data and can take data at a second time point predicted based on the input data at the first time point as output data.

[0159] As another example, the temporal relationship-effect layer module TREL can take data at a first and second time point as input data, and can take data at a third time point predicted based on the input data at the first and second time points as output data.

[0160] As further shown in FIG. 3, in an embodiment, such a temporal relation-effect layer module TREL may comprise a temporal decomposition gate module (TDG), a temporal relation gate module (TRG), and a temporal effect gate module (TEG).

[0161] At this time, in FIG. 3, in order to prevent the characteristics according to the embodiment of the present invention from being deteriorated, it is described that the temporal relationship-effect layer module TREL includes the above-mentioned components.

[0162] However, it is obvious that a person of ordinary skill in the art would understand that, depending on the embodiment, in addition to the components shown in FIG. 3, other general-purpose components may be further included, or some of the components shown in FIG. 3 may be omitted.

[0163] Specifically, the temporal decomposition gate module TDG according to the embodiment may be a deep learning module that receives input data of a predetermined first time point time series data D1, and outputs data (P: hereinafter, time series pattern decomposition data) obtained by decomposing the input first time point time series data D1 into a plurality of mutually different time series patterns via a variational mode decomposition method (VMD).

[0164] To this end, in an embodiment, the temporal decomposition gate module TDG can perform learning according to the following [Formula 1] based on a training data set including a plurality of time series data and time series pattern decomposition data P corresponding to each of the time series data.

[0165]

number

[0166] In this case, the 'k' parameter may be a parameter representing the number of time series patterns decomposed through the variational mode decomposition method (VMD).

[0167] Moreover, the “j” parameter may be a parameter representing the number of dimensions of the hidden layer in the temporal decomposition gate module TDG.

[0168] In addition, The TIFF2025035994000003.tif24162 parameter may be a parameter representing the value of the j-th decomposed time series pattern at a given time point t (t>0).

[0169] In an embodiment, when the temporal decomposition gate module TDG receives predetermined time series data, the temporal decomposition gate module TDG can perform a training process of decomposing the input time series data into a plurality of different time series patterns based on the variation mode decomposition method VMD and learning the patterns through a linear layer based on [Equation 1].

[0170] Therefore, the temporal decomposition gate module TDG trained as described above can acquire the time series pattern decomposition data P from the predetermined first time point time series data D1 based on [Formula 1].

[0171] FIG. 5 is an example of a diagram for explaining output data of a temporally resolved gate module TDG according to an embodiment of the present invention.

[0172] As a result, in the embodiment, the temporal decomposition gate module TDG can output the time series pattern decomposition data P in the form shown in FIG.

[0173] In addition, the temporal relation gate module TRG according to the embodiment may be a deep learning module that receives the output data (i.e., time series pattern decomposition data P) of the temporal decomposition gate module TDG and k (k>0) pieces of concealment state data (H: hereinafter, concealment state matrix data) as input data, and outputs multivariable temporal relation analysis data (R: hereinafter, temporal relation analysis data) between the input data.

[0174] FIG. 6 is an example of a diagram for explaining the concealment state matrix data H according to an embodiment of the present invention.

[0175] Here, as shown in FIG. 6, for reference, the concealed state data may refer to data learned by accumulating past information based on a predetermined time point based on predetermined time series data.

[0176] That is, as an embodiment, the concealment state data can be data learned by accumulating past information based on a predetermined time point on the basis of the predetermined first time point time series data D1.

[0177] Therefore, the concealment state data for a given time t can be t-1 concealment state data, which is data obtained by accumulating and learning past information of the first time point time series data D1 up to time t-1.

[0178] In this case, the “k” parameter in FIG. 6 may be a parameter representing the number of time series patterns decomposed via the variational mode decomposition method (VMD).

[0179] Also, the "n" parameter in FIG. 6 can be a parameter representing the number of dimensions of the hidden state.

[0180] Returning further, in order to output the temporal relationship analysis data R, in an embodiment, the temporal relationship gate module TRG can perform learning according to the following [Formula 2] based on a training data set including a plurality of time series pattern decomposition data P and hidden state matrix data H.

[0181]

number

[0182] In this case, the α parameter can learn the relationship between the decomposed time series pattern at time t and the concealment state at time t-1 using a softmax function, and the input decomposed time series pattern and the concealment state that learned the previous state.

[0183] In addition, the W IH The parameters are the weights of the decomposed time series patterns and the concealment state, and are expressed as I t The parameters are: This means the entire input value of TIFF2025035994000005.tif24162.

[0184] In addition, the W HH The parameter H represents the weight between the hidden states. t-1 means the hidden state value at time t-1.

[0185] Therefore, the temporal relationship gate module TRG trained as described above can acquire the temporal relationship analysis data R based on the predetermined time series pattern decomposition data P and the concealment state matrix data H, based on [Formula 2].

[0186] In other words, the temporal relationship gate module TRG can acquire the temporal relationship analysis data R that analyzes the relationship between the predetermined time series pattern decomposition data P as shown in FIG. 5 and the predetermined concealment state matrix data H as shown in FIG. 6 based on [Formula 2].

[0187] Then, the temporal relationship gate module TRG can provide the obtained temporal relationship analysis data R as output data.

[0188] In addition, the temporal influence gate module TEG of the embodiment can be a deep learning module that receives the output data of the temporal decomposition gate module TDG (i.e., the time series pattern decomposition data P) and k pieces of concealment state data (i.e., the concealment state matrix data H) as input data, and outputs multivariable temporal effect analysis data E (hereinafter, temporal influence analysis data) between the input data.

[0189] For this purpose, in the embodiment, the temporal influence gate module TEG can perform learning according to the following [Formula 3] based on a training data set including a plurality of time series pattern decomposition data P and concealment state matrix data H.

[0190]

number

[0191] In this case, the β parameter can learn the influence between the decomposed time series pattern inputted at time t and the concealment state learned from the previous state using a hyperbolic tangent function, which is the weighted value of the decomposed time series pattern at time t and the concealment state at time t-1.

[0192] In addition, the W IH The parameters are the weights of the decomposed time series patterns and the concealment state, and are expressed as I t The parameters are: This means the entire input value of TIFF2025035994000007.tif20167.

[0193] In addition, the W H The parameter H represents the weight between the hidden states. t-1 means the hidden state value at time t-1.

[0194] Therefore, the temporal influence gate module TEG learned as described above can acquire the temporal influence analysis data E based on the predetermined time series pattern decomposition data P and the concealment state matrix data H, based on [Formula 3].

[0195] In other words, the temporal influence gate module TEG can obtain the temporal influence analysis data E that analyzes the degree of influence between the predetermined time series pattern decomposition data P as shown in FIG. 5 and the predetermined concealment state matrix data H as shown in FIG. 6 based on [Formula 3].

[0196] Then, the temporal influence gate module TEG can provide the acquired temporal relationship analysis data R as output data.

[0197] On the other hand, the temporal relationship and effect layer module TREL, which comprises modules as described above in the embodiments, is able to update the hidden state data.

[0198] Specifically, in an embodiment, the temporal relationship-effect layer module TREL can repeatedly execute functional operations based on the above-mentioned modules (i.e., the temporal decomposition gate module TDG, the temporal relationship gate module TRG, and the temporal influence gate module TEG) based on the time series data up to the Tth time point (T>1).

[0199] In the process, the temporal relationship-effect layer module TREL can perform updates to the hidden state data.

[0200] More specifically, the temporal relation-effect layer module TREL can update the hidden state data at time t through pattern learning based on the hidden state data at time t-1 and the input data at time t.

[0201] In addition, the temporal relation-effect layer module TREL can update the hidden state data at time t+1 through pattern learning based on the hidden state data at time t and the input data at time t+1.

[0202] In this case, the concealed state data at time t can be data in which information based on the input data at time t-1 and the input data at time t have both been learned.

[0203] That is, in the embodiment, the temporal relationship-effect layer module TREL performs the concealment state data update up to the Tth time point through the repetition of the above-mentioned functional operations, thereby realizing the concealment state data update using the input data from a predetermined time point t to a time point T-1.

[0204] At this time, in the embodiment, the temporal relationship-effect layer module TREL may perform a concealment state data update according to the following [Equation 4].

[0205]

number

[0206] At this time, the H t The parameters are calculated by multiplying the learned value of the relationship (α) between the decomposed time series pattern at time t and the concealment state at time t-1 by the learned value of the influence (β) between the decomposed time series pattern at time t and the concealment state at time t-1 using [Equation 2], and summing it with the input decomposed input time series data of the current state to obtain H t can be updated.

[0207] Going back further, this allows the temporal relationship-effect layer module TREL in an embodiment to perform time series prediction deep learning in which a predetermined first time point time series data D1 is used as input data, and a predetermined second time point time series data D2 predicted based on the input first time point time series data D1 is used as output data.

[0208] As described above, in the embodiment, the temporal relationship-effect layer module TREL can be realized with a structure that self-learns the relationship and influence between temporal factors contained in predetermined time series data at the deep learning neural network layer level.

[0209] Then, the temporal relationship-effect layer module TREL can perform time series prediction deep learning based on such a structure (i.e., a deep learning layer structure including a temporal decomposition gate module TDG, a temporal relationship gate module TRG, and a temporal influence gate module TEG).

[0210] Therefore, the temporal relationship-effect layer module TREL can perform time series prediction deep learning based on time series data with high accuracy by taking into account all of the numerous latent factors that may exist in the time series data, regardless of their form.

[0211] In other words, when time series data that changes over time is in multiple heterogeneous data formats, the temporal relationship-effect layer module TREL trained with time series data in a first data format can output time series prediction data for the time series data in the second data format even when it receives input based on time series data in the second data format.

[0212] <How to provide time series prediction deep learning neural network> Hereinafter, a method for providing a time series prediction deep learning neural network (i.e., a method for providing a time series prediction service) by the application 111 according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0213] In the embodiment of the present invention, at least one or more processors of the terminal 100 can execute at least one or more applications 111 stored in at least one or more memories 110 or can operate in a background state.

[0214] Hereinafter, the operation of at least one processor of the terminal 100 to execute commands of the application 111 to provide a time series prediction service will be referred to as "the application 111" for short.

[0215] FIG. 7 is a flowchart illustrating a method for providing a time series prediction deep learning neural network according to an embodiment of the present invention.

[0216] As shown in FIG 7, in the embodiment, an application 111 executed by at least one or more processors of the terminal 100 or operating in a background state can train a temporal relationship-effect layer module TREL (S101).

[0217] Specifically, in an embodiment, the application 111 can train a temporal relationship-effect layer module TREL according to an embodiment of the present invention.

[0218] In other words, the temporal relationship-effect layer module TREL according to the embodiment can be a deep learning module that receives input data of a predetermined first time point time series data D1 and outputs output data of a predetermined second time point time series data D2 predicted based on the input first time point time series data D1.

[0219] More specifically, in an embodiment, the application 111 can train a temporal decomposition gate module TDG of the temporal relationship-effect layer module TREL based on a training data set including a plurality of time series data and time series pattern decomposition data P corresponding to each of the time series data.

[0220] At this time, in the embodiment, the application 111 can perform the temporal decomposition gate module TDG learning based on the above-mentioned [Equation 1].

[0221] In addition, in an embodiment, the application 111 can train the temporal relation gate module TRG of the temporal relation-effect layer module TREL based on a training data set including a plurality of time series pattern decomposition data P and hidden state matrix data H.

[0222] At this time, in the embodiment, the application 111 can perform the temporal relationship gate module TRG learning based on the above-mentioned [Equation 2].

[0223] In addition, in the embodiment, the application 111 can train the temporal influence gate module TEG of the temporal relationship-effect layer module TREL based on a training data set including a plurality of time series pattern decomposition data P and hidden state matrix data H.

[0224] At this time, in the embodiment, the application 111 can perform the temporal influence gating module TEG learning based on the above-mentioned [Equation 3].

[0225] This allows, in an embodiment, the application 111 to train the temporal relationship-effect layer module TREL.

[0226] In addition, in the embodiment, the application 111 can obtain time-series target data (S103).

[0227] Here, the time-series target data in the embodiment may refer to time-series data to be input to the temporal relationship-effect layer module TREL (i.e., time-series data D1 at a predetermined first time point for which time-series prediction is to be performed via the temporal relationship-effect layer module TREL).

[0228] For reference, time series data may refer to a series of sequential data sets collected at specific time intervals over a given period of time.

[0229] Such time series data can be characterized in that the data order is specified according to the flow of time and that there is a correlation between successive data.

[0230] Specifically, in an embodiment, the application 111 can obtain the time-series target data as described above based on a link with an external server and / or a user input.

[0231] In addition, in an embodiment, the application 111 may input the acquired time series target data to the temporal relationship-effect layer module TREL (S105).

[0232] That is, in an embodiment, the application 111 may input the time-series target data obtained as described above as input data to the temporal relationship-effect layer module TREL, which has been trained as described above.

[0233] In addition, in the embodiment, the application 111 can obtain time series prediction data (S107).

[0234] Here, the time series prediction data according to the embodiment may refer to the output data of the temporal relationship-effect layer module TREL that receives the time series target data (i.e., the time series data D2 at a predetermined second point in time obtained through time series prediction deep learning based on the temporal relationship-effect layer module TREL).

[0235] Specifically, in an embodiment, the application 111 may obtain time-series forecast data in conjunction with a temporal relationship-effect layer module TREL, which receives time-series target data.

[0236] More specifically, in an embodiment, the temporal relationship-effect layer module TREL that receives the time-series target data can perform functional operations based on the above-mentioned temporal decomposition gate module TDG, the temporal relationship gate module TRG, and the temporal influence gate module TEG based on the input time-series target data.

[0237] That is, the temporal relationship-effect layer module TREL can work in conjunction with the temporal decomposition gate module TDG, the temporal relationship gate module TRG, and the temporal influence gate module TEG to perform time series prediction deep learning based on time series target data.

[0238] More specifically, the temporal relationship-effect layer module TREL can cooperate with the temporal decomposition gate module TDG to obtain time series pattern decomposition data P according to the time series target data.

[0239] In addition, the temporal relationship-effect layer module TREL can obtain temporal relationship analysis data R based on the time series pattern decomposition data P and the concealment state matrix data H based on the time series target data in cooperation with the temporal relationship gate module TRG.

[0240] In addition, the temporal relationship-effect layer module TREL can obtain temporal effect analysis data E based on the time series pattern decomposition data P and the concealment state matrix data H based on the time series target data in cooperation with the temporal effect gate module TEG.

[0241] At this time, the temporal relationship-effect layer module TREL can perform updates to the concealment state matrix data H based on the acquired data.

[0242] In addition, the temporal relationship-effect layer module TREL can obtain time series prediction data based on the obtained data (ie, the time series pattern decomposition data P, the temporal relationship analysis data R, and the temporal effect analysis data E).

[0243] That is, the temporal relationship-effect layer module TREL can obtain time series prediction data (i.e., second time point time series data D2) based on time series target data (i.e., first time point time series data D1) based on time series pattern decomposition data P, temporal relationship analysis data R, and temporal effect analysis data E obtained based on time series target data.

[0244] The temporal relationship-effect layer module TREL can then provide the obtained time series forecast data to the application 111 .

[0245] This allows the application 111 in the embodiment to obtain time-series prediction data based on the time-series target data from the temporal relationship-effect layer module TREL to which the time-series target data has been input.

[0246] In addition, in the embodiment, the application 111 can provide the acquired time series prediction data (S109).

[0247] That is, in the embodiment, the application 111 can provide the time-series prediction data acquired as described above according to a predetermined method.

[0248] In some embodiments, the application 111 may provide time series forecast data via a predefined application service (eg, a voice recognition service and / or an automated trading service, etc.) based on the time series forecast data.

[0249] In this manner, in the embodiment, the application 111 can provide time series forecast data acquired using the temporal relationship-effect layer module TREL according to the embodiment of the present invention by fusing it to various processes.

[0250] In other words, the application 111, in cooperation with the temporal relationship-effect layer module TREL, can obtain high-quality time-series prediction data based on the time-series target data regardless of the form of the time-series target data, and can support various application services by utilizing the obtained high-quality time-series prediction data.

[0251] Therefore, the application 111 can effectively improve the efficiency and performance of various service processes based on the given time series data.

[0252] As described above, the method and system for providing a time series prediction deep learning neural network according to one embodiment of the present invention provides a deep learning neural network that improves deep learning prediction performance based on time series data based on learning based on temporal relationships and temporal influences, thereby taking into account all of the numerous latent factors that may exist in the time series data, regardless of their form, and having the effect of performing time series prediction deep learning based on the time series data with high accuracy.

[0253] In addition, the method and system for providing a time series prediction deep learning neural network according to one embodiment of the present invention provides a deep learning neural network that is structured to self-learn the relationships and influences between temporal factors contained in specific time series data at the deep learning neural network layer level, thereby achieving time series prediction deep learning with high accuracy (recognition rate), and as a result, it has the effect of improving the performance and quality of various application services that utilize data.

[0254] Hereinafter, a service including a module for recognizing baby and adult voices will be described as an application service using the method for providing a time series prediction deep learning neural network. That is, when adult voices and baby voices are used as time series target data, they have different voice forms, and a method for learning and analyzing adult voices and baby voices having such different forms using one time series prediction deep learning neural network will be described.

[0255] <Temporal Relation-Effect Seq2Seq Layer Module (TSSL)> FIG. 8 is a conceptual diagram for explaining a temporal relationship-effect sequence-to-sequence layer module TSSL according to an embodiment of the present invention.

[0256] As shown in FIG. 8, the temporal relationship-effect sequence-to-sequence layer module TSSL according to an embodiment of the present invention may be a deep learning module that receives predetermined voice data (VD: in an embodiment, infant voice data VD) as input data and receives predetermined service generation data SGD based on the input voice data VD as output data.

[0257] In this case, the audio data according to the embodiment includes a plurality of audio frequency data that change over time, and therefore can have the shape of time-series data.

[0258] Here, the service generation data SGD in the embodiment may refer to data generated by performing a specific task based on predetermined voice data VD, i.e., a task specialized for the service, using a temporal relationship-effect sequence-to-sequence layer module TSSL in the process of a predetermined service (in the embodiment, an early childhood education platform service).

[0259] In an embodiment, if it is an early childhood education platform service, the service process may perform early childhood speech recognition based on a temporal relationship-effect sequence-to-sequence layer module TSSL.

[0260] At this time, the service process can obtain question data based on the specific infant voice data VD during the infant voice recognition process.

[0261] The service process can then generate answer data based on the acquired question data.

[0262] This enables the service process to provide service generation data SGD (here, response data) based on predetermined voice data VD (here, specific baby voice data VD).

[0263] As further shown in FIG. 8, in an embodiment, such a temporal relationship-effect sequence-to-sequence layer module TSSL may comprise a temporal encoder module (TEM), a first linear layer (Linear layer 1), a temporal decoder module (TDM), a transformer module (TFM), and a second linear layer (Linear layer 2).

[0264] At this time, in FIG. 8, in order to prevent degradation of the features according to an embodiment of the present invention, it is explained that the temporal relationship-effect sequence-to-sequence layer module TSSL includes the components as described above.

[0265] However, it is obvious that a person skilled in the art can understand that, depending on the embodiment, in addition to the components shown in FIG. 8, other general-purpose components may be further included, or some of the components shown in FIG. 8 may be omitted.

[0266] Specifically, the temporal encoder module TEM according to the embodiment may be an encoding module comprising N (N>1) temporal relation-effect layer modules TREL.

[0267] In this embodiment, the temporal encoder module TEM may include a number (here, N) of temporal relationship-effect layer modules TREL set according to a user input and / or a preset process algorithm.

[0268] Here, each temporal relation-effect layer module TREL (EREM: hereinafter, encoder relation-effect module) included in the temporal encoder module TEM can be a time series prediction deep learning neural network as shown in FIG. 3, which is a deep learning module that receives predetermined infant voice data VD as input data and outputs latent variable data z learned from the input infant voice data VD.

[0269] That is to say, the temporal encoder module TEM may be configured to comprise a plurality of layers of temporal relation-effect layer modules TREL.

[0270] In addition, the first linear layer according to the embodiment may be a deep learning layer that receives output data (i.e., latent variable data z) of the encoder relation-effect module EREM as input data, and outputs data (hereinafter, variable dimension conversion data) obtained by dimension-converting the input latent variable data z into an input data type of the temporal decoder module TDM and / or the transformer module TFM.

[0271] As an embodiment, the first linear layer may be a layer that outputs variable dimension transformation data generated by performing a linear operation based on the latent variable data z.

[0272] Furthermore, the temporal decoder module TDM according to the embodiment may be a decoding module comprising M (M>1) temporal relation-effect layer modules TREL.

[0273] In this embodiment, the temporal encoder module TEM may include a number (here, M) of temporal relationship-effect layer modules TREL set according to a user input and / or a preset process algorithm.

[0274] Here, each temporal relation-effect layer module TREL (DREM: hereinafter, decoder relation-effect module) included in the temporal decoder module TDM can be a time series prediction deep learning neural network of Figure 3, which is a deep learning module that receives as input data the output data of the first linear layer (i.e., variable dimension transformation data) and outputs as output data the infant voice production data VGD predicted based on the input variable dimension transformation data.

[0275] That is to say, the temporal decoder module TDM may be configured to comprise a temporal relation-effect layer module TREL of a number of layers.

[0276] In addition, the infant voice generation data VGD according to the embodiment may refer to result data acquired by the decoder relation-effect module DREM performing time-series deep learning based on the variable dimension transformation data.

[0277] In this embodiment, the baby voice production data VGD may then comprise production value data for generating the baby voice data VD input to the temporal encoder module TEM.

[0278] Thus, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL can be trained in a direction such that the infant voice data VD, which is the initial input data (i.e., input data to the temporal encoder module TEM), and the output data (i.e., infant voice generation data VGD) of the temporal decoder module TDM based on the infant voice data VD become similar to each other.

[0279] This allows the temporal relationship-effect sequence-to-sequence layer module TSSL to realize a deep learning neural network structure that improves the accuracy (recognition rate) of the speech recognition process based on the given infant speech data VD.

[0280] In addition, the transformer module TFM according to the embodiment can be a deep learning module that takes the output data of the first linear layer (i.e., variable dimension transformation data) as input data and outputs predetermined task data based on the input variable dimension transformation data.

[0281] Here, the task data in the embodiment may refer to data resulting from performing a task specific to the service (in the embodiment, an early childhood education platform service) in the process of a service based on a specified voice data VD.

[0282] For example, in the case of an early childhood education platform service, the task data may include answer data to question data by the infant voice data VD.

[0283] Meanwhile, for reference, the transformer module TFM can be a module including a deep learning neural network that can track inter-element relationships within a given time-series (sequential) data (e.g., word data in a sentence) and learn its context and meaning.

[0284] Such a transformer module TFM can refer to a deep learning module that follows a sequence-to-sequence structure of encoder and decoder, but is realized only with attention or self-attention.

[0285] Specifically, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL can be trained so that the infant voice data VD, which is the initial input data (i.e., input data to the temporal encoder module TEM), and the output data of the temporal decoder module TDM based on the infant voice data VD (i.e., infant voice generation data VGD) have a similarity greater than or equal to a predetermined value.

[0286] Thereafter, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL, which has been sufficiently trained to have a similarity greater than or equal to a predetermined value, can input the predetermined infant speech target data to the temporal encoder module TEM.

[0287] In addition, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL may cooperate with the temporal encoder module TEM that receives the infant voice target data to obtain latent variable data z (hereinafter, latent variable target data) based on the infant voice target data.

[0288] In addition, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL may input the acquired latent variable target data into a first linear layer.

[0289] In addition, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL may cooperate with the first linear layer that receives the latent variable target data to obtain variable dimension transformation data (hereinafter, variable dimension transformation target data) based on the latent variable target data.

[0290] In addition, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL may input the acquired variable dimension transformation target data to the transformer module TFM.

[0291] Then, the transformer module TFM that receives the latent variable target data can generate task data (hereinafter, task target data) based on the input latent variable target data.

[0292] To this end, in an embodiment, the transformer module TFM may be pre-trained based on a predetermined training data set including a plurality of latent variable data z and task data corresponding to each of the plurality of latent variable data z.

[0293] As a result, the temporal relationship-effect sequence-to-sequence layer module TSSL can obtain task target data by variable dimension transformation target data through linkage with the transformer module TFM.

[0294] In this case, according to an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL may additionally train the transformer module TFM based on a training data set further including variable dimension transformation target data and task target data.

[0295] In this manner, in the embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL can realize various task processing for various speech recognition-based services via the transformer module TFM, and in this case, by utilizing the temporal relationship-effect layer module TREL as described above, the accuracy (recognition rate) of the speech recognition task processing can be significantly improved.

[0296] In addition, the second linear layer according to the embodiment may be a deep learning layer that receives output data (i.e., task data) of the transformer module TFM as input data, and outputs data (i.e., service generation data SGD) obtained by dimensionally converting the input task data into the output data type (i.e., service generation data SGD type) of the temporal relationship-effect sequence-to-sequence layer module TSSL.

[0297] In the embodiment, the second linear layer may be a layer that performs a linear operation based on the task data and outputs the generated service generation data SGD.

[0298] As described above, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL can recognize a given voice data VD (in an embodiment, infant voice data VD) with high accuracy using an encoder, decoder, and transformer based on the temporal relationship-effect layer module TREL, thereby improving the quality of the acquired service generation data SGD (in an embodiment, response data, etc.) and further enhancing the performance and usability of various voice recognition-based services (in an embodiment, early childhood education platform services).

[0299] <Method of providing an early childhood education platform service based on time series prediction deep learning neural network> Hereinafter, a method for implementing an early childhood education platform providing service based on a time series prediction deep learning neural network having the temporal relationship-effect layer module TREL by an application 111 executed by at least one processor of a terminal 100 according to an embodiment of the present invention will be described in detail with reference to the attached FIG. 9.

[0300] In the embodiment of the present invention, at least one or more processors of the terminal 100 can execute at least one or more applications 111 stored in at least one or more memories 110 or can operate in a background state.

[0301] Hereinafter, the at least one processor operates to execute the commands of the application 111 to perform the method for realizing the above-mentioned early childhood education platform providing service, which will be referred to as being performed by the application 111.

[0302] FIG. 9 is a flowchart illustrating a method for implementing an early childhood education platform providing service according to an embodiment of the present invention.

[0303] As shown in Figure 9, in an embodiment, the application 111 can train a temporal relationship-effect sequence-to-sequence layer module TSSL (S201).

[0304] Specifically, in an embodiment, the application 111 can train a temporal relationship-effect sequence-to-sequence layer module TSSL, which is composed of N (N>1) temporal relationship-effect layer modules TREL.

[0305] Here, the temporal relationship-effect sequence-to-sequence layer module TSSL of the embodiment may be a deep learning module that receives predetermined voice data (VD: in an embodiment, infant voice data VD) as input data and receives predetermined service generation data SGD (in an embodiment, question and answer data and / or answer data) based on the input voice data VD as output data.

[0306] More specifically, as shown in FIG. 3, in an embodiment, the application 111 can train the temporal decomposition gate module TDG of the temporal relationship-effect sequence-to-sequence layer module TSSL based on a training data set constructed based on a plurality of time series data based on the infant voice data VD and time series pattern decomposition data P corresponding to each of the time series data.

[0307] At this time, in the embodiment, the application 111 can perform the temporal decomposition gate module TDG learning based on the above-mentioned [Equation 1].

[0308] In addition, in the embodiment, the application 111 can train the temporal relationship gate module TRG of the temporal relationship-effect sequence-to-sequence layer module TSSL based on a training data set including a plurality of time series pattern decomposition data P based on the infant voice data VD and hidden state matrix data H.

[0309] At this time, in the embodiment, the application 111 can perform the temporal relationship gate module TRG learning based on the above-mentioned [Equation 2].

[0310] In addition, in the embodiment, the application 111 can train the temporal influence gate module TEG of the temporal relationship-effect sequence-to-sequence layer module TSSL based on a training data set including a plurality of time series pattern decomposition data P based on the infant voice data VD and the hidden state matrix data H.

[0311] At this time, in the embodiment, the application 111 can perform the temporal influence gating module TEG learning based on the above-mentioned [Equation 3].

[0312] This allows, in an embodiment, the application 111 to perform temporal relationship-effect sequence-to-sequence layer module TSSL learning in accordance with an embodiment of the present invention.

[0313] In addition, in an embodiment, the application 111 can train the transformer module TFM of the temporal relationship-effect sequence-to-sequence layer module TSSL based on a predetermined training data set including a plurality of latent variable data z and task data corresponding to each of the plurality of latent variable data z.

[0314] As a result, in an embodiment, application 111 can be trained to use predetermined voice data VD (in an embodiment, infant voice data VD) as input data and to use predetermined service generation data SGD based on the input voice data VD as output data.

[0315] That is, in the embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL automatically analyzes the difference between adult voice and infant voice based on the finally learned temporal encoder module TEM, first linear layer (Linear layer 1), temporal decoder module TDM, transformer module TFM, and / or second linear layer (Linear layer 2) to perform feature learning, and then, based on the learned infant voice features, can recognize a predetermined infant voice data VD input to the temporal relationship-effect sequence-to-sequence layer module TSSL with high accuracy.

[0316] Returning further, in the embodiment, the application 111 can obtain real-time audio (S203).

[0317] Specifically, in an embodiment, the application 111 can obtain real-time audio based on a specific user interface implemented by the early childhood education platform providing service.

[0318] Here, the real-time voice according to the embodiment may be voice data including voices of at least one of a child using the early childhood education platform providing service and / or a guardian of the child.

[0319] In addition, in the embodiment, the application 111 can detect the baby voice data VD from the acquired real-time voice (S205).

[0320] Specifically, in an embodiment, the application 111 can detect the baby voice data VD from the real-time voice file based on a deep learning model that utilizes speaker separation and / or speaker segmentation techniques.

[0321] For example, the application 111 may utilize a bottom-up approach or a top-down approach technique in a method of training a deep learning model using the speaker separation and / or speaker segmentation technique. That is, based on such a deep learning model, at least one or more pieces of baby voice data VD can be detected from the real-time voice uttered by at least one or more speakers.

[0322] In this case, if a plurality of voice data are detected, in the embodiment, the application 111 may determine the baby voice data VD corresponding to a preset pronunciation and / or frequency range. In the following description, it is assumed that the determined baby voice data VD is one.

[0323] For example, the frequency range preset for determining the baby voice data VD may be 270 to 500 Hz, which is the fundamental frequency when an infant speaks.

[0324] In an embodiment, the application 111 may also convert the detected baby voice data VD into a text file.

[0325] In an embodiment, application 111 may also vectorize the converted text infant voice data VD.

[0326] In addition, in an embodiment, the application 111 may input the detected baby voice data VD to the learned temporal relationship-effect sequence-to-sequence layer module TSSL (S207).

[0327] In this embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL may be a deep learning neural network that performs feature learning on a plurality of infant voice data VD, and then combines the learned infant voice features with a transformer module TFM to recognize the input infant voice data VD with a high recognition rate.

[0328] More specifically, the temporal relationship-effect sequence-to-sequence layer module TSSL, in conjunction with the temporal decomposition gate module TDG, the temporal relationship gate module TRG, and the temporal influence gate module TEG, can recognize the input infant voice data VD with a high recognition rate.

[0329] In addition, in an embodiment, the learned temporal relationship-effect sequence-to-sequence layer module TSSL may analyze questions, contents, and / or emotions, etc., contained in the input infant voice data VD.

[0330] Hereinafter, data obtained by analyzing questions, contents, and / or emotions contained in the baby voice data VD may be referred to as question data.

[0331] That is, in the embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL can output question data obtained by analyzing the input baby voice data VD.

[0332] In addition, in an embodiment, the temporal relationship-effect sequence-to-sequence layer module TSSL can output corresponding answer data based on the output question data.

[0333] That is, in the embodiment, the application 111 may generate answer data based on the question data output from the temporal relationship-effect sequence-to-sequence layer module TSSL (S209).

[0334] Here, the answer data according to the embodiment may be data including answers corresponding to the questions, contents, and / or feelings included in the question and answer data.

[0335] In other words, in an embodiment, the application 111 can obtain answer data output based on the temporal relationship-effect sequence-to-sequence layer module TSSL for the question data.

[0336] For example, assuming that the question and answer data output through a series of processes contains the content "What color is an apple?", the answer data generated based on that data can be "It's either red or green."

[0337] In this case, the answer data according to the embodiment is output from the temporal relationship-effect sequence-to-sequence layer module TSSL, but in other embodiments, the answer data can also be generated by the application 111 extracting the answer data from a data set in which question data and answer data are pre-matched.

[0338] In an embodiment, application 111 may provide such pleading data to terminal 100 as text, audio, and / or images.

[0339] In addition, in the embodiment, the application 111 can detect parent voice data from the real-time voice (S211).

[0340] Here, the guardian voice data may be an adult voice that reproduces the same content that the infant is trying to say. In other words, the guardian is considered to be an adult.

[0341] At this time, the method of detecting the guardian voice data is the same as the method of detecting the baby voice data VD in step S205 described above, so the relevant content is applied mutatis mutandis and will not be repeated.

[0342] The reason for detecting the guardian voice data is that adult voice is transmitted as more accurate pronunciation and enunciation than infant voice, and has a higher voice recognition rate, so it can be used to verify the contents of the infant voice data VD.

[0343] In an embodiment, application 111 may also input the parent voice data into a temporal relationship-effect sequence-to-sequence layer module TSSL.

[0344] Thus, in the embodiment, the application 111 can obtain the parental voice analysis data output from the temporal relationship-effect sequence-to-sequence layer module TSSL using the detected parental voice data as input data (S213).

[0345] Here, the parental voice analysis data according to the embodiment may be data including questions, contents, and / or emotions included in the parental voice data.

[0346] In this case, the question data and the parent voice analysis data may differ. This is because the pronunciation of the same word by an infant and an adult may differ. In this embodiment, for convenience of explanation, it is assumed that the parent voice analysis data (i.e., the pronunciation by an adult) is absolutely correct.

[0347] In addition, in the embodiment, the application 111 may fine-tune a predetermined parameter included in the question data based on the acquired parental voice analysis data, or may change the entire question data into parental voice analysis data. In the following description, for convenience of explanation, it is assumed that the question data is changed into parental voice analysis data.

[0348] To this end, in an embodiment, application 111 may compare the question data and / or the parental voice analysis data.

[0349] In this case, in the embodiment, if the contents included in the question data and / or the guardian voice analysis data match, the application 111 may skip step S215 and perform step S217.

[0350] However, in an embodiment, if there is a mismatch between the question data and / or the parental voice analysis data, the application 111 may change the question data to the parental voice analysis data.

[0351] For example, the application 111 can change the word "Ium" contained in the question data to the word "Ilum (name)" contained in the parental voice analysis data.

[0352] In an embodiment, the application 111 can also generate second response data to the parental voice analysis data.

[0353] In other words, by changing the question data to parental voice analysis data, the first answer data corresponding to the question data and the second answer data corresponding to the parental voice analysis data can be different, and therefore the second answer data can be newly generated.

[0354] As a result, even if the question data is erroneously analyzed and a content having a different meaning is output, the erroneously analyzed voice analysis result is correctly corrected according to the parent's voice analysis data, thereby improving the accuracy of the infant voice recognition of the temporal relationship-effect sequence-to-sequence layer module TSSL.

[0355] In addition, in an embodiment, the application 111 may update the temporal relationship-effect sequence-to-sequence layer module TSSL based on the acquired parental voice analysis data (S215).

[0356] To update the temporal relationship-effect sequence-to-sequence layer module TSSL, in an embodiment, application 111 can re-input the infant voice data VD and second response data into the temporal relationship-effect sequence-to-sequence layer module TSSL.

[0357] Thus, the temporal relationship-effect sequence-to-sequence layer module TSSL can be retrained to output the second response data when the infant voice data VD is received.

[0358] In another embodiment, the application 111 can also update the temporal relationship-effect sequence-to-sequence layer module TSSL by fine-tuning the parameters of the temporal relationship-effect sequence-to-sequence layer module TSSL.

[0359] In addition, in the embodiment, the application 111 may provide an early childhood education platform service based on the updated temporal relationship-effect sequence-to-sequence layer module TSSL (S217).

[0360] Here, the infant education platform providing service according to the embodiment may be a service that provides a corresponding predetermined answer as text and / or audio when a user (specifically, an infant) inputs a predetermined voice into the audio sensor 165 of the terminal 100, or provides a program that matches the input voice. The infant education platform providing service may also include an education program that receives a user's voice for predetermined learning materials and content provided as a display and obtains a score.

[0361] For this, the answer data according to the embodiment may include a command for instructing the execution of a predetermined early childhood education program. That is, in the embodiment, the application 111 may preset a predetermined early childhood education program for each predetermined word detected in the answer data.

[0362] For example, if a user inputs the voice "I want to play Hangul," application 111 can execute a Hangul play program that has been pre-matched to the word "play Hangul" in accordance with a command to execute the Hangul play program and provide it to the user.

[0363] In an embodiment, the infant education platform providing service may basically proceed with a predetermined process according to the voice of an infant.

[0364] At this time, the infant education platform providing service can be used by infants alone and / or infants and guardians together, but as mentioned above, simultaneous use by infants and guardians may be recommended since the accuracy of the answer data can be improved by the guardian's voice data.

[0365] Meanwhile, the above-described embodiments of the present invention may be realized in the form of program instructions that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, and the like. The hardware devices may be modified into one or more software modules to perform the processes according to the present invention, and vice versa.

[0366] The specific implementations described in the present invention are merely examples and are not intended to limit the scope of the present invention in any manner. For the sake of brevity, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, connections such as lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be shown as various functional connections, physical connections, or circuit connections that can be substituted or added in an actual device. In addition, unless specifically mentioned as "essential" or "important," a component may not necessarily be necessary for application of the present invention.

[0367] In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, it will be understood by those skilled in the art or those having ordinary knowledge in the art that the present invention can be modified and changed in various ways without departing from the spirit and technical scope of the present invention as described in the claims below. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

Claims

1. 1. A method for providing a time series forecasting deep learning neural network in a time series forecasting application executed by at least one processor of a terminal, the method comprising: training a Temporal Relation-Effect Layer Module; inputting first time point time series data into the temporal relationship-effect layer module; Obtaining second time point time series data according to the first time point time series data from the temporal relationship-effect layer module; providing the second point in time time series data; Including, The temporal relationship-effect layer module includes: A method for providing a time series prediction deep learning neural network, comprising a temporal decomposition gate module, a temporal relation gate module, and a temporal effect gate module.

2. The temporally resolved gating module comprises:

2. The method for providing a time series prediction deep learning neural network according to claim 1, wherein the deep learning module uses the time series data at the first time point as input data, and uses time series pattern decomposition data obtained by decomposing the time series data at the first time point into a plurality of mutually different time series patterns via a variational mode decomposition method as output data.

3. The step of training the temporal relationship-effect layer module includes:

3. The method for providing a time series prediction deep learning neural network according to claim 2, further comprising: training the temporal decomposition gate module based on a training data set comprising a plurality of time series data and time series pattern decomposition data corresponding to each of the plurality of time series data.

4. The temporal relationship gate module includes:

3. The method for providing a time series prediction deep learning neural network according to claim 2, wherein the deep learning module receives as input data concealment state matrix data having k (k>0) pieces of concealment state data and the time series pattern decomposition data, and outputs as output data temporal relationship analysis data, which is data obtained by analyzing a multivariable temporal relationship between the concealment state matrix data and the time series pattern decomposition data.

5. The step of training the temporal relationship-effect layer module includes:

5. The method for providing a time series prediction deep learning neural network according to claim 4, further comprising: training the temporal relationship gate module based on a training data set comprising a plurality of hidden state matrix data and a plurality of time series pattern decomposition data.

6. The temporal impact gating module includes:

3. The method for providing a time series prediction deep learning neural network according to claim 2, wherein the input data is concealment state matrix data having k (k>0) pieces of concealment state data and the time series pattern decomposition data, and the output data is temporal effect analysis data, which is data obtained by analyzing a multivariable temporal effect between the concealment state matrix data and the time series pattern decomposition data.

7. The step of training the temporal relationship-effect layer module includes: The method for providing a time series prediction deep learning neural network according to claim 6, further comprising: training the temporal influence gate module based on a training data set comprising a plurality of hidden state matrix data and a plurality of time series pattern decomposition data.

8. The concealment state data is The method for providing a time series prediction deep learning neural network according to claim 4 or 6, wherein the data is learned by accumulating past information of the first time point time series data based on a predetermined time point.

9. The step of acquiring the second time point time series data includes: updating the concealment state data at a given time t through pattern learning based on the input data at the time t and the concealment state data at time t-1; updating the concealment state data at time t+1 through pattern learning based on the input data at time t+1 and the concealment state data at time t; 9. The method for providing a time series forecasting deep learning neural network according to claim 8, comprising:

10. The step of acquiring the second time point time series data includes:

2. The method for providing a time series prediction deep learning neural network according to claim 1, further comprising a step of acquiring the second time point time series data based on time series pattern decomposition data which is output data of the temporal decomposition gate module, temporal relationship analysis data which is output data of the temporal relationship gate module, and temporal influence analysis data which is output data of the temporal influence degree gate module.

11. The step of providing the second time point time series data includes: The method for providing a time series prediction deep learning neural network according to claim 1, further comprising providing the second time point time series data based on a predetermined application service.

12. At least one memory having a time series forecasting application stored therein; at least one processor that reads a time series forecasting application stored in the memory to provide a time series forecasting deep learning neural network; Equipped with The command for the time series prediction application is: training a Temporal Relation-Effect Layer Module; inputting first time point time series data into the temporal relationship-effect layer module; Obtaining second time point time series data according to the first time point time series data from the temporal relationship-effect layer module; providing the second point in time time series data; Including a command to The temporal relationship-effect layer module includes: A system for providing a time series prediction deep learning neural network, comprising a temporal decomposition gate module, a temporal relationship gate module, and a temporal effect gate module.

13. A method for providing an early childhood education platform service by an early childhood education application executed by at least one processor of a terminal, comprising: training a Temporal Relation-Effect Sequence-to-Sequence Layer Module; acquiring real-time audio; detecting infant voice data from the captured real-time voice; inputting the detected infant speech data into the learned temporal relationship-effect sequence-to-sequence layer module; generating answer data based on the question data output from the time relationship-effect sequence-to-sequence layer module; Providing an early childhood education platform service based on the generated answer data; A method for recognizing infant speech based on a time series predictive deep learning neural network including:

14. the temporal relationship-effect sequence-to-sequence layer module: The method for recognizing infant speech based on a time series prediction deep learning neural network according to claim 13, comprising a temporal encoder module (Temporal Encoder Module) which is an encoding module having a plurality of temporal relationship-effect layer modules, and a temporal decoder module (Temporal Decoder Module) which is a decoding module having a plurality of temporal relationship-effect layer modules.

15. The temporal relationship-effect layer module includes: The method for recognizing infant voice based on a time series prediction deep learning neural network of claim 14, comprising a Temporal Decomposition Gate Module, a Temporal Relation Gate Module, and a Temporal Effect Gate Module.

Images