Data processing device, method, and program

Abstract

A data processing device according to one embodiment comprises: an input unit that receives an input of data including time to a model that determines output value of a monotonically increasing function for data including time and outputs data having a periodic component by using a function for determining a value based on the square of sine for the output value; and an estimation unit that estimates parameters of the model by minimizing an objective function defined by correct answer information about the data having the periodic component and an estimation value of the data having the periodic component output by the model.

Description

Data processing device, method, and program

[0001] Embodiments of the present invention relate to data processing devices, methods, and programs.

[0002] In recent data analysis, much of the data analyzed is represented as periodic data with repeating patterns at regular intervals. For example, the human electrocardiogram waveform, which is often analyzed in the medical field, has a structure in which P waves, QRS waves, and T waves repeat.

[0003] Besides electrocardiogram waveforms, many physiological processes, such as heart rate, blood pressure, and hormone levels, exhibit cyclical patterns, and analyzing these is known to be useful in diagnosing diseases, understanding circadian rhythms, and developing treatments.

[0004] Similarly, data on indoor and outdoor environmental conditions, such as temperature or room temperature, humidity, and carbon dioxide levels, as well as energy consumption data such as electricity or gas, show daily or seasonal cycles, with patterns such as lower temperatures or room temperatures during colder periods, or increased energy consumption due to the need for heating and cooling. The results of analyzing these cycles can be used by power companies for demand forecasting and other purposes.

[0005] If it is known in advance that the data consists only of periodic components, then analysis is possible after an appropriate transformation, such as a Fourier transform. However, in many situations, the data may contain non-periodic components, or it may not be known in advance whether or not it contains periodic components.

[0006] To avoid bias caused by adopting a specific approach in such cases, attempts have been made to model both periodic and aperiodic functions using neural networks (see, for example, Non-Patent Document 1). Since many existing neural network methods are not suitable for appropriately representing periodic components, Non-Patent Document 1 introduces a function called the snake function, "x + sin 2By using a neural network with an activation function based on (x), it is possible to create a model that can capture periodic components. Hereafter, this neural network will be called a snake neural network or SNN.

[0007] Liu Ziyin, Tilman Hartwig, and Masahito Ueda. Neural networks fail to learn periodic functions and how to fix it. Advances in Neural Information Processing Systems, Vol.33, pp. 1583‐1594, 2020.

[0008] However, SNNs are not structured in a way that is suitable for representing multiple data with different periods. For example, while the electrocardiogram waveform mentioned above basically has a repeating structure of P waves, QRS waves, and T waves, the period changes depending on the context, such as being shorter at rest and longer during exercise. This context refers to the changes in the content of the data. SNNs do not have a mechanism to handle such changes in period, and are therefore not suitable for representing multiple waveforms with different periods.

[0009] This invention was made in view of the above circumstances, and its purpose is to provide a data processing device, method, and program that can appropriately represent data having periodic components.

[0010] A data processing device according to one aspect of the present invention includes an input unit that receives data including time as input to a model that outputs data having periodic components using a function that determines the output value of a monotonically increasing function for data including time, and determines a value based on the square of the sine of the output value; and an estimation unit that estimates the parameters of the model by minimizing an objective function defined by the correct information of the data having periodic components and the estimated value of the data having periodic components by the model.

[0011] A data processing method according to one aspect of the present invention is a method performed by a data processing device, comprising: an input unit of the data processing device receiving input data including time to a model that outputs data having periodic components using a function that obtains the output value of a monotonically increasing function for data including time, and obtains a value based on the square of the sine of the output value; and an estimation unit of the data processing device estimating the parameters of the model by minimizing an objective function defined by the correct information of the data having periodic components and the estimated value of the data having periodic components by the model.

[0012] According to the present invention, data having periodic components can be appropriately represented.

[0013] Figure 1 shows an example of a snake function. Figure 2 shows an example of estimation results using an SNN. Figure 3 shows an example of estimation results using a WSNN. Figure 4 shows an example of estimation results using a WSNN in a future context. Figure 5 shows an example of estimation results using a WSNN in a future context. Figure 6 shows an example of estimation results using a WSNN in a future context. Figure 7 shows an example of estimation results using a WSNN in a future context. Figure 8 shows an example of application of a data processing device according to one embodiment of the present invention. Figure 9 is a flowchart showing an example of the overview operation of a data processing device according to one embodiment of the present invention. Figure 10 is a block diagram showing an example of the hardware configuration of a data processing device according to one embodiment of the present invention.

[0014] Hereinafter, embodiments of the present invention will be described. In this embodiment, an example of constructing a Warped Snake Neural Network (WSNN), which is a new neural network that introduces a mechanism capable of responding to changes in the period into the SNN, will be described. The key to constructing the WSNN is to introduce a module (hereinafter referred to as a time warping module) that converts the time information received as an input using a monotonically increasing function. By this time warping module, a mechanism for adjusting the scale of the time axis or changing the speed of time progression is introduced, and a neural network structure suitable for expressing a plurality of waveforms with different periods can be obtained.

[0015] Next, the snake function and the neural network and SNN using this will be described. The snake function having the parameters shown in the following (1) is defined as in the following (2).

[0016]

[0017] The parameter a shown in (2) can be treated as a learnable one or can be a fixed value.

[0018] FIG. 1 is a diagram showing an example of the snake function. The snake function is proposed as an activation function that brings excellent properties when expressing a neural network for a function having periodicity and is disclosed in Non-Patent Document 1. When a simple sin function is used as the activation function to introduce periodicity, since the same function is obtained when shifted by 2π, infinitely many local minima occur in the solution, making it difficult to optimize the parameters.

[0019] The snake function uses “x + sin 2 (x)” to avoid the above occurrence. It is theoretically shown by a theorem called the Universal Extrapolation Theorem that a neural network with one hidden layer using the snake function can represent any periodic function under a sufficient number of neurons and can learn periodicity in an unbounded region.

[0020] Also, it has been experimentally reported that the relu function or the tanh function, which are standard activation functions of neural networks, can accurately approximate waveforms with periodicity that are difficult to represent.

[0021] By using the above snake function as the activation function of the neural network instead of existing functions such as the relu function or the tanh function, an SNN is constructed. For example, in a fully connected neural network having only L hidden layers, the neural network often used in regression problems and the like is a neural network that uses the relu function as the activation function as shown in the following (3), (4), and (5). u (1) =h Linear (x;W in ,b in ), z (1) =f relu (u (1) ) …(3) u (l) =h Linear (z (l－1) ;W (l) ,b (l) ), z (l) =f relu (u (l) ), (l = 2,…,L) …(4) y = h Linear (z (L) ;W out ,b out ) …(5) However, x and y are the input and output of the neural network, respectively, and the following (6) is the output of the hidden layer.

[0022]

[0023] Also, h Linear and f relu are a linear combination function and a relu function defined by the following (7). h Linear (x;W,b) = Wx + b, f relu (u) = max(u,0) …(7) Also, the following (8) are the parameters of the linear combination function.

[0024]

[0025] SNNs can be constructed by replacing all or part of the Relu function in the above neural network with the snake function. In other words, when constructing an SNN based on (3), (4), and (5) above, the SNN can be defined as follows: (9), (10), and (11). (1) =h Linear (x;W in ,b in ), z (1) =f Snake (u (1) ;a (1) ) …(9) u (l) =h Linear (z (l－1) ;W (l) ,b (l) ), z (l) =f Snake (u (l) ;a (l) ), (l=2,…,L) …(10) y=h Linear (z (L) ;W out ,b out ) ... (11) However, (12) below are the parameters of the snake function.

[0026]

[0027] Similarly, an SNN can be constructed using the snake function based on the structure of any neural network. Furthermore, it is possible to use neural networks that take image information as input, such as CNNs (Convolutional Neural Networks), VAEs (Variational Auto-Encoders), or GANs (Generative Adversarial Networks), or neural networks that handle time-series information, such as RNNs (Recurrent Neural Networks), LSTMs (Long Short-Term Memory), or Transformers, as the base. Therefore, regardless of the neural network structure, the following notation (13) will be used to represent the input and output of the SNN: y = SNN(x;η) … (13) where η represents all the parameters internally possessed by the SNN.

[0028] As mentioned above, SNNs are neural networks that can represent periodic components, but they lack a mechanism to handle contextual changes in period, making them unsuitable for representing multiple waveforms with different periods.

[0029] In this embodiment, a new method called WSNN, suitable for modeling data with periodic components whose period length varies depending on the context, was constructed by utilizing a neural network that incorporates two elements: an activation function using a snake function and a time-warping module. The time-warping module, as shown below, introduces a mechanism that is not present in SNNs, which adjusts the scale of the time axis and changes the speed at which time progresses.

[0030] (Formulation) Before showing the model structure of WSNN, let's clarify the input-output relationship of WSNN. The inputs of WSNN are time information and context information. The time information is represented by (14) below, and the context information is represented by (15) below.

[0031]

[0032] In (14), d represents the vector size of the contextual information. The output of the WSNN is some output value at input time x under context c. This output value is represented in (16) below and is, for example, a value observed by a sensor such as electrocardiogram voltage, or a label indicating whether the device is operational.

[0033]

[0034] Assuming that n data representing the input-output relationship are given as shown in (17) below, these will be used as data for learning the above input-output relationship.

[0035]

[0036] Contextual information c is applied to maintain the generality of the model and the data being handled, and the following descriptions can be applied in much the same way even if contextual information c is not used or does not exist.

[0037] (Time Warping Module) As mentioned above, the time warping module is a module that adjusts the scale of the time axis and changes the speed at which time progresses. Since time is an irreversible variable, it is necessary to perform modeling that preserves monotonically increasing properties with respect to the time variable x. Examples of monotonically increasing functions include constant functions, linear functions, parabolic functions, hyperbolic functions, exponential functions, logistic functions, sigmoid functions, or the cumulative density function of any probability distribution. Furthermore, since the sum of monotonically increasing functions is a monotonically increasing function, a weighted sum calculated using positive weight parameters of the above-mentioned monotonically increasing functions may also be used. Therefore, for example, the following modeling example can be considered.

[0038] (Example 1) This example describes a model that takes a product with one-dimensional context information. The size of the context information is d=1 and the value is positive, that is, the value shown in (18) below.

[0039]

[0040] The function g shown in (19) below at this time Ex1 g is a monotonic function in x. Ex1(x,c)=cx …(19) (Example 2) This example explains mixed modeling of the sigmoid function. K parameters take positive values ​​by taking the exponent of the sigmoid function in the following (20) exp(w k The function g shown in (21) below is weighted by ). Ex2 This is a monotonic function with respect to x. σ(x) = 1 / {1 + exp(-x)} … (20) Note that (22) below are the parameters of the model.

[0041]

[0042] In addition to the above, models that represent monotonically increasing functions using neural networks can also be used, such as the model disclosed in Reference 1, "Hyunho Kim and Jong-Seok Lee. Scalable monotonic neural networks. In The Twelfth International Conference on Learning Representations, 2024." Hereafter, regardless of the structure used as the time-warping module, the function representing this input and output will be denoted as function g.

[0043] (WSNN Model Structure) The WSNN model structure, with the time-warping module introduced above, is shown in (23) and (24) below. v=g(x,c;τ) …(23) y=SNN(v;η) …(24) where τ represents all the parameters internally held by the time-warping module. Also, v corresponds to the output of the time-warping module for the input (x,c). If the data shown in (25) below, which represents the input-output relationship to be estimated, is available, then the input x i The output value y is the correct information for the output in this case. i The parameters θ=(τ,η) of the WSNN can be estimated by minimizing the objective function shown in (27) below, which is defined using the output estimates shown in (26) below, output by the WSNN.

[0044]

[0045] However, the output estimate is expressed in (28) below.

[0046]

[0047] Furthermore, the function l can be any function used as a loss function in neural network training, such as the squared error, absolute error, or cross-entropy error. In addition, to minimize the objective function shown in (27) above, any optimization method used for neural network parameter estimation, such as the pseudo-Newton method, stochastic gradient method, or Adam, can be used.

[0048] (Application Example) As one application example of this embodiment, the results of applying it to artificially generated biosignal data are shown. Here, based on reference 2, "Reza Sameni, Mohammad B Shamsollahi, Christian Jutten, and Gari D Clifford. A nonlinear bayesian filtering framework for ecg denoising. IEEE transactions on Biomedical Engineering, Vol.54, No.12, pp. 2172-2185, 2007," an electrocardiogram waveform is generated using the differential equations shown in (29) and (30) below.

[0049]

[0050] However, the following (31) holds true, and following Reference 2, the following (32) was set to the value in Table 1 in Reference 2. Δθ i =(θ-θ i )mod(2π) …(31) z0=0,α i ,b i ,θ i…(32) Using this equation, 50 electrocardiogram waveforms are generated for each of the following (33) relating to the parameter ξ of the differential equation, and the output value y is used as the value obtained by adding Gaussian noise to the waveform at each time x, which is divided into 200 equal intervals from 0 to 40. ξ=0.6,0.8,1.0,1.2,1.4 …(33) Furthermore, the parameter ξ of the differential equation is treated as contextual information, and training data for WSNN is created as follows (34).

[0051]

[0052] This data is unsuitable for use with existing SNNs because the period differs for each value of ξ.

[0053] Figure 2 shows an example of estimation results using an SNN. Figure 3 shows an example of estimation results using a WSNN. In Figure 2, the function g shown in "Example 1" above is used as the time warping module. Ex1 The following shows the visualization of the estimation results using an SNN without a time warping module when the total number of hidden layers is set to L=2. In Figure 3, the function g shown in "Example 1" above is used as the time warping module. Ex1 The following shows the visualization of the estimation results by WSNN when the total number of hidden layers is set to L=2.

[0054] As shown in Figure 2, SNN fails to capture waveforms well because it contains data with different periods, whereas WSNN succeeds in capturing waveforms, as shown in Figure 3.

[0055] Figures 4, 5, 6, and 7 show examples of estimation results by WSNN. Figure 4 shows the visualization of the WSNN output for context information not used in training (ξ=0.3), Figure 5 shows the visualization of the WSNN output for context information not used in training (ξ=0.5), Figure 6 shows the visualization of the WSNN output for context information not used in training (ξ=0.7), and Figure 7 shows the visualization of the WSNN output for context information not used in training (ξ=0.9).

[0056] Contextual information not used in training is contextual information not included in the training data. WSNN successfully captures waveforms regardless of the ξ value, confirming that it is a method capable of handling changes in period.

[0057] In this embodiment, it becomes possible to estimate and predict with greater accuracy data that contains periodic components and whose period changes depending on the context. For example, it becomes possible to analyze in more detail than with existing methods electrocardiogram waveforms that have a period that changes depending on the context, such as a short period at rest and a long period during exercise, measurement data related to human walking or arm swing that moves in time with the rhythm of music, or energy consumption data of air conditioners that operate in accordance with changes in temperature or room temperature.

[0058] Next, the configuration of the data processing device according to this embodiment and the operation of each part will be described. Figure 8 is a diagram showing an example of application of the data processing device according to one embodiment of the present invention. Figure 9 is a flowchart showing an example of the general operation of the data processing device according to one embodiment of the present invention. As shown in Figure 8, the data processing device 100 includes an input data processing unit 10, a parameter estimation unit 20, a parameter processing unit 30, a recording unit 40, and an input / output unit 50. The recording unit 40 has an input data recording unit 41 and a parameter recording unit 42. The input / output unit 50 is capable of inputting and outputting data to and from an external device 200.

[0059] The input data processing unit 10 of the data processing device 100 receives the data shown in (35) below and setting parameters related to the structure and optimization of the neural network, and stores them in the input data recording unit 41 of the recording unit 40 (S10).

[0060]

[0061] Optimization setting parameters include, for example, the number of hidden layers L or the learning rate.

[0062] Next, the parameter estimation unit 20 takes the data shown in (35) above and the setting parameters related to the structure and optimization of the neural network stored in the input data recording unit 41 as input, and estimates the WSNN parameter θ by minimizing the objective function shown in (27) above (S20). After that, the parameter estimation unit 20 stores the estimated parameter θ in the parameter recording unit 42 of the recording unit 40.

[0063] The parameter processing unit 30 outputs the parameter θ stored in the parameter recording unit 42 to the external device 200 via the input / output unit 50 (S30).

[0064] In this embodiment, the data shown in (35) above can include any data such as (1) biosignal data and data measured by wearable devices or cameras for human behavior, such as swinging the legs and arms or walking; (2) data acquired by IoT (Internet of Things) devices, such as data related to the environment, such as temperature or room temperature, humidity, and carbon dioxide levels; and (3) data related to energy consumption.

[0065] Figure 10 is a block diagram showing an example of the hardware configuration of a data processing device according to one embodiment of the present invention. In the example shown in Figure 10, the data processing device 100 according to the above embodiment is composed of, for example, a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to this hardware processor 111A via a bus 115. The input / output interface 113 corresponds to the input / output unit 50 shown in Figure 8.

[0066] The communication interface 114 includes, for example, one or more wireless communication interface units, enabling the transmission and reception of information with the communication network. As the wireless interface, for example, an interface employing a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0067] The input / output interface 113 is connected to an input device 500 and an output device 600, which are attached to the data processing device 100 and used by users. The input device 500 and the output device 600 correspond to the external device 200 shown in Figure 8.

[0068] The input / output interface 113 can capture operation data entered by a user or the like through an input device 500 such as a keyboard, touch panel, or touchpad, and can also output output data to an output device 600, including a display device using liquid crystal or organic EL (electroluminescence), for display. The input device 500 and output device 600 may be devices built into the data processing device 100, or they may be input and output devices of other information terminals that can communicate with the data processing device 100 via a network.

[0069] The program memory 111B is a non-temporary tangible storage medium in which a non-volatile memory that can be written to and read at any time, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), is used in combination with another non-volatile memory such as ROM (Read Only Memory), and can store programs necessary for executing various control processes, etc., according to one embodiment.

[0070] The data memory 112 is a tangible storage medium that, for example, uses a combination of the above-mentioned non-volatile memory and volatile memory such as RAM (Random Access Memory), and can be used to store various data or information acquired and created during the process of various operations.

[0071] A data processing device 100 according to one embodiment of the present invention may be configured as a data processing device having an input data processing unit 10, a parameter estimation unit 20, a parameter processing unit 30, and an input / output unit 50 as software-based processing functions, as shown in Figure 8.

[0072] The storage devices and recording unit 40 used as work memory by each part of the data processing device 100 may be configured using the data memory 112 shown in Figure 10. However, the storage area configured by these storage devices is not an essential component within the data processing device 100, and may be an area provided in an external storage medium such as a USB (Universal Serial Bus) memory, or in a storage device such as a database server located in the cloud.

[0073] The processing functions in the input data processing unit 10, parameter estimation unit 20, parameter processing unit 30, and input / output unit 50 described above can all be realized by having the hardware processor 111A read and execute a program stored in the program memory 111B. Some or all of these processing functions may be realized in various other forms, including application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).

[0074] Furthermore, the methods described in each embodiment can be stored as programs (software means) that can be executed by a computer on recording media such as magnetic disks (floppy disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, MOs, etc.), and semiconductor memories (ROMs, RAMs, flash memories, etc.), and can also be transmitted and distributed via communication media. The programs stored on the media also include configuration programs that configure the computer to run software means (including not only the execution program but also tables or data structures). The computer implementing this device reads the program recorded on the recording media and, if necessary, constructs the software means using the configuration program, and executes the above-described processes by controlling the operation of this software means. Note that the recording media referred to in this specification are not limited to those for distribution, but also include storage media such as magnetic disks or semiconductor memories provided inside the computer or in devices connected via a network.

[0075] It should be noted that the present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention.

[0076] 100...Data processing unit 200...External device 10...Input data processing unit 20...Parameter estimation unit 30...Parameter processing unit 40...Recording unit 41...Input data recording unit 42...Parameter recording unit 50...Input / output unit

Claims

1. A data processing device comprising: an input unit that accepts data including time as input to a model that outputs data having periodic components using a function that calculates the output value of a monotonically increasing function for data including time, and calculates a value based on the square of the sine of the output value; and an estimation unit that estimates the parameters of the model by minimizing an objective function defined by the ground truth information of the data having periodic components and the estimated value of the data having periodic components by the model.

2. The data output from the model is data in which the period of the component changes in accordance with the change in content, and the model outputs data having the periodic component and in which the period of the component changes, by using a function that obtains the output value of a monotonically increasing function for data that further includes information indicating the change in content, and obtains a value based on the square of the sine of the output value.

3. A data processing method performed by a data processing device, comprising: an input unit of the data processing device receiving input data including time to a model that outputs data having periodic components using a function that obtains the output value of a monotonically increasing function for data including time, and obtains a value based on the square of the sine of the output value; and an estimation unit of the data processing device estimating the parameters of the model by minimizing an objective function defined by the correct information of the data having periodic components and the estimated value of the data having periodic components by the model.

4. A data processing program that causes a processor to function as a component of the data processing apparatus described in claim 1 or 2.

Images