Information processing device, information processing method, and program

Abstract

An information processing device (1) comprises: a monotonic increase neural network (24-1); and a first calculation unit (24-2) that calculates a cumulative intensity function on the basis of the output from the monotonic increase neural network and the product of a parameter and time.

Description

[Technical field]

[0001] The embodiments relate to an information processing device, an information processing method, and a program. [Background technology]

[0002] Methods using point processes are known as a method for predicting the occurrence of various events such as equipment failures, human behavior, crimes, earthquakes, infectious diseases, etc. A point process is a probabilistic model that describes the timing of the occurrence of an event.

[0003] Neural networks (NNs) are known as a technology that can model point processes quickly and accurately. Monotonically increasing neural networks (MNNs) have been proposed as one type of neural network. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Takahiro Omi, et al., “Fully Neural Network based Model for General Temporal Point Processes,” 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada Summary of the Invention [Problem to be solved by the invention]

[0005] However, the monotonically increasing neural network may be inferior in expressiveness to the normal neural network. In addition, the monotonically increasing neural network may lack stability in the learning process due to the disappearance or divergence of the gradient of the activation function. The above-mentioned problems of the monotonically increasing neural network become particularly prominent when predicting events over the long term.

[0006] The present invention has been made in light of the above circumstances, and an object of the present invention is to provide a means for enabling long-term prediction of events. [Means for solving the problem]

[0007] An information processing device of one aspect includes a monotonically increasing neural network, and a first calculation unit that calculates a cumulative intensity function based on an output from the monotonically increasing neural network and a product of a parameter and time. Effect of the Invention

[0008] According to an embodiment, a means may be provided that allows for long-term prediction of events. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an event prediction device according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of a configuration of a learning function of the event prediction device according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a configuration of a series in a learning dataset of the event prediction device according to the first embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of a configuration of a prediction function of the event prediction device according to the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a configuration of prediction data in the event prediction device according to the first embodiment. [Figure 6]FIG. 6 is a flowchart showing an example of a learning operation in the event prediction device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of a prediction operation in the event prediction device according to the first embodiment. [Figure 8] FIG. 8 is a block diagram showing an example of a configuration of a learning function of an event prediction device according to a first modified example. [Figure 9] FIG. 9 is a block diagram showing an example of a configuration of a prediction function of an event prediction device according to a first modified example. [Figure 10] FIG. 10 is a flowchart showing an example of a learning operation in the event prediction device according to the first modified example. [Figure 11] FIG. 11 is a flowchart showing an example of a prediction operation in the event prediction device according to the first modified example. [Figure 12] FIG. 12 is a block diagram showing an example of a configuration of a learning function of an event prediction device according to the second modification. [Figure 13] FIG. 13 is a block diagram showing an example of a configuration of a prediction function of an event prediction device according to the second modification. [Figure 14] FIG. 14 is a flowchart showing an example of an outline of a learning operation in the event prediction device according to the second modification. [Figure 15] FIG. 15 is a flowchart showing an example of a first update process in the event prediction device according to the second modified example. [Figure 16] FIG. 16 is a flowchart showing an example of a second update process in the event prediction device according to the second modified example. [Figure 17] FIG. 17 is a flowchart showing an example of a prediction operation in the event prediction device according to the second modification. [Figure 18] FIG. 18 is a block diagram showing an example of the configuration of a learning function of an event prediction device according to the second embodiment. [Figure 19] FIG. 19 is a block diagram illustrating an example of a configuration of a prediction function of an event prediction device according to the second embodiment. [Figure 20]FIG. 20 is a flowchart showing an example of a learning operation in the event prediction device according to the second embodiment. [Figure 21] FIG. 21 is a flowchart showing an example of a prediction operation in the event prediction device according to the second embodiment. [Figure 22] FIG. 22 is a block diagram showing an example of the configuration of a learning function of an event prediction device according to the third modified example. [Diagram 23] FIG. 23 is a block diagram showing an example of a configuration of a prediction function of an event prediction device according to the third modified example. [Figure 24] FIG. 24 is a flowchart showing an example of an outline of a learning operation in the event prediction device according to the third modified example. [Diagram 25] FIG. 25 is a flowchart showing an example of a first update process in the event prediction device according to the third modified example. [Figure 26] FIG. 26 is a flowchart showing an example of a second update process in the event prediction device according to the third modified example. [Figure 27] FIG. 27 is a flowchart showing an example of a prediction operation in the event prediction device according to the third modified example. [Figure 28] FIG. 28 is a block diagram showing an example of the configuration of a latent expression calculation unit of an event prediction device according to the fourth modified example. [Figure 29] FIG. 29 is a block diagram showing an example of the configuration of an intensity function calculation unit of an event prediction device according to a fifth modified example. [Diagram 30] FIG. 30 is a block diagram showing an example of the configuration of a first intensity function calculation unit of an event prediction device according to a sixth modified example. [Diagram 31] FIG. 31 is a block diagram showing an example of the configuration of a second intensity function calculation unit of an event prediction device according to a sixth modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, several embodiments will be described with reference to the drawings. In the following description, components having the same function and configuration are given the same reference symbol. In addition, when multiple components having the same reference symbol are to be distinguished from each other, they are distinguished from each other by an additional reference symbol (e.g., a hyphen and a number such as "-1") that is added following the common reference symbol.

[0011] 1. First embodiment A description will be given of an information processing device according to the first embodiment. In the following, an event prediction device will be described as an example of the information processing device according to the first embodiment.

[0012] The event prediction device has a learning function and a prediction function. The learning function is a function of meta-learning a point process. The prediction function is a function of predicting the occurrence of an event based on the point process learned by the learning function. An event is an occurrence that occurs discretely in continuous time. Specifically, for example, an event is a user's purchasing behavior on an EC (Electronic Commerce) site.

[0013] 1.1 Configuration The configuration of the event prediction device according to the first embodiment will be described.

[0014] 1.1.1 Hardware Configuration Fig. 1 is a block diagram showing an example of a hardware configuration of an event prediction device according to the first embodiment. As shown in Fig. 1, the event prediction device 1 includes a control circuit 10, a memory 11, a communication module 12, a user interface 13, and a drive 14.

[0015] The control circuit 10 is a circuit that overall controls each component of the event prediction device 1. The control circuit 10 includes a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and the like.

[0016] The memory 11 is a storage device of the event prediction device 1. The memory 11 includes, for example, a hard disk drive (HDD), a solid state drive (SSD), and a memory card. The memory 11 stores information used for the learning operation and the prediction operation in the event prediction device 1. The memory 11 also stores a learning program for causing the control circuit 10 to execute the learning operation, and a prediction program for causing the control circuit 10 to execute the prediction operation.

[0017] The communication module 12 is a circuit used for transmitting and receiving data to and from the outside of the event prediction device 1 via a network.

[0018] The user interface 13 is a circuit for communicating information between a user and the control circuit 10. The user interface 13 includes input devices and output devices. The input devices include, for example, a touch panel and an operation button. The output devices include, for example, an LCD (Liquid Crystal Display), an EL (Electroluminescence) display, and a printer. The user interface 13 outputs, for example, the execution results of various programs received from the control circuit 10 to the user.

[0019] The drive 14 is a device for reading a program stored in a storage medium 15. The drive 14 includes, for example, a CD (Compact Disk) drive, a DVD (Digital Versatile Disk) drive, and the like.

[0020] The storage medium 15 is a medium that accumulates information such as a program by electrical, magnetic, optical, mechanical, or chemical action. The storage medium 15 may store a learning program and a prediction program.

[0021] 1.1.2 Learning function configuration FIG. 2 is a block diagram showing an example of a configuration of a learning function of the event prediction device according to the first embodiment.

[0022] The CPU of the control circuit 10 loads the learning program stored in the memory 11 or the storage medium 15 into the RAM. The CPU of the control circuit 10 then interprets and executes the learning program loaded into the RAM to control the memory 11, the communication module 12, the user interface 13, the drive 14, and the storage medium 15. As a result, as shown in Fig. 2, the event prediction device 1 functions as a computer including a data extraction unit 21, an initialization unit 22, a latent expression calculation unit 23, an intensity function calculation unit 24, an update unit 25, and a determination unit 26. The memory 11 of the event prediction device 1 also stores a learning dataset 20 and learned parameters 27 as information used in the learning operation.

[0023] The training dataset 20 is, for example, a collection of event sequences of multiple users at a certain EC site. Alternatively, the training dataset 20 is a collection of event sequences of a certain user at multiple EC sites. The training dataset 20 has multiple sequences Ev. When the training dataset 20 is a collection of event sequences of multiple users at a certain EC site, each sequence Ev corresponds to, for example, a user. When the training dataset 20 is a collection of event sequences of a certain user at multiple EC sites, each sequence Ev corresponds to, for example, an EC site. Each sequence Ev is calculated based on the time period [0, t e ], the occurrence time t of I events i (1≦i≦I) (I is an integer equal to or greater than 1). The number of events I in each series Ev may be different from one another. That is, the data length of each series Ev may be any length.

[0024] The data extraction unit 21 extracts a sequence Ev from the training dataset 20. The data extraction unit 21 further extracts a support sequence Es and a query sequence Eq from the extracted sequence Ev. The data extraction unit 21 transmits the support sequence Es and the query sequence Eq to the latent expression calculation unit 23 and the update unit 25, respectively.

[0025] Fig. 3 is a diagram showing an example of a configuration of a sequence of a learning dataset of the event prediction device according to the first embodiment. As shown in Fig. 3, the support sequence Es and the query sequence Eq are subsequences of the sequence Ev.

[0026] The support sequence Es is the sequence Ev in the period [0,t s ] is a subsequence corresponding to (Es={t i |0≦t i ≦t s}). Time t s is from time 0 to time t e It is arbitrarily determined within the range of less than.

[0027] The query sequence Eq is the time series of the sequence Ev over a period [t s ,t q ] is a subsequence corresponding to (Eq={t i |t s <t i ≦t q}). Time t q is the time t s Larger time t e It is arbitrarily determined within the following range.

[0028] Referring again to FIG. 2, the configuration of the learning function of the event prediction device 1 will be described.

[0029] The initialization unit 22 initializes a plurality of parameters p1, p2, and β based on the rule X. The initialization unit 22 transmits the initialized plurality of parameters p1 to the latent expression calculation unit 23. The initialization unit 22 transmits the initialized plurality of parameters p2 and β to the intensity function calculation unit 24. The plurality of parameters p1, p2, and β will be described later.

[0030] Rule X involves applying random numbers generated according to a distribution whose mean is equal to or less than 0 to parameters. For example, examples of applying rule X to a neural network having multiple layers include Xavier initialization and He initialization. Xavier initialization initializes parameters according to a normal distribution with mean 0 and standard deviation 1 / √n when the number of nodes in the previous layer is n. He initialization initializes parameters according to a normal distribution with mean 0 and standard deviation √(2 / n) when the number of nodes in the previous layer is n.

[0031] The latent expression calculation unit 23 calculates a latent expression z based on the support sequence Es. The latent expression z is data representing the characteristics of the event occurrence timing in the sequence Ev. The latent expression calculation unit 23 transmits the calculated latent expression z to the intensity function calculation unit 24.

[0032] Specifically, the latent expression calculation unit 23 includes a neural network 23-1. The neural network 23-1 is a mathematical model that is modeled to receive a sequence as input and output a latent expression. The neural network 23-1 is configured to be able to receive variable-length data. A plurality of parameters p1 are applied to the neural network 23-1 as weights and bias terms. The neural network 23-1 to which the plurality of parameters p1 are applied receives a support sequence Es as input and outputs a latent expression z. The neural network 23-1 transmits the output latent expression z to the intensity function calculation unit 24.

[0033] The intensity function calculation unit 24 calculates an intensity function λ(t) based on the latent expression z and time t. The intensity function λ(t) is a function of time indicating the likelihood of an event occurring in a future time period (e.g., occurrence probability). The intensity function calculation unit 24 transmits the calculated intensity function λ(t) to the update unit 25.

[0034] Specifically, the intensity function calculation unit 24 includes a monotonically increasing neural network 24-1, a cumulative intensity function calculation unit 24-2, and an automatic differentiation unit 24-3.

[0035] The monotonically increasing neural network 24-1 is a mathematical model that is modeled to calculate a monotonically increasing function defined by the latent expression and time as an output. A plurality of weights and bias terms based on a plurality of parameters p2 are applied to the monotonically increasing neural network 24-1. If the weights among the plurality of parameters p2 include negative values, the negative values ​​are converted to non-negative values ​​by an operation such as taking the absolute value. If the weights among the plurality of parameters p2 are non-negative values, the plurality of parameters p2 may be applied as they are as weights and bias terms to the monotonically increasing neural network 24-1. That is, each weight applied to the monotonically increasing neural network 24-1 is a non-negative value. The monotonically increasing neural network 24-1 to which the plurality of parameters p2 are applied calculates an output f(z,t) as a scalar value according to a monotonically increasing function defined by the latent expression z and time t. The monotonically increasing neural network 24-1 transmits the output f(z,t) to the cumulative intensity function calculation unit 24-2.

[0036] The cumulative intensity function calculation unit 24-2 calculates the cumulative intensity function Λ(t) based on the parameter β and the output f(z, t) in accordance with the following equation (1).

[0037]

number

[0038] As shown in equation (1), the cumulative intensity function Λ(t) is obtained by adding a term βt that increases in proportion to time t to the outputs f(z,t) and f(z,0) from the monotonically increasing neural network 24-1. The cumulative intensity function calculation unit 24-2 transmits the calculated cumulative intensity function Λ(t) to the automatic differentiation unit 24-3.

[0039] The automatic differentiation unit 24-3 automatically differentiates the cumulative intensity function Λ(t) to calculate the intensity function λ(t). The automatic differentiation unit 24-3 transmits the calculated intensity function λ(t) to the update unit 25.

[0040] The update unit 25 updates the multiple parameters p1, p2, and β based on the intensity function λ(t) and the query sequence Eq. The updated multiple parameters p1, p2, and β are applied to the neural network 23-1, the monotonically increasing neural network 24-1, and the cumulative intensity function calculation unit 24-2, respectively. The update unit 25 also transmits the updated multiple parameters p1, p2, and β to the determination unit 26.

[0041] Specifically, the update unit 25 includes an evaluation function calculation unit 25-1 and an optimization unit 25-2.

[0042] The evaluation function calculation unit 25-1 calculates an evaluation function L(Eq) based on the intensity function λ(t) and the query sequence Eq. The evaluation function L(Eq) is, for example, a negative log-likelihood. The evaluation function calculation unit 25-1 transmits the calculated evaluation function L(Eq) to the optimization unit 25-2.

[0043] The optimization unit 25-2 optimizes a plurality of parameters p1, p2, and β based on the evaluation function L(Eq). For example, the backpropagation method is used for the optimization. The optimization unit 25-2 updates a plurality of parameters p1, p2, and β applied to the neural network 23-1, the monotonically increasing neural network 24-1, and the cumulative intensity function calculation unit 24-2 with the optimized plurality of parameters p1, p2, and β.

[0044] The determination unit 26 determines whether or not a condition is satisfied based on the updated parameters p1, p2, and β. The condition may be, for example, that the number of times the parameters p1, p2, and β are transmitted to the determination unit 26 (i.e., the number of parameter update loops) is equal to or greater than a threshold. The condition may be, for example, that the amount of change in the values ​​of the parameters p1, p2, and β before and after update is equal to or less than a threshold. If the condition is not satisfied, the determination unit 26 causes the data extraction unit 21, the latent expression calculation unit 23, the intensity function calculation unit 24, and the update unit 25 to repeatedly execute a parameter update loop. If the condition is satisfied, the determination unit 26 ends the parameter update loop and stores the last updated parameters p1, p2, and β in the memory 11 as learned parameters 27. In the following description, the multiple parameters in the learned parameters 27 are referred to as p1 to be distinguished from parameters before learning. * , p2 * , and β * It is written as follows.

[0045] With the above-described configuration, the event prediction device 1 has a function of generating learned parameters 27 based on the learning dataset 20.

[0046] 1.1.3 Prediction Function Configuration FIG. 4 is a block diagram illustrating an example of a configuration of a prediction function of the event prediction device according to the first embodiment.

[0047] The CPU of the control circuit 10 loads the prediction program stored in the memory 11 or the storage medium 15 into the RAM. The CPU of the control circuit 10 then interprets and executes the prediction program loaded into the RAM to control the memory 11, the communication module 12, the user interface 13, the drive 14, and the storage medium 15. As a result, as shown in FIG. 4, the event prediction device 1 further functions as a computer including a latent expression calculation unit 23, an intensity function calculation unit 24, and a predicted sequence generation unit 29. The memory 11 of the event prediction device 1 further stores prediction data 28 as information used in the prediction operation. In FIG. 4, a plurality of parameters p1 are input from the learned parameters 27 to the neural network 23-1, the monotonically increasing neural network 24-1, and the cumulative intensity function calculation unit 24-2, respectively. * , p2 * , and β * is shown when it is applied.

[0048] When the training data set 20 is a collection of event sequences of multiple users at a certain EC site, the prediction data 28 corresponds to, for example, a new user's event sequence for the next week. When the training data set 20 is a collection of event sequences of a user at multiple EC sites, the prediction data 28 corresponds to, for example, a user's event sequence for the next week at another EC site.

[0049] 5 is a diagram showing an example of the configuration of prediction data of the event prediction device according to the first embodiment. As shown in FIG. 5, the prediction data 28 includes a prediction sequence Es * The prediction sequence Es * is information including the occurrence time of an event that occurred before the period to be predicted. * is the period Ts * =[0,ts * ] occurred during * Time t of an event i (1≦i≦I * ) (I * is an integer greater than or equal to 1).

[0050] That is, period Ts * The period Tq following * =(ts * ,tq * ] is the period in which the prediction operation predicts the occurrence of an event. In the following, the period Tq * The information including the predicted event occurrence time is the prediction sequence Eq. * Let us assume that.

[0051] Referring again to FIG. 4, the configuration of the prediction function of the event prediction device 1 will be described.

[0052] The latent expression calculation unit 23 inputs the prediction sequence Es in the prediction data 28 to the neural network 23-1. * Enter multiple parameters p1 * The neural network 23-1 to which the above is applied is the prediction sequence Es * is used as input, and the latent representation z * The neural network 23-1 outputs the output latent representation z * is transmitted to the monotonically increasing neural network 24-1 in the intensity function calculation unit 24.

[0053] Multiple parameters p2 * The monotonically increasing neural network 24-1 to which * and the output f * (z, t). The monotonically increasing neural network 24-1 calculates the output f * (z, t) is transmitted to the cumulative intensity function calculation unit 24-2.

[0054] The cumulative intensity function calculation unit 24-2 calculates the parameter β according to the above-mentioned formula (1). * and output f * Based on (z,t), the cumulative intensity function Λ * (t). The cumulative intensity function calculation unit 24-2 calculates the calculated cumulative intensity function Λ * (t) is sent to the automatic differentiation unit 24-3.

[0055] The automatic differentiation unit 24-3 calculates the cumulative intensity function Λ * By automatically differentiating (t), the intensity function λ * (t). The automatic differentiation unit 24-3 calculates the calculated intensity function λ * (t) is sent to the predicted sequence generator 29.

[0056] The predicted sequence generator 29 calculates the intensity function λ * Based on (t), the forecast sequence Eq * The predicted sequence generator 29 generates the predicted sequence Eq * The predicted sequence generator 29 outputs the intensity function λ * (t) may be output to the user. Note that the predicted sequence Eq * To generate the above, for example, a simulation using the Lewis method or the like is performed. Information regarding the Lewis method is as follows.

[0057] Yosihiko Ogata, “On Lewis' Simulation Method for Point Processes,” IEEE Transactions on Information Theory, Vol.27, Issue.1, January 1981 With the above configuration, the event prediction device 1 calculates the prediction sequence Es based on the learned parameters 27. * The forecast series Eq * It has the ability to predict.

[0058] 1.2. Operation Next, the operation of the event prediction device according to the first embodiment will be described.

[0059] 1.2.1 Learning behavior Fig. 6 is a flowchart showing an example of the learning operation in the event prediction device according to the embodiment 1. In the example of Fig. 6, it is assumed that the learning dataset 20 is stored in the memory 11 in advance.

[0060] 6, in response to a user's instruction to start a learning operation (start), the initialization unit 22 initializes a number of parameters p1, p2, and β based on a rule X (S10). For example, the initialization unit 22 initializes the multiple parameters p1 and p2 based on Xavier's initialization or He's initialization. The initialization unit 22 also applies a random number generated according to a distribution whose average is 0 or less to the parameter β. The multiple parameters p1, p2, and β initialized by the process of S10 are applied to the neural network 23-1, the monotonically increasing neural network 24-1, and the cumulative intensity function calculation unit 24-2, respectively.

[0061] The data extraction unit 21 extracts a sequence Ev from the learning dataset 20. Next, the data extraction unit 21 further extracts a support sequence Es and a query sequence Eq from the extracted sequence Ev (S11).

[0062] The neural network 23-1 to which the multiple parameters p1 initialized in the process of S10 have been applied calculates a latent representation z using the support sequence Es extracted in the process of S11 as an input (S12).

[0063] The monotonically increasing neural network 24-1, to which multiple parameters p2 initialized in the process of S10 have been applied, calculates outputs f(z, t) and f(z, 0) according to the latent representation z calculated in the process of S12 and a monotonically increasing function defined by time t (S13).

[0064] The cumulative intensity function calculation unit 24-2, to which the parameter β initialized in the process of S10 has been applied, calculates the cumulative intensity function Λ(t) based on the outputs f(z, t) and f(z, 0) calculated in the process of S13 (S14).

[0065] The automatic differentiation unit 24-3 calculates the intensity function λ(t) based on the cumulative intensity function Λ(t) calculated in the process of S14 (S15).

[0066] The update unit 25 updates a plurality of parameters p1, p2, and β based on the intensity function λ(t) calculated in S15 and the query sequence Eq extracted in the process of S11 (S16). Specifically, the evaluation function calculation unit 25-1 calculates an evaluation function L(Eq) based on the intensity function λ(t) and the query sequence Eq. The optimization unit 25-2 calculates a plurality of optimized parameters p1, p2, and β based on the evaluation function L(Eq) using the backpropagation method. The optimization unit 25-2 applies the optimized plurality of parameters p1, p2, and β to the neural network 23-1, the monotonically increasing neural network 24-1, and the cumulative intensity function calculation unit 24-2, respectively.

[0067] The determination unit 26 determines whether or not the condition is satisfied based on the multiple parameters p1, p2, and β (S17).

[0068] If the condition is not satisfied (S17; no), the data extractor 21 extracts a new support sequence Es and a query sequence Eq from the learning dataset 20 (S11). Then, the processes of S12 to S17 are executed based on the extracted new support sequence Es and query sequence Eq and the multiple parameters p1, p2, and β updated in the process of S16. Thus, the update process of the multiple parameters p1, p2, and β is repeated until it is determined in the process of S17 that the condition is satisfied.

[0069] If the condition is satisfied (S17; yes), the decision unit 26 sets the parameters p1, p2, and β last updated in the process of S16 as p1 * , p2 * , and β * The learned parameters are stored in the learned parameters 27 (S18).

[0070] When the process of S18 ends, the learning operation in the event prediction device 1 ends (ends).

[0071] 1.2.2 Predictive Behavior 7 is a flowchart showing an example of a prediction operation in the event prediction device according to the first embodiment. In the example of FIG. 7, a plurality of parameters p1 in the learned parameters 27 are predicted by a learning operation executed in advance. * , p2 * , and β * are applied to the neural network 23-1, the monotonically increasing neural network 24-1, and the cumulative intensity function calculation unit 24-2, respectively. In the example of FIG. 7, it is assumed that prediction data 28 is stored in the memory 11.

[0072] As shown in FIG. 7, in response to a command from a user to start a prediction operation (start), multiple parameters p1 * The neural network 23-1 to which the above is applied is the prediction sequence Es * is used as input, and the latent representation z * is calculated (S20).

[0073] Multiple parameters p2 * The monotonically increasing neural network 24-1 to which the above is applied uses the latent representation z * , and the output f * (z,t) and f * (z, 0) is calculated (S21).

[0074] Parameter β * The cumulative intensity function calculation unit 24-2 to which the above has been applied outputs f * (z,t) and f * Based on (z,0), the cumulative intensity function Λ * (t) is calculated (S22).

[0075] The automatic differentiation unit 24-3 calculates the cumulative intensity function Λ calculated in the process of S22. * Based on (t), the intensity function λ * (t) is calculated (S23).

[0076] The predicted sequence generator 29 calculates the intensity function λ *Based on (t), the forecast sequence Eq * Then, the predicted sequence generating unit 29 generates the predicted sequence Eq * is output to the user.

[0077] When the process of S24 ends, the prediction operation in the event prediction device 1 ends (ends).

[0078] 1.3 Effects of the First Embodiment According to the first embodiment, the monotonically increasing neural network 24-1 is configured to calculate outputs f(z,t) and f(z,0) according to a monotonically increasing function defined by the latent expression z of the support sequence Es and time t. The cumulative intensity function calculation unit 24-2 calculates the cumulative intensity function Λ(t) based on the outputs f(z,t) and f(z,0) and the product βt of the parameter β and time t. This eliminates the need for the monotonically increasing neural network 24-1 to express increments over time, and only requires the monotonically increasing neural network 24-1 to express periodic changes. This can ease the requirements for expressiveness required for the output of the monotonically increasing neural network 24-1. The cumulative intensity function calculation unit 24-2 can calculate the cumulative intensity function Λ(t) while compensating for the limit of the expressiveness of the monotonically increasing neural network 24-1 by the parameter β.

[0079] In addition, the automatic differentiation unit 24-3 calculates the intensity function λ(t) for the point process based on the cumulative intensity function Λ(t). This allows the monotonically increasing neural network 24-1 to be used for modeling the point process. Therefore, the monotonically increasing neural network 24-1 can be used to perform long-term prediction of events.

[0080] Furthermore, the update unit 25 updates the parameter β based on the intensity function λ(t) and the query sequence Eq. This makes it possible to adjust the parameter β to a value suitable for modeling the point process using the learning dataset 20.

[0081] 1.4 First variant In the above-described first embodiment, the parameter β is directly initialized and updated, but the present invention is not limited to this. For example, the parameter β may be indirectly calculated via a plurality of parameters that are directly initialized and updated. The following mainly describes configurations and operations that are different from the first embodiment. Furthermore, descriptions of configurations and operations equivalent to those of the first embodiment will be omitted as appropriate.

[0082] 1.4.1 Learning function configuration Fig. 8 is a block diagram showing an example of the configuration of the learning function of the event prediction device according to Modification 1. As shown in Fig. 8, the intensity function calculation unit 24 further includes a neural network 24-4.

[0083] The initialization unit 22 initializes a plurality of parameters p1, p2, and p3 based on the rule X. The initialization unit 22 transmits the initialized plurality of parameters p1, p2, and p3 to the neural network 23-1, the monotonically increasing neural network 24-1, and the neural network 24-4, respectively. The plurality of parameters p3 will be described later.

[0084] The neural network 24-4 is a mathematical model that is modeled to receive a sequence as input and output one parameter. A plurality of parameters p3 are applied to the neural network 24-4 as weights and bias terms. The neural network 24-4 to which the plurality of parameters p3 are applied receives all events or the number of events in the support sequence Es as input and outputs a parameter β. The neural network 24-4 transmits the output parameter β to the cumulative intensity function calculation unit 24-2.

[0085] The optimization unit 25-2 optimizes a plurality of parameters p1, p2, and p3 based on the evaluation function L(Eq). For example, the backpropagation method is used for the optimization. The optimization unit 25-2 updates a plurality of parameters p1, p2, and p3 applied to the neural network 23-1, the monotonically increasing neural network 24-1, and the neural network 24-4 with the optimized plurality of parameters p1, p2, and p3.

[0086] The determination unit 26 determines whether or not a condition is satisfied based on the updated multiple parameters p1, p2, and p3. The condition may be, for example, that the number of times the multiple parameters p1, p2, and p3 are transmitted to the determination unit 26 (i.e., the number of parameter update loops) is equal to or greater than a threshold. The condition may be, for example, that the amount of change in the values ​​of the multiple parameters p1, p2, and p3 before and after the update is equal to or less than a threshold. If the condition is not satisfied, the determination unit 26 causes the data extraction unit 21, the latent expression calculation unit 23, the intensity function calculation unit 24, and the update unit 25 to repeatedly execute a parameter update loop. If the condition is satisfied, the determination unit 26 ends the parameter update loop and stores the last updated multiple parameters p1, p2, and p3 in the memory 11 as learned parameters 27. In the following description, the multiple parameters in the learned parameters 27 are referred to as p1, p2, and p3 in order to distinguish them from parameters before learning. * , p2 * , and p3 * It is written as follows.

[0087] With the above-mentioned configuration, the event prediction device 1 has a function of generating a parameter β based on a plurality of parameters p3.

[0088] 1.4.2 Prediction Function Configuration Fig. 9 is a block diagram showing an example of the configuration of the prediction function of the event prediction device according to the first modified example. As shown in Fig. 9, the event prediction device 1 further functions as a computer including a latent expression calculation unit 23, an intensity function calculation unit 24, and a predicted sequence generation unit 29. The memory 11 of the event prediction device 1 further stores prediction data 28 as information used in the prediction operation. In Fig. 9, a plurality of parameters p1 are input from the learned parameters 27 to the neural network 23-1, the monotonically increasing neural network 24-1, and the neural network 24-4, respectively. * , p2 * , and p3 * is shown when it is applied.

[0089] Multiple parameters p3 * The neural network 24-4 to which the above is applied generates the prediction sequence Es * Based on this, the parameter β * The neural network 24-4 calculates the calculated parameter β * to the cumulative intensity function calculation unit 24-2.

[0090] With the above configuration, the event prediction device 1 calculates the prediction sequence Es based on the learned parameters 27. * The forecast series Eq * It has the ability to predict.

[0091] 1.4.3 Learning behavior Fig. 10 is a flowchart showing an example of the learning operation in the event prediction device according to Modification 1. In the example of Fig. 10, it is assumed that the learning dataset 20 is stored in the memory 11 in advance.

[0092] 10, in response to an instruction from a user to start a learning operation (START), the initialization unit 22 initializes a plurality of parameters p1, p2, and p3 based on the rule X (S30). The plurality of parameters p1, p2, and p3 initialized by the process of S30 are applied to the neural network 23-1, the monotonically increasing neural network 24-1, and the neural network 24-4, respectively.

[0093] The data extraction unit 21 extracts a sequence Ev from the learning dataset 20. Next, the data extraction unit 21 further extracts a support sequence Es and a query sequence Eq from the extracted sequence Ev (S31).

[0094] The neural network 23-1 to which the multiple parameters p1 initialized in the process of S30 have been applied calculates a latent representation z using the support sequence Es extracted in the process of S31 as an input (S32).

[0095] The monotonically increasing neural network 24-1, to which the multiple parameters p2 initialized in the process of S30 have been applied, calculates outputs f(z, t) and f(z, 0) according to the latent representation z calculated in the process of S32 and a monotonically increasing function defined by time t (S33).

[0096] The neural network 24-4 to which the multiple parameters p3 initialized in the process of S30 have been applied calculates a parameter β using the support sequence Es extracted in the process of S31 as an input (S34).

[0097] The cumulative intensity function calculation unit 24-2 calculates the cumulative intensity function Λ(t) based on the outputs f(z,t) and f(z,0) calculated in the process of S33 and the parameter β calculated in the process of S34 (S35).

[0098] The automatic differentiation unit 24-3 calculates the intensity function λ(t) based on the cumulative intensity function Λ(t) calculated in the process of S35 (S36).

[0099] The update unit 25 updates the multiple parameters p1, p2, and p3 based on the intensity function λ(t) calculated in S36 and the query sequence Eq extracted in the process of S31 (S37). Specifically, the evaluation function calculation unit 25-1 calculates the evaluation function L(Eq) based on the intensity function λ(t) and the query sequence Eq. The optimization unit 25-2 calculates the multiple optimized parameters p1, p2, and p3 based on the evaluation function L(Eq) using the backpropagation method. The optimization unit 25-2 applies the multiple optimized parameters p1, p2, and p3 to the neural network 23-1, the monotonically increasing neural network 24-1, and the neural network 24-4, respectively.

[0100] The determination unit 26 determines whether or not the condition is satisfied based on the multiple parameters p1, p2, and p3 (S38).

[0101] If the condition is not satisfied (S38; no), the data extractor 21 extracts a new support sequence Es and a query sequence Eq from the learning dataset 20 (S31). Then, the processes of S32 to S38 are executed based on the extracted new support sequence Es and query sequence Eq and the multiple parameters p1, p2, and p3 updated in the process of S37. Thus, the update process of the multiple parameters p1, p2, and p3 is repeated until it is determined in the process of S38 that the condition is satisfied.

[0102] If the condition is satisfied (S38; yes), the decision unit 26 updates the parameters p1, p2, and p3 last updated in the process of S37 as follows: * , p2 * , and p3 * The learned parameters are stored in the learned parameters 27 (S39).

[0103] When the process of S39 ends, the learning operation in the event prediction device 1 ends (ends).

[0104] 1.4.4 Predictive Behavior 11 is a flowchart showing an example of a prediction operation in the event prediction device according to the first modification. In the example of FIG. 11, a plurality of parameters p1 in the learned parameters 27 are predicted by a learning operation executed in advance. * , p2 * , and p3 * are applied to the neural network 23-1, the monotonically increasing neural network 24-1, and the neural network 24-4, respectively. In the example of FIG. 11, it is also assumed that the prediction data 28 is stored in the memory 11.

[0105] As shown in FIG. 11, in response to a user's instruction to start a prediction operation (start), a plurality of parameters p1 * The neural network 23-1 to which the above is applied is the prediction sequence Es * is used as input, and the latent representation z * is calculated (S40).

[0106] Multiple parameters p2 * The monotonically increasing neural network 24-1 to which the above is applied is the latent representation z * , and the output f * (z,t) and f * (z, 0) is calculated (S41).

[0107] Multiple parameters p3 * The neural network 24-4 to which the above is applied calculates the prediction sequence Es * With input, parameter β * is calculated (S42).

[0108] Parameter β calculated in the process of S42 * The cumulative intensity function calculation unit 24-2 to which the above has been applied outputs f * (z,t) and f * Based on (z,0), the cumulative intensity function Λ * (t) is calculated (S43).

[0109] The automatic differentiation unit 24-3 calculates the cumulative intensity function Λ calculated in the process of S43. * Based on (t), the intensity function λ * (t) is calculated (S44).

[0110] The predicted sequence generator 29 calculates the intensity function λ * Based on (t), the forecast sequence Eq * Then, the predicted sequence generating unit 29 generates the predicted sequence Eq * is output to the user.

[0111] When the process of S45 ends, the prediction operation in the event prediction device 1 ends (ends).

[0112] 1.4.5 Effects of the First Modification According to the first modification, the neural network 24-4 is configured to input all events included in the support series Es or the number I of events included in the support series Es, and output the parameter β. This allows the value of the parameter β to be changed according to the support series Es. This allows the expressive power of the parameter β to be improved. This allows the long-term prediction accuracy of events to be improved.

[0113] 1.5 Second variant In the above-described first embodiment, a case has been described in which a neural network that inputs a support sequence Es and outputs a latent representation z is used to model the intensity function λ(t), but the present invention is not limited to this. For example, the modeling of the intensity function λ(t) may be realized by combining it with a meta-learning method such as MAML (Model-Agnostic Meta-Learning). The following mainly describes configurations and operations that are different from the first embodiment. Furthermore, descriptions of configurations and operations equivalent to those of the first embodiment will be omitted as appropriate.

[0114] 1.5.1 Learning Function Configuration FIG. 12 is a block diagram showing an example of a configuration of a learning function of an event prediction device according to the second modification.

[0115] 12, the event prediction device 1 functions as a computer including a data extraction unit 31, an initialization unit 32, a first intensity function calculation unit 33A, a second intensity function calculation unit 33B, a first update unit 34A, a second update unit 34B, a first determination unit 35A, and a second determination unit 35B. In addition, the memory 11 of the event prediction device 1 stores a learning dataset 30 and learned parameters 36 as information used in the learning operation.

[0116] The training dataset 30 and the data extraction unit 31 are equivalent to the training dataset 20 and the data extraction unit 21 in the first embodiment. That is, the data extraction unit 31 extracts a support sequence Es and a query sequence Eq from the training dataset 30.

[0117] The initialization unit 32 initializes a plurality of parameters p2 and β based on the rule X. The initialization unit 22 transmits the initialized plurality of parameters p2 and β to the first intensity function calculation unit 33A. In the following, the set of the plurality of parameters p2 and β is also referred to as a parameter set θ{p2, β}. In addition, the plurality of parameters p2 and β in the parameter set θ{p2, β} are also referred to as a plurality of parameters θ{p2} and θ{β}, respectively.

[0118] The first intensity function calculation unit 33A calculates the intensity function λ1(t) based on the time t. The first intensity function calculation unit 33A transmits the calculated intensity function λ1(t) to the first update unit 34A.

[0119] Specifically, the first intensity function calculation unit 33A includes a monotonically increasing neural network 33A-1, a cumulative intensity function calculation unit 33A-2, and an automatic differentiation unit 33A-3.

[0120] The monotonically increasing neural network 33A-1 is a mathematical model that is modeled to calculate a monotonically increasing function defined by time as an output. A plurality of weights and bias terms based on a plurality of parameters θ{p2} are applied to the monotonically increasing neural network 33A-1. Each weight applied to the monotonically increasing neural network 33A-1 is a non-negative value. The monotonically increasing neural network 33A-1 to which the plurality of parameters θ{p2} are applied calculates an output f1(t) according to a monotonically increasing function defined by time t. The monotonically increasing neural network 33A-1 transmits the calculated output f1(t) to the cumulative intensity function calculation unit 33A-2.

[0121] The cumulative intensity function calculation unit 33A-2 calculates the cumulative intensity function Λ1(t) based on the parameter θ{β} and the output f1(t) in accordance with the following equation (2).

[0122]

number

[0123] As shown in equation (2), the cumulative intensity function Λ1(t) is obtained by adding a term βt that increases in proportion to time t to the outputs f1(t) and f1(0) from the monotonically increasing neural network 33A-1. The cumulative intensity function calculation unit 33A-2 transmits the calculated cumulative intensity function Λ1(t) to the automatic differentiation unit 33A-3.

[0124] The automatic differentiation unit 33A-3 calculates the intensity function λ1(t) by automatically differentiating the cumulative intensity function Λ1(t). The automatic differentiation unit 33A-3 transmits the calculated intensity function λ1(t) to the first update unit 34A.

[0125] The first update unit 34A updates the parameter set θ{p2, β} based on the intensity function λ1(t) and the support sequence Es. The updated parameters θ{p2} and θ{β} are applied to the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2, respectively. The first update unit 34A also transmits the updated parameter set θ{p2, β} to the first judgment unit 35A.

[0126] Specifically, the first updating unit 34A includes an evaluation function calculation unit 34A-1 and an optimization unit 34A-2.

[0127] The evaluation function calculation unit 34A-1 calculates an evaluation function L1(Es) based on the intensity function λ1(t) and the support sequence Es. The evaluation function L1(Es) is, for example, a negative log-likelihood. The evaluation function calculation unit 34A-1 transmits the calculated evaluation function L1(Es) to the optimization unit 34A-2.

[0128] The optimization unit 34A-2 optimizes the parameter set θ{p2, β} based on the evaluation function L1(Es). For example, the backpropagation method is used for the optimization. The optimization unit 34A-2 updates the parameter set θ{p2, β} applied to the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2 with the optimized parameter set θ{p2, β}.

[0129] The first determination unit 35A determines whether or not the first condition is satisfied based on the updated parameter set θ{p2, β}. The first condition may be, for example, that the number of times the parameter set θ{p2, β} is transmitted to the first determination unit 35A (i.e., the number of update loops of the parameter set in the first intensity function calculation unit 33A and the first update unit 34A) is equal to or greater than a threshold. The first condition may be, for example, that the amount of change in the value of the parameter set θ{p2, β} before and after the update is equal to or less than a threshold. Hereinafter, the update loop of the parameter set in the first intensity function calculation unit 33A and the first update unit 34A is also referred to as an inner loop.

[0130] If the first condition is not satisfied, the first judgment unit 35A repeatedly executes the update by the inner loop. If the first condition is satisfied, the first judgment unit 35A ends the update by the inner loop and transmits the last updated parameter set θ{p2, β} to the second intensity function calculation unit 33B. In the following description, the parameter set transmitted to the second intensity function calculation unit 33B in the learning function is referred to as θ'{p2, β} in order to distinguish it from the parameter set before learning.

[0131] The second intensity function calculation unit 33B calculates the intensity function λ2(t) based on the time t. The second intensity function calculation unit 33B transmits the calculated intensity function λ2(t) to the second update unit 34B.

[0132] Specifically, the second intensity function calculation unit 33B includes a monotonically increasing neural network 33B-1, a cumulative intensity function calculation unit 33B-2, and an automatic differentiation unit 33B-3.

[0133] The monotonically increasing neural network 33B-1 is a mathematical model that is modeled to calculate a monotonically increasing function defined by time as an output. A plurality of parameters θ'{p2} are applied to the monotonically increasing neural network 33B-1 as weights and bias terms. The monotonically increasing neural network 33B-1 to which the plurality of parameters θ'{p2} are applied calculates an output f2(t) according to a monotonically increasing function defined by time t. The monotonically increasing neural network 33B-1 transmits the calculated output f2(t) to the cumulative intensity function calculation unit 33B-2.

[0134] The cumulative intensity function calculation unit 33B-2 calculates the cumulative intensity function Λ2(t) based on the parameter θ'{β} and the output f2(t) according to the above formula (2). The cumulative intensity function Λ2(t) is obtained by adding a term βt that increases in proportion to time t to the outputs f2(t) and f2(0) from the monotonically increasing neural network 33B-1. The cumulative intensity function calculation unit 33B-2 transmits the calculated cumulative intensity function Λ2(t) to the automatic differentiation unit 33B-3.

[0135] The automatic differentiation unit 33B-3 calculates the intensity function λ2(t) by automatically differentiating the cumulative intensity function Λ2(t). The automatic differentiation unit 33B-3 transmits the calculated intensity function λ2(t) to the second update unit 34B.

[0136] The second update unit 34B updates the parameter set θ{p2, β} based on the intensity function λ2(t) and the query sequence Eq. The updated parameters θ{p2} and θ{β} are applied to the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2, respectively. The second update unit 34B also transmits the updated parameter set θ{p2, β} to the second judgment unit 35B.

[0137] Specifically, the second updating unit 34B includes an evaluation function calculation unit 34B-1 and an optimization unit 34B-2.

[0138] The evaluation function calculation unit 34B-1 calculates an evaluation function L2(Eq) based on the intensity function λ2(t) and the query sequence Eq. The evaluation function L2(Eq) is, for example, a negative log-likelihood. The evaluation function calculation unit 34B-1 transmits the calculated evaluation function L2(Eq) to the optimization unit 34B-2.

[0139] The optimization unit 34B-2 optimizes the parameter set θ{p2, β} based on the evaluation function L2(Eq). For example, the backpropagation method is used to optimize the parameter set θ{p2, β}. More specifically, the optimization unit 34B-2 calculates the second derivative of the evaluation function L2(Eq) with respect to the parameter set θ{p2, β} using the parameter set θ'{p2, β}, and optimizes the parameter set θ{p2, β}. Then, the optimization unit 34B-2 updates the parameter set θ{p2, β} applied to the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2 with the optimized parameter set θ{p2, β}.

[0140] The second determination unit 35B determines whether or not the second condition is satisfied based on the updated parameter set θ{p2, β}. The second condition may be, for example, that the number of times the parameter set θ{p2, β} is transmitted to the second determination unit 35B (i.e., the number of update loops of the parameter set in the second intensity function calculation unit 33B and the second update unit 34B) is equal to or greater than a threshold. The second condition may be, for example, that the amount of change in the value of the parameter set θ{p2, β} before and after the update is equal to or less than a threshold. Hereinafter, the update loop of the parameter set in the second intensity function calculation unit 33B and the second update unit 34B is also referred to as an outer loop.

[0141] If the second condition is not satisfied, the second judgment unit 35B repeatedly executes the update of the parameter set by the outer loop. If the second condition is satisfied, the second judgment unit 35B ends the update of the parameter set by the outer loop and stores the last updated parameter set θ{p2, β} in the memory 11 as the learned parameters 36. In the following description, the parameter set in the learned parameters 36 is referred to as θ{p2 * ,β *} should be written.

[0142] With the above-described configuration, the event prediction device 1 has a function of generating learned parameters 36 based on the learning dataset 30.

[0143] 1.5.2 Prediction Function Configuration FIG. 13 is a block diagram showing an example of a configuration of a prediction function of an event prediction device according to the second modification.

[0144] 13, the event prediction device 1 further functions as a computer including a first intensity function calculation unit 33A, a first update unit 34A, a first determination unit 35A, a second intensity function calculation unit 33B, and a predicted sequence generation unit 38. The memory 11 of the event prediction device 1 further stores prediction data 37 as information used in the prediction operation. The configuration of the prediction data 37 is equivalent to that of the prediction data 28 in the first embodiment.

[0145] In FIG. 13, the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2 are supplied with a parameter set θ{p2 * ,β *} is applied.

[0146] Multiple parameters θ{p2 * The monotonically increasing neural network 33A-1 to which the {right arrow over (t)} is applied generates an output f1 according to a monotonically increasing function defined by time t. * (t). The monotonically increasing neural network 33A-1 calculates the calculated output f1 * (z, t) is transmitted to the cumulative intensity function calculation unit 33A-2.

[0147] The cumulative intensity function calculation unit 33A-2 calculates the parameter θ{β *} and output f1 * Based on (z,t), the cumulative intensity function Λ1 * The cumulative intensity function calculation unit 33A-2 calculates the calculated cumulative intensity function Λ * (t) is sent to the automatic differentiation unit 33A-3.

[0148] The automatic differentiation unit 33A-3 calculates the cumulative intensity function Λ * By automatically differentiating (t), the intensity function λ1 * (t). The automatic differentiation unit 33A-3 calculates the calculated intensity function λ1 * (t) is transmitted to the first judgment unit 35A.

[0149] The evaluation function calculation unit 34A-1 calculates the intensity function λ1* (t) and the forecast series Es * Based on this, the evaluation function L1(Es * ) is calculated. Evaluation function L1(Es * ) is, for example, a negative log-likelihood. The evaluation function calculation unit 34A-1 calculates the evaluation function L1(Es * ) to the optimization unit 34A-2.

[0150] The optimization unit 34A-2 calculates the evaluation function L1(Es * ) based on the parameter set θ{p2 * ,β *}. For example, the backpropagation method is used for the optimization. The optimization unit 34A-2 optimizes the optimized parameter set θ{p2 * ,β *}, the parameter set θ{p2 * ,β *} to update.

[0151] The first judgment unit 35A judges whether the updated parameter set θ{p2 * ,β *}, it is determined whether the third condition is satisfied. The third condition is, for example, * ,β *} may be equal to or greater than a threshold. * ,β *} before and after the update may be equal to or less than a threshold.

[0152] If the third condition is not satisfied, the first judgment unit 35A repeatedly executes the update of the parameter set by the inner loop. If the third condition is satisfied, the first judgment unit 35A ends the update of the parameter set by the inner loop and executes the last updated parameter set θ{p2 * ,β *} to the second intensity function calculation unit 33B. In the following description, the parameter set transmitted to the second intensity function calculation unit 33B in the prediction function is referred to as θ′{p2 * ,β *} should be written.

[0153] Parameter θ'{p2 * The monotonically increasing neural network 33B-1 to which the} is applied generates an output f2 according to a monotonically increasing function defined by time t. * (t). The monotonically increasing neural network 33B-1 calculates the calculated output f2 * (t) is transmitted to the cumulative intensity function calculation unit 33B-2.

[0154] The cumulative intensity function calculation unit 33B-2 calculates the parameter θ′{β *} and output f2 * Based on (t), the cumulative intensity function Λ2 * The cumulative intensity function calculation unit 33B-2 calculates the calculated cumulative intensity function Λ2 * (t) is transmitted to the automatic differentiation unit 33B-3.

[0155] The automatic differentiation unit 33B-3 calculates the cumulative intensity function Λ * By automatically differentiating (t), the intensity function λ2 * (t). The automatic differentiation unit 33B-3 calculates the calculated intensity function λ2 * (t) is sent to the predicted sequence generator 38.

[0156] The predicted sequence generator 38 uses the intensity function λ * Based on (t), the forecast sequence Eq * The predicted sequence generator 38 generates the predicted sequence Eq * is output to the user. Note that the predicted sequence Eq * To generate the , a simulation using, for example, the Lewis method is performed.

[0157] With the above configuration, the event prediction device 1 calculates the prediction sequence Es based on the learned parameters 36. * The forecast series Eq * It has the ability to predict.

[0158] 1.5.3 Learning behavior Fig. 14 is a flowchart showing an example of an outline of the learning operation in the event prediction device according to Modification 2. In the example of Fig. 14, it is assumed that the learning dataset 30 is stored in the memory 11 in advance.

[0159] 14, in response to an instruction to start a learning operation from a user (START), the initialization unit 32 initializes a parameter set θ{p2, β} based on rule X (S50). The parameter set θ{p2, β} initialized by the process of S30 is applied to the first intensity function calculation unit 33A.

[0160] The data extraction unit 31 extracts a sequence Ev from the learning dataset 30. Next, the data extraction unit 31 further extracts a support sequence Es and a query sequence Eq from the extracted sequence Ev (S51).

[0161] The first intensity function calculation unit 33A and the first update unit 34A, to which the parameter set θ{p2, β} initialized in the process of S50 has been applied, execute a first update process for the parameter set θ{p2, β} (S52). The details of the first update process will be described later.

[0162] After the process of S52, the first judgment unit 35A judges whether or not the first condition is satisfied based on the parameter set θ{p2, β} updated in the process of S52 (S53).

[0163] If the first condition is not satisfied (S53; no), the first intensity function calculation unit 33A to which the parameter set θ{p2, β} updated in the process of S52 is applied, and the first update unit 34A execute the first update process again (S52). In this manner, the first update process is repeated (inner loop) until it is determined in the process of S53 that the first condition is satisfied.

[0164] If the first condition is satisfied (S53; yes), the first judgment unit 35A applies the parameter set θ{p2, β} last updated in the processing of S52 to the second intensity function calculation unit 33B as the parameter set θ'{p2, β} (S54).

[0165] The second intensity function calculation unit 33B to which the parameter set θ′{p2, β} has been applied and the second update unit 34B execute a second update process for the parameter set θ{p2, β} (S55). The second update process will be described in detail later.

[0166] After the process of S55, the second determination unit 35B determines whether or not the second condition is satisfied based on the parameter set θ{p2, β} updated in the process of S55 (S56).

[0167] If the second condition is not satisfied (S56; no), the data extraction unit 31 extracts new support sequence Es and query sequence Eq (S51). Then, the inner loop and the second update process are repeated (outer loop) until it is determined in the process of S56 that the second condition is satisfied.

[0168] If the second condition is satisfied (S56; yes), the second judgment unit 35B determines that the parameter set θ{p2, β} last updated in the process of S55 is the parameter set θ{p2 * ,β *} and stored in the learned parameters 36 (S57).

[0169] When the process of S57 ends, the learning operation in the event prediction device 1 ends (ends).

[0170] 15 is a flowchart showing an example of a first update process in the event prediction device according to Modification 2. The processes of S52-1 to S52-4 shown in FIG.

[0171] After (starting) the processing of S51, the monotonically increasing neural network 33A-1 to which the multiple parameters θ{p2} initialized in the processing of S50 have been applied calculates outputs f1(t) and f1(0) according to a monotonically increasing function defined by time t (S52-1).

[0172] The cumulative intensity function calculation unit 33A-2, to which the parameter θ{β} initialized in the processing of S50 has been applied, calculates the cumulative intensity function Λ1(t) based on the outputs f1(t) and f1(0) calculated in the processing of S52-1 (S52-2).

[0173] The automatic differentiation unit 33A-3 calculates the intensity function λ1(t) based on the cumulative intensity function Λ1(t) calculated in the process of S52-2 (S52-3).

[0174] The first update unit 34A updates the parameter set θ{p2, β} based on the intensity function λ1(t) calculated in S52-3 and the support sequence Es extracted in the process of S51 (S52-4). Specifically, the evaluation function calculation unit 34A-1 calculates the evaluation function L1(Es) based on the intensity function λ1(t) and the support sequence Es. The optimization unit 34A-2 calculates an optimized parameter set θ{p2, β} based on the evaluation function L1(Es) using the backpropagation method. The optimization unit 34A-2 applies the optimized parameter set θ{p2, β} to the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2.

[0175] When the process of S52-4 ends, the first update process ends (END).

[0176] 16 is a flowchart showing an example of a second update process in the event prediction device according to Modification 2. The processes of S55-1 to S55-4 shown in FIG.

[0177] After the process of S54 (start), the monotonically increasing neural network 33B-1 to which multiple parameters θ′{p2} have been applied calculates outputs f2(t) and f2(0) according to a monotonically increasing function defined by time t (S55-1).

[0178] The cumulative intensity function calculation section 33B-2 to which the parameter θ'{β} has been applied calculates the cumulative intensity function Λ2(t) based on the outputs f2(t) and f2(0) calculated in the process of S55-1 (S55-2).

[0179] The automatic differentiation unit 33B-3 calculates the intensity function λ2(t) based on the cumulative intensity function Λ2(t) calculated in the process of S55-2 (S55-3).

[0180] The second update unit 34B updates the parameter set θ{p2, β} based on the intensity function λ2(t) calculated in S55-3 and the query sequence Eq extracted in the process of S51 (S55-4). Specifically, the evaluation function calculation unit 34B-1 calculates the evaluation function L2(Eq) based on the intensity function λ2(t) and the query sequence Eq. The optimization unit 34B-2 calculates an optimized parameter set θ{p2, β} based on the evaluation function L2(Eq) using the backpropagation method. The optimization unit 34B-2 applies the optimized parameter set θ{p2, β} to the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2.

[0181] When the process of S55-4 ends, the second update process ends (END).

[0182] 1.5.4 Predictive Behavior 17 is a flowchart showing an example of a prediction operation in the event prediction device according to the second modification. In the example of FIG. 17, a parameter set θ{p2 * ,β * 17, it is assumed that the above formula} is applied to the first intensity function calculation unit 33A. In addition, it is assumed that the prediction data 37 is stored in the memory 11 in the example of FIG.

[0183] As shown in FIG. 17, in response to a command from the user to start the prediction operation (start), multiple parameters θ{p2 * The monotonically increasing neural network 33A-1 to which the {right arrow over (t)} is applied generates an output f1 according to a monotonically increasing function defined by time t. * (t) and f1 * (0) is calculated (S60).

[0184] Parameter θ{β * The cumulative intensity function calculation unit 33A-2 to which the} has been applied is the output f1 calculated in the process of S40. * (t) and f1 * Based on (0), the cumulative intensity function Λ1 * (t) is calculated (S61).

[0185] The automatic differentiation unit 33A-3 calculates the cumulative intensity function Λ1 * Based on (t), the intensity function λ1 * (t) is calculated (S62).

[0186] The first update unit 34A updates the intensity function λ1 calculated in S62. * (t) and prediction sequence Es * Based on this, the parameter set θ{p2 * ,β * Specifically, the evaluation function calculation unit 34A-1 updates the intensity function λ1 * (t) and prediction sequence Es * Based on this, the evaluation function L1(Es * The optimization unit 34A-2 calculates the evaluation function L1(Es * ) based on the optimized parameter set θ{p2 * ,β * The optimization unit 34A-2 calculates the optimized parameter set θ{p2 * ,β *} is applied to the monotonically increasing neural network 33A-1 and the cumulative intensity function calculation unit 33A-2.

[0187] The first judgment unit 35A determines whether the parameter set θ{p2 * ,β *}, it is determined whether or not the third condition is satisfied (S64).

[0188] If the third condition is not satisfied (S64; no), the parameter set θ{p2 * ,β * The first intensity function calculation unit 33A to which the parameter set θ{p2 * ,β *} update process is repeated (inner loop).

[0189] If the third condition is satisfied (S64; yes), the first judgment unit 35A determines whether the parameter set θ{p2 * ,β *}, θ'{p2 * ,β *} and applied to the second intensity function calculation section 33B (S65).

[0190] Multiple parameters θ'{p2 * The monotonically increasing neural network 33B-1 to which the} is applied generates an output f2 according to a monotonically increasing function defined by time t. * (t) and f2 * (0) is calculated (S66).

[0191] Parameter θ'{β * The cumulative intensity function calculation unit 33B-2 to which} has been applied is the output f2 calculated in the process of S66. * (t) and f2 * Based on (0), the cumulative intensity function Λ2 * (t) is calculated (S67).

[0192] The automatic differentiation unit 33B-3 calculates the cumulative intensity function Λ2 * Based on (t), the intensity function λ2 *(t) is calculated (S68).

[0193] The predicted sequence generator 38 uses the intensity function λ2 calculated in S68. * Based on (t), the forecast sequence Eq * Then, the predicted sequence generating unit 38 generates the predicted sequence Eq * is output to the user.

[0194] When the process of S69 ends, the prediction operation in the event prediction device 1 ends (ends).

[0195] 1.5.5 Effects of the second modified example According to the second modification, the first intensity function calculation unit 33A to which the parameter set θ{p2, β} is applied calculates the intensity function λ1(t) using time t as input. The first update unit 34A updates the parameter set θ{p2, β} to the parameter set θ'{p2, β} based on the intensity function λ1(t) and the support sequence Es. The second intensity function calculation unit 33B to which the parameter set θ'{p2, β} is applied calculates the intensity function λ2(t) using time t as input. The second update unit 34B updates the parameter set θ{p2, β} based on λ2(t) and the query sequence Eq. This makes it possible to model a point process even when a meta-learning method such as MAML is used.

[0196] In this case, the cumulative intensity function calculation unit 33A-2 calculates the cumulative intensity function Λ1(t) based on the outputs f1(t) and f1(0) and the parameter θ{β}. The cumulative intensity function calculation unit 33B-2 calculates the cumulative intensity function Λ2(t) based on the outputs f2(t) and f2(0) and the parameter θ'{β}. This makes it possible to ease the requirements for expressiveness required for the outputs of the monotonically increasing neural networks 33A-1 and 33B-1. Therefore, it is possible to achieve the same effects as the first embodiment.

[0197] 2. Second embodiment Next, an information processing device according to a second embodiment will be described.

[0198] The information processing device according to the second embodiment differs from the first embodiment in that the weights of the multiple parameters p2 are initialized with random numbers generated according to a distribution with a positive mean. The second embodiment also differs from the first embodiment in that the parameter β is not used.

[0199] The information processing device according to the second embodiment is not limited to a configuration for meta-learning a point process like the information processing device according to the first embodiment, but may also be applied to a configuration for learning a point process without meta-learning. The information processing device according to the second embodiment may also be applied to a configuration for solving a regression problem for which monotonically increasing property is desired to be guaranteed. An example of a regression problem for which monotonically increasing property is desired to be guaranteed is a problem for estimating credit risk from a loan usage amount. The information processing device according to the second embodiment may also be applied to a configuration for solving a problem for which a neural network that ensures reversible transformation is used. Examples of problems for which a neural network that ensures reversible transformation is used include density estimation of empirical distributions, VAE (Variational Auto-Encoders), speech synthesis, likelihood-free inference, probabilistic programming, and image generation.

[0200] In the following, as an example of an information processing device according to the second embodiment, an event prediction device that performs meta-learning of a point process, similar to the first embodiment, will be described. In the following, configurations and operations that differ from the first embodiment will be mainly described. Descriptions of configurations and operations equivalent to those of the first embodiment will be omitted as appropriate.

[0201] 2.1 Configuration The configuration of an event prediction device according to the second embodiment will be described.

[0202] 2.1.1 Learning function configuration Fig. 18 is a block diagram showing an example of the configuration of a learning function of an event prediction device according to the second embodiment. Fig. 18 corresponds to Fig. 2 in the first embodiment.

[0203] 18, the event prediction device 1 functions as a computer including a data extraction unit 41, an initialization unit 42, a latent expression calculation unit 43, an intensity function calculation unit 44, an update unit 45, and a determination unit 46. In addition, the memory 11 of the event prediction device 1 stores a learning dataset 40 and learned parameters 47 as information used in the learning operation.

[0204] The configurations of the training dataset 40 and the data extraction unit 41 are equivalent to the configurations of the training dataset 20 and the data extraction unit 21 in Fig. 2 of the first embodiment. That is, the data extraction unit 41 extracts a support sequence Es and a query sequence Eq from the training dataset 40.

[0205] The initialization unit 42 initializes a plurality of parameters p1 based on a rule X. The initialization unit 42 transmits the initialized plurality of parameters p1 to the latent expression calculation unit 43. The initialization unit 42 also initializes weights of the plurality of parameters p2 based on a rule Y. The initialization unit 42 may initialize a bias term of the plurality of parameters p2 based on a rule X. The initialization unit 42 transmits the initialized plurality of parameters p2 to the intensity function calculation unit 44.

[0206] Rule Y involves applying random numbers generated according to a distribution with a positive mean to the weights. For example, the following three examples are examples of the application of rule Y to neural networks with multiple layers.

[0207] The first example is a method of setting all weights to fixed positive values. Specific examples of fixed positive values ​​are, for example, 0.01 or 2.0×10 -3 etc. apply.

[0208] The second example is a method for initializing weights according to a normal distribution with mean α1 and standard deviation √(α2 / n), where n is the number of nodes in the layer. Specific examples of α1 and α2 are 3.0×10 -4 , and 7.0×10 -3is applied. Any positive value can be applied to both α1 and α2. Also, the standard deviation may simply be α2.

[0209] The third example is a method of initializing weights according to a uniform distribution with a minimum value α3 and a maximum value α4. Here, α3 may be any real number equal to or greater than 0, and α4 may be any positive real number.

[0210] The configuration of the latent expression calculation unit 43 is equivalent to the configuration of the latent expression calculation unit 23 in FIG. 2 of the first embodiment. That is, the latent expression calculation unit 43 calculates the latent expression z based on the support sequence Es. The latent expression calculation unit 43 transmits the calculated latent expression z to the intensity function calculation unit 44.

[0211] The intensity function calculation unit 44 calculates an intensity function λ(t) based on the latent expression z and time t. The intensity function calculation unit 44 transmits the calculated intensity function λ(t) to the update unit 45. Specifically, the intensity function calculation unit 44 includes a monotonically increasing neural network 44-1, a cumulative intensity function calculation unit 44-2, and an automatic differentiation unit 44-3. The configurations of the monotonically increasing neural network 44-1 and the automatic differentiation unit 44-3 are equivalent to the configurations of the monotonically increasing neural network 24-1 and the automatic differentiation unit 24-3 in FIG. 2 of the first embodiment.

[0212] The monotonically increasing neural network 44-1 to which multiple parameters p2 are applied calculates an output f(z, t) according to a monotonically increasing function defined by the latent expression z and time t. The monotonically increasing neural network 44-1 transmits the calculated output f(z, t) to the cumulative intensity function calculation unit 44-2.

[0213] The cumulative intensity function calculation unit 44-2 calculates the cumulative intensity function Λ(t) based on the output f(z, t) in accordance with the following equation (3).

[0214]

number

[0215] As shown in equation (3), the cumulative intensity function Λ(t) in the second embodiment differs from the cumulative intensity function Λ(t) in the first embodiment in that a term that increases in proportion to time t is not added. The cumulative intensity function calculation unit 44-2 transmits the calculated cumulative intensity function Λ(t) to the automatic differentiation unit 44-3.

[0216] The automatic differentiation unit 44-3 automatically differentiates the cumulative intensity function Λ(t) to calculate the intensity function λ(t). The automatic differentiation unit 44-3 transmits the calculated intensity function λ(t) to the update unit 45.

[0217] The update unit 45 updates the parameters p1 and p2 based on the intensity function λ(t) and the query sequence Eq. The updated parameters p1 and p2 are applied to the neural network 43-1 and the monotonically increasing neural network 44-1, respectively. The update unit 45 also transmits the updated parameters p1 and p2 to the determination unit 46.

[0218] Specifically, the update unit 45 includes an evaluation function calculation unit 45-1 and an optimization unit 45-2. The configuration of the evaluation function calculation unit 45-1 is equivalent to the configuration of the evaluation function calculation unit 25-1 in FIG.

[0219] The evaluation function calculation unit 45-1 calculates an evaluation function L(Eq) based on the intensity function λ(t) and the query sequence Eq. The evaluation function calculation unit 45-1 transmits the calculated evaluation function L(Eq) to the optimization unit 45-2.

[0220] The optimization unit 45-2 optimizes a plurality of parameters p1 and p2 based on the evaluation function L(Eq). For example, the backpropagation method is used for the optimization. The optimization unit 45-2 updates a plurality of parameters p1 and p2 applied to the neural network 43-1 and the monotonically increasing neural network 44-1 with the optimized plurality of parameters p1 and p2.

[0221] The determination unit 46 determines whether or not a condition is satisfied based on the updated multiple parameters p1 and p2. The condition may be, for example, that the number of times the multiple parameters p1 and p2 are transmitted to the determination unit 46 (i.e., the number of parameter update loops) is equal to or greater than a threshold. The condition may be, for example, that the amount of change in the values ​​of the multiple parameters p1 and p2 before and after the update is equal to or less than a threshold. If the condition is not satisfied, the determination unit 46 causes the data extraction unit 41, the latent expression calculation unit 43, the intensity function calculation unit 44, and the update unit 45 to repeatedly execute a parameter update loop. If the condition is satisfied, the determination unit 46 ends the parameter update loop and stores the last updated multiple parameters p1 and p2 in the memory 11 as learned parameters 47. In the following description, the multiple parameters in the learned parameters 47 are referred to as p1 and p2 in order to distinguish them from parameters before learning. * and p2 * It is written as follows.

[0222] With the above-described configuration, the event prediction device 1 has a function of generating learned parameters 47 based on the learning dataset 40.

[0223] 2.1.2 Prediction Function Configuration Fig. 19 is a block diagram showing an example of a configuration of a prediction function of an event prediction device according to the second embodiment. Fig. 19 corresponds to Fig. 4 in the first embodiment.

[0224] 19, the event prediction device 1 further functions as a computer including a latent expression calculation unit 43, an intensity function calculation unit 44, and a prediction sequence generation unit 49. The memory 11 of the event prediction device 1 further stores prediction data 48 as information used in the prediction operation. In FIG. 19, a plurality of parameters p1 are input from the learned parameters 47 to the neural network 43-1 and the monotonically increasing neural network 44-1, respectively. * and p2 * is shown when it is applied.

[0225] The configuration of the prediction data 48 is equivalent to the configuration of the prediction data 28 in FIG. 4 of the first embodiment. That is, the prediction sequence Es in the prediction data 48 is input to the neural network 43-1. * Enter multiple parameters p1 * The neural network 43-1 to which the above is applied is the prediction sequence Es * is used as input, and the latent representation z * The neural network 43-1 outputs the output latent representation z * is transmitted to the monotonically increasing neural network 44-1 in the intensity function calculation unit 44.

[0226] Multiple parameters p2 * The monotonically increasing neural network 44-1 to which the above is applied is a latent representation z * and the output f * (z, t). The monotonically increasing neural network 44-1 calculates the calculated output f * (z, t) is sent to the cumulative intensity function calculation unit 44-2.

[0227] The cumulative intensity function calculation unit 44-2 calculates the output f according to the above-mentioned equation (3). * Based on (z,t), the cumulative intensity function Λ * The cumulative intensity function calculation unit 44-2 calculates the calculated cumulative intensity function Λ * (t) is sent to the automatic differentiation unit 44-3.

[0228] The automatic differentiation unit 44-3 calculates the cumulative intensity function Λ * By automatically differentiating (t), the intensity function λ * (t). The automatic differentiation unit 44-3 calculates the calculated intensity function λ * (t) is sent to the predicted sequence generation unit 49.

[0229] The configuration of the predicted sequence generator 49 is equivalent to the configuration of the predicted sequence generator 29 in FIG. 4 of the first embodiment. That is, the predicted sequence generator 49 calculates the intensity function λ *Based on (t), the forecast sequence Eq * The predicted sequence generator 49 generates the predicted sequence Eq * is output to the user.

[0230] With the above configuration, the event prediction device 1 calculates the prediction sequence Es based on the learned parameters 47. * The forecast series Eq * It has the ability to predict.

[0231] 2.2 Operation Next, the operation of the event prediction device according to the second embodiment will be described.

[0232] 2.2.1 Learning behavior Fig. 20 is a flowchart showing an example of a learning operation in the event prediction device according to the second embodiment. Fig. 20 corresponds to Fig. 6 in the second embodiment. In the example of Fig. 20, it is assumed that a learning dataset 20 is stored in advance in the memory 11.

[0233] As shown in FIG. 20, in response to an instruction from a user to start a learning operation (start), the initialization unit 42 initializes a plurality of parameters p1 and bias terms of a plurality of parameters p2 based on a rule X (S70).

[0234] Next, the initialization unit 42 initializes the weights of the multiple parameters p2 based on the rule Y (S71). For example, the initialization unit 42 initializes the weights of the multiple parameters p2 using any of the above-mentioned first to third example techniques. The multiple parameters p1 and p2 initialized by the processes of S60 and S61 are applied to the neural network 43-1 and the monotonically increasing neural network 44-1, respectively.

[0235] The data extraction unit 41 extracts a sequence Ev from the learning dataset 40. Next, the data extraction unit 41 further extracts a support sequence Es and a query sequence Eq from the extracted sequence Ev (S72).

[0236] The neural network 43-1 to which the multiple parameters p1 initialized in the process of S70 have been applied calculates a latent expression z using the support sequence Es extracted in the process of S72 as an input (S73).

[0237] The monotonically increasing neural network 44-1 to which the multiple parameters p2 initialized in the process of S71 have been applied calculates outputs f(z, t) and f(z, 0) according to the latent representation z calculated in the process of S73 and a monotonically increasing function defined by time t (S74).

[0238] The cumulative intensity function calculation unit 44-2 calculates the cumulative intensity function Λ(t) based on the outputs f(z, t) and f(z, 0) calculated in the process of S74 (S75).

[0239] The automatic differentiation unit 44-3 calculates the intensity function λ(t) based on the cumulative intensity function Λ(t) calculated in the process of S75 (S76).

[0240] The update unit 45 updates the multiple parameters p1 and p2 based on the intensity function λ(t) calculated in S76 and the query sequence Eq extracted in the process of S72 (S77). Specifically, the evaluation function calculation unit 45-1 calculates the evaluation function L(Eq) based on the intensity function λ(t) and the query sequence Eq. The optimization unit 45-2 calculates the multiple optimized parameters p1 and p2 based on the evaluation function L(Eq) using the backpropagation method. The optimization unit 45-2 applies the multiple optimized parameters p1 and p2 to the neural network 43-1 and the monotonically increasing neural network 44-1, respectively.

[0241] The determination unit 46 determines whether or not the condition is satisfied based on the multiple parameters p1 and p2 (S78).

[0242] If the condition is not satisfied (S78; no), the data extraction unit 41 extracts a new support sequence Es and a query sequence Eq from the training dataset 40 (S72). Then, the processes of S73 to S78 are executed based on the multiple parameters p1 and p2 updated in the process of S77. Thus, the update process of the multiple parameters p1 and p2 is repeated until it is determined in the process of S78 that the condition is satisfied.

[0243] If the condition is satisfied (S78; yes), the determination unit 46 updates the parameters p1 and p2 last updated in the process of S77 as follows: * and p2 * The learned parameters are stored in the learned parameters 47 (S79).

[0244] When the process of S79 ends, the learning operation in the event prediction device 1 ends (ends).

[0245] 2.2.2 Predictive Behavior Fig. 21 is a flowchart showing an example of a prediction operation in the event prediction device according to the second embodiment. Fig. 21 corresponds to Fig. 7 in the first embodiment. In the example of Fig. 7, a learning operation executed in advance has been performed to predict the prediction result of a plurality of parameters p1 in the learned parameters 47. * and p2 * are applied to the neural network 43-1 and the monotonically increasing neural network 44-1, respectively. In the example of FIG.

[0246] As shown in FIG. 21, in response to a user's instruction to start a prediction operation (start), a plurality of parameters p1 * The neural network 43-1 to which the above is applied is the prediction sequence Es * is used as input, and the latent representation z * is calculated (S80).

[0247] Multiple parameters p2 * The monotonically increasing neural network 44-1 to which the above is applied uses the latent representation z* , and the output f * (z,t) and f * (z, 0) is calculated (S81).

[0248] The cumulative intensity function calculation unit 44-2 calculates the output f * (z,t) and f * Based on (z,0), the cumulative intensity function Λ * (t) is calculated (S82).

[0249] The automatic differentiation unit 44-3 calculates the cumulative intensity function Λ calculated in the process of S82. * Based on (t), the intensity function λ * (t) is calculated (S83).

[0250] The predicted sequence generator 49 calculates the intensity function λ * Based on (t), the forecast sequence Eq * Then, the predicted sequence generating unit 49 generates the predicted sequence Eq * is output to the user.

[0251] When the process of S84 ends, the prediction operation in the event prediction device 1 ends (ends).

[0252] 2.3 Effects of the Second Embodiment According to the second embodiment, the initialization unit 42 initializes the weights of the multiple parameters p2 based on a distribution with a positive mean. Specifically, the initialization unit 42 initializes the weights of the multiple parameters p2 with a positive fixed value. Alternatively, the initialization unit 42 initializes the weights of the multiple parameters p2 with random numbers generated according to a normal distribution with a mean of α1 and a standard deviation of √(α2 / n) (α1 and α2 are positive real numbers). Alternatively, the initialization unit 42 initializes the weights of the multiple parameters p2 with random numbers generated according to a uniform distribution with a minimum value of α3 and a maximum value of α4 (α3 is a real number equal to or greater than 0, and α4 is a positive real number). This makes it possible to diversify the output of the activation function in the monotonically increasing neural network 44-1 and to suppress the gradient vanishing of the activation function.

[0253] 2.4 Third variant Various modifications may be applied to the second embodiment described above. For example, in the second embodiment described above, a case has been described in which a neural network is used to input the latent expression z calculated from the learning data set 20 and the time t to be predicted when modeling the intensity function λ(t), but this is not limited thereto. For example, as in the second modified example, the modeling of the intensity function λ(t) may be realized by combining it with a meta-learning method such as MAML. In the following, configurations and operations different from the second embodiment will be mainly described. Then, descriptions of configurations and operations equivalent to those of the second embodiment will be omitted as appropriate.

[0254] 2.4.1 Learning function configuration FIG. 22 is a block diagram showing an example of the configuration of a learning function of an event prediction device according to the third modified example.

[0255] 22, the event prediction device 1 functions as a computer including a data extraction unit 51, an initialization unit 52, a first intensity function calculation unit 53A, a second intensity function calculation unit 53B, a first update unit 54A, a second update unit 54B, a first determination unit 55A, and a second determination unit 55B. In addition, the memory 11 of the event prediction device 1 stores a learning dataset 50 and learned parameters 56 as information used in the learning operation.

[0256] The configurations of the training dataset 50 and the data extraction unit 51 are equivalent to those of the training dataset 40 and the data extraction unit 41 in the second embodiment shown in Fig. 18. That is, the data extraction unit 51 extracts a support sequence Es and a query sequence Eq from the training dataset 50.

[0257] The initialization unit 52 initializes the weights of the multiple parameters p2 based on rule Y. The initialization unit 52 may initialize the bias term of the multiple parameters p2 based on rule X. The initialization unit 52 transmits the initialized multiple parameters p2 to the first intensity function calculation unit 53A. In the third modified example, the set of the multiple parameters p2 is also called a parameter set θ{p2}.

[0258] The first intensity function calculation unit 53A calculates the intensity function λ1(t) based on the time t. The first intensity function calculation unit 53A transmits the calculated intensity function λ1(t) to the first update unit 54A.

[0259] Specifically, the first intensity function calculation unit 53A includes a monotonically increasing neural network 53A-1, a cumulative intensity function calculation unit 53A-2, and an automatic differentiation unit 53A-3.

[0260] The monotonically increasing neural network 53A-1 is a mathematical model that is modeled to calculate a monotonically increasing function defined by time as an output. A plurality of weights and bias terms based on the parameter set θ{p2} are applied to the monotonically increasing neural network 53A-1. Each weight applied to the monotonically increasing neural network 53A-1 is a non-negative value. The monotonically increasing neural network 53A-1 to which the parameter set θ{p2} is applied calculates an output f1(t) according to a monotonically increasing function defined by time t. The monotonically increasing neural network 53A-1 transmits the calculated output f1(t) to the cumulative intensity function calculation unit 53A-2.

[0261] The cumulative intensity function calculation unit 53A-2 calculates the cumulative intensity function Λ1(t) based on the parameter θ{β} and the output f1(t) in accordance with the following equation (4).

[0262]

number

[0263] As shown in equation (4), the cumulative intensity function Λ1(t) differs from the cumulative intensity function Λ1(t) in the second modification in that a term that increases in proportion to time t is not added. The cumulative intensity function calculation unit 53A-2 transmits the calculated cumulative intensity function Λ1(t) to the automatic differentiation unit 53A-3.

[0264] The automatic differentiation unit 53A-3 calculates the intensity function λ1(t) by automatically differentiating the cumulative intensity function Λ1(t). The automatic differentiation unit 53A-3 transmits the calculated intensity function λ1(t) to the first update unit 54A.

[0265] The first update unit 54A updates the parameter set θ{p2} based on the intensity function λ1(t) and the support sequence Es. The updated parameter set θ{p2} is applied to the monotonically increasing neural network 53A-1. The first update unit 54A also transmits the updated parameter set θ{p2} to the first judgment unit 55A.

[0266] Specifically, the first updating unit 54A includes an evaluation function calculation unit 54A-1 and an optimization unit 54A-2.

[0267] The evaluation function calculation unit 54A-1 calculates an evaluation function L1(Es) based on the intensity function λ1(t) and the support sequence Es. The evaluation function L1(Es) is, for example, a negative log-likelihood. The evaluation function calculation unit 54A-1 transmits the calculated evaluation function L1(Es) to the optimization unit 54A-2.

[0268] The optimization unit 54A-2 optimizes the parameter set θ{p2} based on the evaluation function L1(Es). For example, the backpropagation method is used for the optimization. The optimization unit 54A-2 updates a plurality of parameters p2 applied to the monotonically increasing neural network 53A-1 and the cumulative intensity function calculation unit 53A-2 with the optimized parameter set θ{p2}.

[0269] The first determination unit 55A determines whether or not the first condition is satisfied based on the updated parameter set θ{p2}. The first condition may be, for example, that the number of times the parameter set θ{p2} is transmitted to the first determination unit 55A (i.e., the number of update loops of the parameter set in the first intensity function calculation unit 53A and the first update unit 54A) is equal to or greater than a threshold. The first condition may be, for example, that the amount of change in the value of the parameter set θ{p2} before and after the update is equal to or less than a threshold. Hereinafter, the update loop of the parameter set in the first intensity function calculation unit 53A and the first update unit 54A is also referred to as an inner loop.

[0270] If the first condition is not satisfied, the first judgment unit 55A repeatedly executes the update of the parameter set by the inner loop. If the first condition is satisfied, the first judgment unit 55A ends the update of the parameter set by the inner loop and transmits the last updated parameter set θ{p2} to the second intensity function calculation unit 53B. In the following description, the parameter set transmitted to the second intensity function calculation unit 53B in the learning function is referred to as θ'{p2} in order to distinguish it from the parameter set before learning.

[0271] The second intensity function calculation unit 53B calculates the intensity function λ2(t) based on the time t. The second intensity function calculation unit 53B transmits the calculated intensity function λ2(t) to the second update unit 54B.

[0272] Specifically, the second intensity function calculation section 53B includes a monotonically increasing neural network 53B-1, a cumulative intensity function calculation section 53B-2, and an automatic differentiation section 53B-3.

[0273] The monotonically increasing neural network 53B-1 is a mathematical model that is modeled to calculate a monotonically increasing function defined by time as an output. Weights and bias terms based on the parameter set θ'{p2} are applied to the monotonically increasing neural network 53B-1. The monotonically increasing neural network 53B-1 to which the parameter set θ'{p2} is applied calculates an output f2(t) according to a monotonically increasing function defined by time t. The monotonically increasing neural network 53B-1 transmits the calculated output f2(t) to the cumulative intensity function calculation unit 53B-2.

[0274] The cumulative intensity function calculation unit 53B-2 calculates the cumulative intensity function Λ2(t) based on the output f2(t) according to the above-mentioned formula (4). The cumulative intensity function Λ2(t) does not include a term that increases in proportion to the time t. The cumulative intensity function calculation unit 53B-2 transmits the calculated cumulative intensity function Λ2(t) to the automatic differentiation unit 53B-3.

[0275] The automatic differentiation unit 53B-3 calculates the intensity function λ2(t) by automatically differentiating the cumulative intensity function Λ2(t). The automatic differentiation unit 53B-3 transmits the calculated intensity function λ2(t) to the second update unit 54B.

[0276] The second update unit 54B updates the parameter set θ{p2} based on the intensity function λ2(t) and the query sequence Eq. The updated parameter set θ{p2} is applied to the monotonically increasing neural network 53A-1. The second update unit 54B also transmits the updated parameter set θ{p2} to the second determination unit 55B.

[0277] Specifically, the second updating unit 54B includes an evaluation function calculation unit 54B-1 and an optimization unit 54B-2.

[0278] The evaluation function calculation unit 54B-1 calculates an evaluation function L2(Eq) based on the intensity function λ2(t) and the query sequence Eq. The evaluation function L2(Eq) is, for example, a negative log-likelihood. The evaluation function calculation unit 54B-1 transmits the calculated evaluation function L2(Eq) to the optimization unit 54B-2.

[0279] The optimization unit 54B-2 optimizes the parameter set θ{p2} based on the evaluation function L2(Eq). For example, the backpropagation algorithm is used to optimize the parameter set θ{p2}. More specifically, the optimization unit 54B-2 calculates the second derivative of the evaluation function L2(Eq) with respect to the parameter set θ{p2} using the parameter set θ'{p2}, and optimizes the parameter set θ{p2}. Then, the optimization unit 54B-2 updates the parameter set θ{p2} to be applied to the monotonically increasing neural network 53A-1 with the optimized parameter set θ{p2}.

[0280] The second determination unit 55B determines whether or not the second condition is satisfied based on the updated parameter set θ{p2}. The second condition may be, for example, that the number of times the parameter set θ{p2} is transmitted to the second determination unit 55B (i.e., the number of update loops of the parameter set in the second intensity function calculation unit 53B and the second update unit 54B) is equal to or greater than a threshold. The second condition may be, for example, that the amount of change in the value of the parameter set θ{p2} before and after the update is equal to or less than a threshold. Hereinafter, the update loop of the parameter set in the second intensity function calculation unit 53B and the second update unit 54B is also referred to as an outer loop.

[0281] If the second condition is not satisfied, the second judgment unit 55B repeatedly executes the update of the parameter set by the outer loop. If the second condition is satisfied, the second judgment unit 55B ends the update of the parameter set by the outer loop and stores the last updated parameter set θ{p2} in the memory 11 as the learned parameters 56. In the following description, the parameter set in the learned parameters 56 is referred to as θ{p2} in order to distinguish it from the parameter set before learning by the outer loop. *} should be written.

[0282] With the above-described configuration, the event prediction device 1 has a function of generating learned parameters 56 based on the learning dataset 50.

[0283] 2.4.2 Prediction Function Configuration FIG. 23 is a block diagram showing an example of a configuration of a prediction function of an event prediction device according to the third modified example.

[0284] 23, the event prediction device 1 further functions as a computer including a first intensity function calculation unit 53A, a first update unit 54A, a first determination unit 55A, a second intensity function calculation unit 53B, and a predicted sequence generation unit 58. The memory 11 of the event prediction device 1 further stores prediction data 57 as information used in the prediction operation. The configuration of the prediction data 57 is the same as that of the prediction data 48 in the second embodiment, and therefore a description thereof will be omitted.

[0285] In FIG. 23, the monotonically increasing neural network 53A-1 is supplied with a parameter set θ{p2 *} is applied.

[0286] Parameter set θ{p2 * The monotonically increasing neural network 53A-1 to which the {right arrow over (t)} is applied generates an output f1 according to a monotonically increasing function defined by time t. * (t). The monotonically increasing neural network 53A-1 calculates the calculated output f1 * (z, t) is transmitted to cumulative intensity function calculation unit 53A-2.

[0287] The cumulative intensity function calculation unit 53A-2 calculates the output f1 according to the above-mentioned equation (4). * Based on (t), the cumulative intensity function Λ1 * The cumulative intensity function calculation unit 53A-2 calculates the calculated cumulative intensity function Λ * (t) is transmitted to the automatic differentiation unit 53A-3.

[0288] The automatic differentiation unit 53A-3 calculates the cumulative intensity function Λ * By automatically differentiating (t), the intensity function λ1 * (t). The automatic differentiation unit 53A-3 calculates the calculated intensity function λ1 *(t) is transmitted to the first judgment unit 55A.

[0289] The evaluation function calculation unit 54A-1 calculates the intensity function λ1 * (t) and the forecast series Es * Based on this, the evaluation function L1(Es * ) is calculated. Evaluation function L1(Es * ) is, for example, a negative log-likelihood. The evaluation function calculation unit 54A-1 calculates the evaluation function L1(Es * ) to the optimization unit 54A-2.

[0290] The optimization unit 54A-2 calculates the evaluation function L1(Es * ) based on the parameter set θ{p2 *}. For example, the backpropagation method is used for the optimization. The optimization unit 54A-2 optimizes the optimized parameter set θ{p2 *}, the parameter set {p2 *} to update.

[0291] The first judgment unit 55A judges whether the updated parameter set θ{p2 *}, it is determined whether the third condition is satisfied. The third condition is, for example, *} may be equal to or greater than a threshold. *} before and after the update may be equal to or less than a threshold.

[0292] If the third condition is not satisfied, the first judgment unit 55A repeatedly executes the update of the parameter set by the inner loop. If the third condition is satisfied, the first judgment unit 55A ends the update of the parameter set by the inner loop and executes the last updated parameter set θ{p2 *} to the second intensity function calculation unit 53B. In the following description, the parameter set transmitted to the second intensity function calculation unit 53B in the prediction function is referred to as θ′{p2 *} should be written.

[0293] Parameter set θ'{p2 * The monotonically increasing neural network 53B-1 to which the {right arrow over (t)} is applied generates an output f2 according to a monotonically increasing function defined by time t. * (t). The monotonically increasing neural network 53B-1 calculates the calculated output f2 * (t) is transmitted to the cumulative intensity function calculation unit 53B-2.

[0294] The cumulative intensity function calculation unit 53B-2 calculates the output f2 according to the above-mentioned equation (4). * Based on (t), the cumulative intensity function Λ2 * The cumulative intensity function calculation unit 53B-2 calculates the calculated cumulative intensity function Λ2 * (t) is transmitted to the automatic differentiation unit 53B-3.

[0295] The automatic differentiation unit 53B-3 calculates the cumulative intensity function Λ * By automatically differentiating (t), the intensity function λ2 * (t). The automatic differentiation unit 53B-3 calculates the calculated intensity function λ2 * (t) is sent to the predicted sequence generator 58.

[0296] The predicted sequence generator 58 uses the intensity function λ * Based on (t), the forecast sequence Eq * The predicted sequence generator 58 generates the predicted sequence Eq * is output to the user.

[0297] With the above configuration, the event prediction device 1 calculates the prediction sequence Es based on the learned parameters 56. * The forecast series Eq * It has the ability to predict.

[0298] 2.4.3 Learning behavior Fig. 24 is a flowchart showing an example of an outline of the learning operation in the event prediction device according to Modification 3. In the example of Fig. 24, it is assumed that a learning dataset 50 is stored in the memory 11 in advance.

[0299] As shown in FIG. 24, in response to an instruction from a user to start a learning operation (start), the initialization unit 52 initializes the bias term in the parameter set θ{p2} based on the rule X (S90).

[0300] The initialization unit 52 initializes the weights in the parameter set θ{p2} based on the rule Y (S91). For example, the initialization unit 52 initializes the weights in the parameter set θ{p2} based on any of the above-mentioned first to third example techniques. The parameter set θ{p2} initialized by the processes of S90 and S91 is applied to the first intensity function calculation unit 53A.

[0301] The data extraction unit 51 extracts a sequence Ev from the learning dataset 50. Next, the data extraction unit 51 further extracts a support sequence Es and a query sequence Eq from the extracted sequence Ev (S92).

[0302] The first intensity function calculation unit 53A and the first update unit 54A to which the parameter set θ{p2} initialized in the processes of S90 and S91 has been applied execute a first update process for the parameter set θ{p2} (S93). The details of the first update process will be described later.

[0303] The first determination unit 55A determines whether or not the first condition is satisfied based on the parameter set θ{p2} updated in the process of S93 (S94).

[0304] If the first condition is not satisfied (S94; no), the first intensity function calculation unit 53A to which the parameter set θ{p2} updated in the process of S93 is applied, and the first update unit 54A execute the first update process again (S93). In this manner, the first update process is repeated (inner loop) until it is determined in the process of S94 that the first condition is satisfied.

[0305] If the first condition is satisfied (S94; yes), the first judgment unit 55A applies the parameter set θ{p2} last updated in the processing of S93 to the second intensity function calculation unit 53B as the parameter set θ'{p2} (S95).

[0306] The second intensity function calculation unit 53B to which the parameter set θ′{p2} has been applied, and the second update unit 54B execute a second update process for the parameter set θ{p2} (S96). The second update process will be described in detail later.

[0307] The second determination unit 55B determines whether or not the second condition is satisfied based on the parameter set θ{p2} updated in the process of S96 (S97).

[0308] If the second condition is not satisfied (S97; no), the data extraction unit 51 extracts new support sequence Es and query sequence Eq (S92). Then, the inner loop and the second update process are repeated (outer loop) until it is determined in the process of S97 that the second condition is satisfied.

[0309] If the second condition is satisfied (S97; yes), the second judgment unit 55B determines that the parameter set θ{p2} last updated in the process of S96 is the parameter set θ{p2 *} and stored in the learned parameters 56 (S98).

[0310] When the process of S98 ends, the learning operation in the event prediction device 1 ends (ends).

[0311] 25 is a flowchart showing an example of a first update process in the event prediction device according to Modification Example 3. The processes of S93-1 to S93-4 shown in FIG.

[0312] After (starting) the processing of S92, the monotonically increasing neural network 53A-1 to which the parameter set θ{p2} initialized in the processing of S90 and S91 has been applied calculates outputs f1(t) and f1(0) according to a monotonically increasing function defined by time t (S93-1).

[0313] The cumulative intensity function calculation section 53A-2 calculates a cumulative intensity function Λ1(t) based on the outputs f1(t) and f1(0) calculated in the process of S93-1 (S93-2).

[0314] The automatic differentiation unit 53A-3 calculates the intensity function λ1(t) based on the cumulative intensity function Λ1(t) calculated in the process of S93-2 (S93-3).

[0315] The first update unit 54A updates the parameter set θ{p2} based on the intensity function λ1(t) calculated in S93-3 and the support sequence Es extracted in the process of S92 (S93-4). Specifically, the evaluation function calculation unit 54A-1 calculates the evaluation function L1(Es) based on the intensity function λ1(t) and the support sequence Es. The optimization unit 54A-2 calculates an optimized parameter set θ{p2} based on the evaluation function L1(Es) using the backpropagation method. The optimization unit 54A-2 applies the optimized parameter set θ{p2} to the monotonically increasing neural network 53A-1 and the cumulative intensity function calculation unit 53A-2.

[0316] When the process of S93-4 ends, the first update process ends (END).

[0317] 26 is a flowchart showing an example of a second update process in the event prediction device according to Modification Example 3. The processes of S96-1 to S96-4 shown in FIG.

[0318] After the process of S95 (start), the monotonically increasing neural network 53B-1 to which the parameter set θ′{p2} has been applied calculates outputs f2(t) and f2(0) according to a monotonically increasing function defined by time t (S96-1).

[0319] The cumulative intensity function calculation section 53B-2 calculates the cumulative intensity function Λ2(t) based on the outputs f2(t) and f2(0) calculated in the process of S96-1 (S96-2).

[0320] The automatic differentiation section 53B-3 calculates the intensity function λ2(t) based on the cumulative intensity function Λ2(t) calculated in the process of S96-2 (S96-3).

[0321] The second update unit 54B updates the parameter set θ{p2} based on the intensity function λ2(t) calculated in S96-3 and the query sequence Eq extracted in the process of S92 (S96-4). Specifically, the evaluation function calculation unit 54B-1 calculates the evaluation function L2(Eq) based on the intensity function λ2(t) and the query sequence Eq. The optimization unit 54B-2 calculates an optimized parameter set θ{p2} based on the evaluation function L2(Eq) using the backpropagation method. The optimization unit 54B-2 applies the optimized parameter set θ{p2} to the monotonically increasing neural network 53A-1 and the cumulative intensity function calculation unit 53A-2.

[0322] When the process of S96-4 ends, the second update process ends (END).

[0323] 2.4.4 Predictive Behavior 27 is a flowchart showing an example of a prediction operation in the event prediction device according to the third modification. In the example of FIG. 27, a parameter set θ{p2 * 27, it is assumed that the above formula} is applied to the first intensity function calculation unit 53A. In addition, it is assumed that the prediction data 57 is stored in the memory 11 in the example of FIG.

[0324] As shown in FIG. 27, in response to a command from a user to start a prediction operation (start), the parameter set θ{p2 * The monotonically increasing neural network 53A-1 to which the {right arrow over (t)} is applied generates an output f1 according to a monotonically increasing function defined by time t.* (t) and f1 * (0) is calculated (S100).

[0325] The cumulative intensity function calculation unit 53A-2 calculates the output f1 calculated in the process of S100. * (t) and f1 * Based on (0), the cumulative intensity function Λ1 * (t) is calculated (S101).

[0326] The automatic differentiation unit 53A-3 calculates the cumulative intensity function Λ1 * Based on (t), the intensity function λ1 * (t) is calculated (S102).

[0327] The first update unit 54A updates the intensity function λ1 calculated in S102. * (t) and prediction sequence Es * Based on this, the parameter set θ{p2 * Specifically, the evaluation function calculation unit 54A-1 updates the intensity function λ1 * (t) and prediction sequence Es * Based on this, the evaluation function L1(Es * The optimization unit 54A-2 calculates the evaluation function L1(Es * ) based on the optimized parameter set θ{p2 * The optimization unit 54A-2 calculates the optimized parameter set θ{p2 *} is applied to the monotonically increasing neural network 53A-1.

[0328] After the process of S103, the first judgment unit 55A judges whether the parameter set θ{p2 *}, it is determined whether or not the third condition is satisfied (S104).

[0329] If the third condition is not satisfied (S104; no), the parameter set θ{p2 *The first intensity function calculation unit 53A to which the parameter set θ{p2 *} update process is repeated (inner loop).

[0330] If the third condition is satisfied (S104; yes), the first judgment unit 55A judges whether the parameter set θ{p2 *}, θ'{p2 *} and applied to the second intensity function calculation unit 53B (S105).

[0331] The parameter set θ'{p2 * The monotonically increasing neural network 53B-1 to which the {right arrow over (t)} is applied generates an output f2 according to a monotonically increasing function defined by time t. * (t) and f2 * (0) is calculated (S106).

[0332] The cumulative intensity function calculation unit 53B-2 calculates the output f2 * (t) and f2 * Based on (0), the cumulative intensity function Λ2 * (t) is calculated (S107).

[0333] The automatic differentiation unit 53B-3 calculates the cumulative intensity function Λ2 * Based on (t), the intensity function λ2 * (t) is calculated (S108).

[0334] The predicted sequence generator 58 uses the intensity function λ2 calculated in S108. * Based on (t), the forecast sequence Eq * Then, the predicted sequence generating unit 58 generates the predicted sequence Eq * is output to the user.

[0335] When the process of S109 ends, the prediction operation in the event prediction device 1 ends (ends).

[0336] 2.4.5 Effects of the third modified example According to the third modification, the first intensity function calculation unit 53A to which the parameter set θ{p2} is applied calculates the intensity function λ1(t) using time t as input. The first update unit 54A updates the parameter set θ{p2} to the parameter set θ'{p2} based on the intensity function λ1(t) and the support sequence Es. The second intensity function calculation unit 53B to which the parameter set θ'{p2} is applied calculates the intensity function λ2(t) using time t as input. The second update unit 54B updates the parameter set θ{p2} based on λ2(t) and the query sequence Eq. This makes it possible to model a point process even when a meta-learning method such as MAML is used.

[0337] In this case, the cumulative intensity function calculation unit 53A-2 calculates the cumulative intensity function Λ1(t) based on the outputs f1(t) and f1(0). The cumulative intensity function calculation unit 53B-2 calculates the cumulative intensity function Λ2(t) based on the outputs f2(t) and f2(0). This makes it possible to ease the requirements for expressiveness required for the outputs of the monotonically increasing neural networks 53A-1 and 53B-1. This makes it possible to achieve the same effects as the second embodiment.

[0338] 3. Third embodiment Next, an information processing device according to a third embodiment will be described.

[0339] In the third embodiment, the calculation method of the cumulative intensity function Λ(t) in the first embodiment and the initialization method according to the rule Y in the second embodiment are used in combination. In this case, in addition to the outputs f(z, t) and f(z, 0), a term βt that increases in proportion to time t is added to the cumulative intensity function Λ(t). Furthermore, as the weight of the multiple parameters p2, random numbers generated according to a distribution with a positive mean, for example, random numbers generated by any of the methods in the first to third examples described above, are applied.

[0340] According to the third embodiment, it is possible to simultaneously achieve the effects of the first embodiment and the effects of the second embodiment, thereby making it possible to more stably predict long-term events.

[0341] 4. Other Modifications Various modifications may be applied to the above-mentioned first to third embodiments and first to third modified examples. In the following, for the modifications to the first to third embodiments and the first modified example, differences from the first embodiment will be described. In addition, for the modifications to the second and third modified examples, differences from the second modified example will be described.

[0342] 4.1 Fourth variant In the above-mentioned first to third embodiments and first modified example, the cases where no mark or additional information is attached to each event have been described, but this is not limited thereto. For example, each event may be attached with a mark or additional information. The mark or additional information attached to each event may be, for example, what the user has purchased, the payment method, etc. In the following, for simplicity, the mark or additional information will be simply referred to as a "mark."

[0343] 28 is a block diagram showing an example of the configuration of a latent expression calculation unit of an event prediction device according to a fourth modified example. The latent expression calculation unit 23 further includes a neural network 23-2. In the example of FIG. 28, the support sequence Es is calculated based on the event occurrence time t i , and the sequence of pairs of marks mi (Es={(t i ,mi)}).

[0344] Neural network 23-2 is marked m i With input, mark m i Considering the parameter NN2(m i ) is a mathematical model that is modeled to output. The neural network 23-2 outputs the event occurrence time t i Then, output NN2(m i ), we obtain the sequence Es'={[ti NN2(m i )]}. The neural network 23-2 transmits the generated sequence Es' to the neural network 23-1.

[0345] The neural network 23-1 receives the sequence Es′ as input and outputs the latent expression z. The neural network 23-1 transmits the output latent expression z to the intensity function calculation unit 24.

[0346] Although not shown in Fig. 28, multiple parameters are applied to the neural network 23-2. The multiple parameters applied to the neural network 23-2 are initialized by the initialization unit 22 and updated by the update unit 25, similar to the multiple parameters p1, p2, and β.

[0347] With the above configuration, the latent expression calculation unit 23 calculates the mark m i The latent expression z can be calculated while taking into account the above. This can improve the accuracy of event prediction.

[0348] 4.2 Fifth variant In the above-mentioned first to third embodiments and first modified example, the case where additional information is not added to the series has been described, but this is not limited thereto. For example, additional information may be added to the series. The additional information added to the series is, for example, user attribute information such as the user's gender and age.

[0349] 29 is a block diagram showing an example of the configuration of an intensity function calculation unit of an event prediction device according to Modification 5. The intensity function calculation unit 24 further includes neural networks 24-5 and 24-6.

[0350] The neural network 24-5 is a mathematical model that is modeled so as to receive additional information a as input and output parameters NN3(a) that take the additional information a into consideration. The neural network 24-5 transmits the output parameters NN3(a) to the neural network 24-6.

[0351] The neural network 24-6 receives the latent representation z and the parameter NN3(a) and outputs the latent representation z'=NN4([z, NN3(a)]) that takes into account the additional information a. The neural network 24-6 transmits the output latent representation z' to the monotonically increasing neural network 24-1.

[0352] The monotonically increasing neural network 24-1 calculates an output f(z',t) according to a monotonically increasing function defined by the latent expression z' and time t. The monotonically increasing neural network 24-1 transmits the calculated output f(z',t) to the cumulative intensity function calculation unit 24-2.

[0353] The configurations of the cumulative intensity function calculation unit 24-2 and the automatic differentiation unit 24-3 are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0354] Although not shown in Fig. 29, a plurality of parameters are applied to each of the neural networks 24-5 and 24-6. The plurality of parameters applied to the neural networks 24-5 and 24-6 are initialized by the initialization unit 22 and updated by the update unit 25, similar to the plurality of parameters p1, p2, and β.

[0355] With the above configuration, the intensity function calculation unit 24 can calculate the output f(z', t) while taking into account the additional information a, thereby improving the accuracy of event prediction.

[0356] 4.3 Sixth Variation In the above-described second and third modified examples, the case where the additional information is not added to the sequence Es has been described, but this is not limiting. For example, the additional information may be added to the sequence.

[0357] Fig. 30 is a block diagram showing an example of a configuration of a first intensity function calculation unit of an event prediction device according to Modification 6. Fig. 31 is a block diagram showing an example of a configuration of a second intensity function calculation unit of an event prediction device according to Modification 6. The first intensity function calculation unit 33A and the second intensity function calculation unit 33B further include neural networks 33A-4 and 33B-4, respectively.

[0358] The neural networks 33A-4 and 33B-4 are mathematical models that are modeled so as to receive additional information a and output parameters NN5(a) that take the additional information a into consideration. The neural networks 33A-4 and 33B-4 transmit the output parameters NN5(a) to the monotonically increasing neural networks 33A-1 and 33B-1, respectively.

[0359] The monotonically increasing neural networks 33A-1 and 33B-1 calculate outputs f1(t) and f2(t), respectively, according to a monotonically increasing function defined by a parameter NN5(a) and time t. Here, the outputs f1(t) and f2(t) are both expressed as MNN([t, NN5(a)]). The monotonically increasing neural network 33A-1 transmits the calculated output f1(t) to the cumulative intensity function calculation unit 33A-2. The monotonically increasing neural network 33B-1 transmits the calculated output f2(t) to the cumulative intensity function calculation unit 33B-2.

[0360] The configurations of the cumulative intensity function calculation units 33A-2 and 33B-2 and the automatic differentiation units 33A-3 and 33B-3 are the same as those in the second modified example, and therefore description thereof will be omitted.

[0361] Although not shown in Fig. 30 and Fig. 31, a plurality of parameters are applied to each of the neural networks 33A-4 and 33B-4. The plurality of parameters applied to the neural network 33A-4 are initialized by the initialization unit 32 and updated by the first update unit 34A, similar to the parameter set θ{p2, β}. The plurality of parameters applied to the neural network 33B-4 are used for updating by the second update unit 34B, similar to the parameter set θ'{p2, β}.

[0362] With the above configuration, the first intensity function calculation unit 33A and the second intensity function calculation unit 33B can calculate the outputs f1(t) and f2(t), respectively, while taking into account the additional information a. This can improve the accuracy of event prediction.

[0363] 4.4 Other In the above-mentioned first to third embodiments and first to sixth modified examples, the dimension of an event is described as one dimension of time, but is not limited to this. For example, the dimension of an event can be expanded to any number of dimensions equal to or greater than two (for example, three dimensions of space-time).

[0364] In the above-described first to third embodiments and first to sixth modified examples, the learning operation and the prediction operation are executed by a program stored in the event prediction device 1, but the present invention is not limited to this. For example, the learning operation and the prediction operation may be executed by a computing resource on the cloud.

[0365] In the above-mentioned second, third and sixth modified examples, the first and second intensity function calculation units, the first and second update units, and the first and second judgment units are described as separate functional blocks, but this is not limited thereto. For example, the first and second intensity function calculation units, the first and second update units, and the first and second judgment units may be realized by the same functional block.

[0366] The present invention is not limited to the above-mentioned embodiment, and can be modified in various ways without departing from the gist of the present invention. The embodiments may be combined as appropriate, and in that case, the combined effect can be obtained. Furthermore, the above-mentioned embodiment includes various inventions, and various inventions can be extracted by combinations selected from the multiple components disclosed. For example, if the problem can be solved and the effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention. [Explanation of symbols]

[0367] 1... event prediction device, 10... control circuit, 11... memory, 12... communication module, 13... user interface, 14... drive, 15... storage medium, 20, 30, 40, 50... learning data set, 21, 31, 41, 51... data extraction unit, 22, 32, 42, 52... initialization unit, 23, 43... latent expression calculation unit, 23-1, 23-2, 24-4, 24-5, 25-6, 33A-4, 33B-4, 43-1... neural network, 24, 44... intensity function calculation unit, 33A, 53A... first intensity function calculation unit, 33B, 53B... second intensity function calculation unit, 24-1, 33A-1, 33B-1, 44-1, 53A-1, 53B-1... monotonically increasing neural network, 24-2 ,33A-2,33B-2,44-2,53A-2,53B-2...cumulative intensity function calculation section,24-3,33A-3,33B-3,44-3,53A-3,53B-3... Automatic differentiation section, 25,45...update section, 34A,54A...first update section, 34B,54B...second update section, 25-1,34A-1,34B-1,45-1,54A-1,54 B-1... evaluation function calculation unit, 25-2, 34A-2, 34B-2, 45-2, 54A-2, 54B-2... optimization unit, 26, 46... judgment unit, 35A, 55A... first judgment unit, 35B, 55B... second judgment unit, 27, 36, 47, 56... learned parameters, 28, 37, 48, 57... prediction data, 29, 38, 49, 58... prediction sequence generation unit.

Claims

1. Monotonically increasing neural network, a first calculation unit that calculates a cumulative intensity function based on an output from the monotonically increasing neural network and a product of a parameter and time; An information processing device comprising:

2. A second calculation unit is further provided to calculate an intensity function related to a point process based on the calculated cumulative intensity function.

2. The information processing device according to claim 1.

3. An update unit that updates the parameters based on the calculated intensity function.

3. The information processing device according to claim 2.

4. The neural network further includes a neural network that receives as input all events included in a sequence including a plurality of events arranged discretely in continuous time, or the number of the plurality of events included in the sequence, and outputs the parameter.

2. The information processing device according to claim 1.

5. The method further includes an initialization unit that initializes a plurality of weights applied to the monotonically increasing neural network based on a distribution having a positive average.

2. The information processing device according to claim 1.

6. outputting a monotonically increasing function from the monotonically increasing neural network; Calculating a cumulative intensity function based on the output monotonically increasing function and a product of a parameter and time; An information processing method comprising:

7. A program for causing a computer to function as each unit of the information processing device according to any one of claims 1 to 5.

Images