Behavior generation system and behavior generation method

Abstract

The aim is to leverage the characteristics of the information processing execution process performed by skilled individuals. [Solution] The behavior generation device 1 includes a generator 131 that generates pseudo-model behaviors statistically similar to true model behaviors, with each of the sequence of state transitions of multiple events corresponding to model behaviors being the true model behavior, and a discriminator 132 that distinguishes between the pseudo-model behaviors generated by the generator 131 and the true model behavior. It also includes a third learning unit 13 configured to learn multiple generative models, each having an objective function set with the probability distribution of model behaviors and non-model behaviors determined from the ratios estimated by the first learning unit 11, and a second parameter estimated by the second learning unit 12.

Description

[Technical Field]

[0001] The present invention relates to a behavior generation system and a behavior generation method. [Background technology]

[0002] In recent years, with the advancement of artificial intelligence (AI) technology, technologies that support the creation and implementation of programs have been widely researched. In particular, creating source code for AI requires advanced statistical knowledge and algorithmic design skills, which can be a significant burden for beginners and those without expertise.

[0003] For example, Patent Document 1 discloses a technology for evaluating and selecting the next processing or operation to be performed based on the system's state and history information. The system disclosed in Patent Document 1 scores processing by considering past execution history and contextual information. However, the technology described in Patent Document 1 cannot fully utilize past execution history in a way that is useful for subsequent learning and application.

[0004] Furthermore, Patent Document 2 discloses a technology that uses a learning model including reinforcement learning to generate the execution content of information processing or control consisting of multiple operations based on user commands. However, when using reinforcement learning, the learning result depends heavily on the reward design, so it can be difficult to design an appropriate reward when the execution process of information processing is complex.

[0005] Furthermore, even when employing GAIL (Generative Adversarial Imitation Learning), which uses inverse reinforcement learning for rewards, there were cases where learning did not converge, and cases where the desired reward was not obtained. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2025-1568 [Patent Document 2] Patent No. 7595817 [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] Conventional technologies have not fully utilized the unique characteristics of the information processing process performed by skilled individuals.

[0008] This invention was made to solve the above-mentioned problems and aims to utilize the characteristics of the information processing execution process performed by skilled personnel. [Means for solving the problem]

[0009] To solve the above-mentioned problems, the behavior generation device according to the present invention is configured to learn a first parameter that represents the ratio between the probability distribution of exemplary behavior, which is a sequence of state transitions of multiple events corresponding to an exemplary execution process of information processing, and the probability distribution of non-executive behavior, which is a sequence of state transitions of multiple events corresponding to a non-executive execution process of information processing that deviates from the range of exemplary behavior, and to estimate the ratio based on the learned first parameter; and to treat each of the sequences of state transitions of multiple events representing the execution process of information processing as an observed value, and each observed value as a conditionally independent discrete value, and to generate classes corresponding to the exemplary behavior class and the non-executive behavior class based on the frequency of occurrence of each observed value. The system comprises: a second learning unit configured to estimate a second parameter of a rascal probability distribution model; a generator that generates pseudo-exemplary behaviors statistically similar to the true exemplary behavior, with each of the sequence of state transitions of the plurality of events corresponding to the exemplary behavior being the true exemplary behavior; a discriminator that distinguishes between the pseudo-exemplary behaviors generated by the generator and the true exemplary behavior; a third learning unit configured to learn a plurality of generative models, each having an objective function set by the probability distribution of the exemplary behavior and the probability distribution of the non-exemplary behavior determined from the ratio estimated by the first learning unit, and the second parameter estimated by the second learning unit; and a generation unit configured to generate the pseudo-exemplary behaviors using each of the plurality of trained generators obtained by the learning of the third learning unit.

[0010] Furthermore, in the behavior generation device according to the present invention, the generation unit may further include a component configured to construct an entire sequence of state transitions of multiple events based on the pseudo-model behavior generated using each of the multiple trained generators, and to obtain a description of the information processing.

[0011] Furthermore, the behavior generation device according to the present invention may also include a presentation unit configured to present a description of the information processing obtained by the aforementioned components.

[0012] Furthermore, in the behavior generation device according to the present invention, the probability distribution of the exemplary behavior is the probability density function of the exemplary behavior, the probability distribution of the non-exemplary behavior is the probability density function of the non-exemplary behavior, and the ratio may be the density ratio of the probability density function of the exemplary behavior and the probability density function of the non-exemplary behavior.

[0013] To solve the above-mentioned problems, the behavior generation method according to the present invention includes a first learning step in which a first learning step is to learn a first parameter that represents the ratio between the probability distribution of exemplary behavior, which is a sequence of state transitions of multiple events corresponding to an exemplary execution process of the information processing, and the probability distribution of non-executive behavior, which is a sequence of state transitions of multiple events corresponding to a non-executive execution process of the information processing that deviates from the range of exemplary behavior, and estimate the ratio based on the learned first parameter; and a first learning step in which each of the sequences of state transitions of multiple events representing the execution process of the information processing is treated as an observed value, each observed value is treated as a conditionally independent discrete value, and based on the frequency of occurrence of each observed value, the behavior generation method corresponds to the class of exemplary behavior and the class of non-executive behavior, respectively. The system comprises: a second learning step of estimating a second parameter of a class-based probability distribution model; a generator that generates pseudo-exemplary behaviors statistically similar to the true exemplary behavior, with each of the sequences of state transitions of the plurality of events corresponding to the exemplary behavior being the true exemplary behavior; a discriminator that distinguishes between the pseudo-exemplary behaviors generated by the generator and the true exemplary behavior; a third learning step of learning a plurality of generative models, each having an objective function set by the probability distribution of the exemplary behavior and the probability distribution of the non-exemplary behavior determined from the ratio estimated in the first learning step, and the second parameter estimated in the second learning step; and a generation step of generating the pseudo-exemplary behaviors using each of the plurality of trained generators obtained through learning in the third learning step.

[0014] Furthermore, the behavior generation method according to the present invention may further include a configuration step in which, based on the pseudo-model behavior generated using each of the plurality of trained generators, an entire sequence of state transitions of multiple events is constructed, and a description of the information processing is obtained.

[0015] Furthermore, the behavior generation method according to the present invention may further include a presentation step for presenting a description of the information processing obtained in the configuration step. [Effects of the Invention]

[0016] According to the present invention, the system includes a generator that generates pseudo-exemplary behaviors statistically similar to true exemplary behaviors, with each of the sequence of state transitions of multiple events corresponding to exemplary behavior being treated as a true exemplary behavior, and a discriminator that distinguishes between the pseudo-exemplary behaviors generated by the generator and true exemplary behaviors. The system learns multiple generative models, each having an objective function set with the probability distributions of exemplary behaviors and non-exemplary behaviors determined from the ratios estimated by the first learning unit, and a second parameter estimated by the second learning unit. Therefore, it is possible to utilize the characteristics of the information processing execution process performed by experts. [Brief explanation of the drawing]

[0017] [Figure 1] Figure 1 is a block diagram showing the configuration of a behavior generation device according to an embodiment of the present invention. [Figure 2] Figure 2 is a diagram illustrating the overview of the behavior generation device according to this embodiment. [Figure 3] Figure 3 is a diagram illustrating the third learning unit included in the behavior generation device according to this embodiment. [Figure 4] Figure 4 is a diagram illustrating the third learning unit included in the behavior generation device according to this embodiment. [Figure 5] Figure 5 is a diagram illustrating the third learning unit included in the behavior generation device according to this embodiment. [Figure 6]Figure 6 is a block diagram showing an example of the hardware configuration of the behavior generation device according to this embodiment. [Figure 7] Figure 7 is a flowchart showing the operation of the behavior generation device according to this embodiment. [Figure 8] Figure 8 is a flowchart showing the operation of the behavior generation device according to this embodiment. [Figure 9] Figure 9 is a flowchart showing the operation of the behavior generation device according to this embodiment. [Figure 10] Figure 10 is a flowchart showing the operation of the behavior generation device according to this embodiment. [Modes for carrying out the invention]

[0018] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to Figures 1 to 10.

[0019] [Overview of the Behavior Generation Device] First, with reference to Figure 2, an overview of the behavior generation device 1 according to an embodiment of the present invention will be described. The behavior generation device 1 generates behaviors represented by a sequence of state transitions of multiple events, which reflect the characteristics of the execution process of information processing related to source code performed by an expert.

[0020] Figure 2 schematically illustrates the execution process of information processing as a sequence of state transitions of multiple events. In Figure 2, as an example, source code is treated as a finite automaton that takes a sequence of events as input. The model shown in Figure 2 is an example of representing the execution process of information processing as a sequence of events that occur as user operations or system operations. In this embodiment, information processing refers to a series of processes performed on a computer in relation to program execution, including writing, editing, saving, executing, modifying, and re-executing source code.

[0021] In Figure 2, the starting state S corresponds to the state immediately after launching an integrated development environment (IDE) and displaying the source code editing screen. In this state, the system is waiting for user input. In Figure 2, the goal state G corresponds to the state where the source code has been executed successfully and the desired processing result or output has been obtained.

[0022] Each node shown in Figure 2 represents a state during the execution of information processing, and this state corresponds to the processing status immediately after a certain event occurs. Specific examples of states include waiting for execution, editing in progress, source code execution in progress, and error display in progress.

[0023] In Figure 2, each arrow represents an event that triggers a transition from one state to the next. Specific examples of events include clicking the execute button (click), entering the source code editing area with the mouse (mouseenter), inputting data into the source code (keydown, keypress), saving the code (click or keyboard shortcut), continuing the execution of an ongoing process (internal event), receiving an error notification (internal event), clicking the mouse on the error location (click), and re-executing the program (click).

[0024] For example, when the "click the execute button" event occurs, the execution process associated with that event (e.g., the run() or execute() function) is called, and depending on the result of that process, the system transitions to a state such as a successful completion state or an error display state. The branching arrows in Figure 2 represent these state transitions based on differences in processing results.

[0025] The self-loop shown in Figure 2 represents the repeated occurrence of events that transition to the same state. For example, in a source code editing screen, if character input events (keydown, keypress) occur consecutively, or if mouse movement events (mousemove) occur continuously, this is represented as a self-loop to the same state.

[0026] There can be multiple different paths from the starting state S to the goal state G. For example, when an experienced programmer executes source code, they often click the execute button after minimal editing and select a path that reaches the goal state G without generating errors. On the other hand, when a novice executes the code, an error occurs after clicking the execute button, and operations such as clicking the error location, inputting corrections, saving, and re-executing are repeated, and a path that reaches the goal state G via multiple branches or self-loops may be selected.

[0027] For example, in Figure 2, node "1" can represent the editing state, node "1-1" can represent the state during editing, and node "1-1-1" can represent the state immediately after a key input. In this case, when a key input event (keydown) occurs, the state transitions from node "1-1" to node "1-1-1", and this transition is represented by the arrow in Figure 2.

[0028] A single path from the starting state S to the goal state G corresponds to a single accepting word in an automaton, and in this embodiment, such a path is treated as a "sequence of event state transitions." Among the paths from the starting state S to the goal state G shown in Figure 2, a sequence of event state transitions corresponding to a path that is deemed desirable in terms of efficiency, stability, or minimizing unnecessary transitions, in light of pre-set criteria, is defined as a "model behavior, which is a sequence of multiple event state transitions corresponding to an exemplary information processing execution process." For example, the sequence of event state transitions corresponding to the information processing execution process by the aforementioned expert is treated as an example of model behavior.

[0029] Among the paths shown in Figure 2, a sequence of event state transitions corresponding to a path that contains many branches or self-loops, or a path that contains redundant transitions, compared to a path corresponding to exemplary behavior, is called "a non-exemplary behavior," which is a sequence of event state transitions corresponding to an execution process of non-exemplary information processing that deviates from the scope of exemplary behavior.

[0030] Here, s trepresents the state of information processing at time t, and e t is the state s t from state s t+1 to state s t represents an event that causes a transition. Due to the occurrence of event e t state s t+1 transitions to the next state s t+1 = δ(s t , e t ). The series of state transitions of events from the start state s0 (the start state S in Figure 2) to the goal state s T (the goal state G in Figure 2) can be represented as (s0, e0, s1, e1, s2,..., e T-1 , s T ). Alternatively, focusing on events, it can be represented as E = (e0, e1,..., e T-1 ), or focusing on states, it can be represented as S = (s0, s1,..., s T ).

[0031] For example, the series of state transitions of events in the process of executing source code is represented as follows. s0: Editing screen display state e0: Click on the execute button s1: Source code execution state e1: Error occurrence notification (internal event) s2: Error display state e2: Click on the error location s3: Editing operation state e3: Input correction (keydown) s4: Editing completion state e4: Click on the re - execute button s5: Normal termination state

[0032] This sequence can be represented as (s0, e0, s1, e1, s2, e2, s3, e3, s4, e4, s5). The execution process focusing on events is represented as E=(e0, e1, e2, e3, e4). The exemplary behavior corresponding to the execution process by an expert is represented as a sequence of state transitions defined by events, for example, E=(e0, e4). In exemplary behavior, there are no error transitions, and the branching has few self-loops. On the other hand, the non-exemplary behavior corresponding to the execution process by a novice is represented as a sequence of state transitions defined by events, for example, E=(e0, e1, e2, e3, e4). The execution process by a novice is represented as a sequence of state transitions of redundant events, including error transitions and corrective operations.

[0033] [Functional blocks of the behavior generation device] Next, the functional blocks of the behavior generation device 1 according to this embodiment will be described with reference to the block diagram in Figure 1. As shown in Figure 1, the behavior generation device 1 comprises an acquisition unit 10, a first learning unit 11, a second learning unit 12, a third learning unit 13, a generation unit 14, a component unit 15, a presentation unit 16, and a storage unit 17.

[0034] The acquisition unit 10 acquires a sequence of state transitions of multiple events that represent the execution process of information processing. More specifically, the acquisition unit 10 acquires a sequence of state transitions of multiple events corresponding to an exemplary execution process of information processing as exemplary behavior. The acquisition unit 10 also acquires a sequence of state transitions of multiple events corresponding to an unexemplary information processing execution process that deviates from the scope of exemplary behavior as unexemplary behavior. The acquisition unit 10 can acquire sequence data that has been pre-classified as exemplary or unexemplary behavior and stored locally based on pre-set criteria. Alternatively, the acquisition unit 10 can acquire sequence data from an external source. The acquisition unit 10 may also automatically determine and classify exemplary and unexemplary behavior based on the number of transitions, number of branches, number of self-loops, or status of reaching the goal state included in the sequence of state transitions, and then acquire the sequence data.

[0035] The first learning unit 11 learns a parameter (first parameter) that represents the ratio between the probability distribution of exemplary behavior, which is a sequence of state transitions of multiple events corresponding to an exemplary execution process of information processing, and the probability distribution of non-exemplary behavior, which is a sequence of state transitions of multiple events corresponding to an non-exemplary execution process of information processing that deviates from the range of exemplary behavior, and estimates the ratio based on the learned parameter.

[0036] The first learning unit 11 learns parameters representing the density ratio (ratio) of the probability density functions, which are the probability distributions of exemplary and non-exemplary behaviors, using sequential data of exemplary and non-exemplary behaviors. The first learning unit 11 also estimates the density ratio from the learned parameters. In the following, we will focus particularly on observable events and treat the execution process of information processing as a sequence of event state transitions. The ratio may include, for example, the density ratio itself, as well as logarithmic values, normalized values, monotonically transformed values ​​based on the density ratio, or scores representing the relative magnitudes of the probability distributions. The ratio may also be a parameterized value obtained as the output of the learning model.

[0037] Here, the set of training data related to exemplary behavior is D = {x (1) ,x (2) ,…,x (N)}, the set of training data related to non-exemplary behavior is D'={x' (1) ,x' (2) ,…,x' (N’) Let x be defined as an observed data that focuses on events corresponding to operations or processes that occur during the execution of information processing, and expresses the state transitions defined by these events as features. Each observed data x (n) It is M-dimensional, and x (n) =(x1 (n) , x2 (n) ,…,x M (n) ) is expressed as follows. When assuming feature quantities corresponding to M events in the execution process of information processing, Each observational data x (n) It is represented as an M-dimensional vector. Each component x i (n)This indicates an observed value corresponding to the i-th transition position in the sequence of event state transitions, or an observed value corresponding to the i-th event type. This observed value includes whether or not the event occurred, its frequency, or its intensity.

[0038] In the following, D is defined as exemplary behavior and D' as non-exemplary behavior. Let p(x) be the probability density function of exemplary behavior D, and p'(x) be the probability density function of non-exemplary behavior D'. The probability density function p(x) of exemplary behavior D represents the probability distribution of observed data x, which is represented as a feature vector of the state transition sequence of events corresponding to exemplary behavior. The probability density function of non-exemplary behavior D' represents the probability distribution of observed data x, which is represented as a feature vector of the state transition sequence of events corresponding to non-exemplary behavior. The probability density ratio r(x) between the probability density function p(x) of exemplary behavior D and the probability density function p'(x) of non-exemplary behavior D' is given by equation (1).

number

[0039]

number

[0040] The basis function ψ(x) is defined using the RBF (Radial Basis Function) kernel and expressed by the following equation (3).

number

[0041] Here, based on equation (1) above, the specific form of equation (3) above, where the number of basis vectors b is the number of training data N (b=N), is given by the following equation (4).

number

[0042] Equation (4) above expresses the density ratio as a linear sum of RBFs centered on all learning points. Here, we introduce the generalized Kullback-Leibler divergence, which measures the information-theoretic distance between the non-negative functions f and g shown in equation (5).

number

[0043] In density ratio estimation, f=p(x) and g=r θ Substitute p'(x) into equation (5) above, and the objective function is given by equation (6).

number

[0044] In equation (6) above, each x n , x' n’ The optimization objective function obtained by approximating the integral with an empirical distribution that sets all values ​​except θ to 0, ignoring terms that do not depend on the parameter θ, and removing constants is expressed by the following equation (7).

number

[0045] By minimizing J(θ) in equation (7) above, the density ratio r θ This is estimated. Since J(θ) is a convex function, the first learning unit 11 updates the parameter θ from the initial value to convergence using the parameter θ update formula by gradient descent shown in equation (8) below.

number

[0046] The result of specifically calculating the gradient in equation (7) above is expressed by the following equation (9).

number

[0047] In equation (9) above, the first term represents the contribution from non-exemplary behavior, and the second term represents the contribution from exemplary behavior. Since the second term is dominant in equation (9), stable estimation is possible when the number of exemplary behavior data N is large. In other words, even when the number of non-exemplary behavior data N' is small, it is possible to stably determine the parameter θ.

[0048] Furthermore, before using equations (7) to (9) above to find the optimal solution for parameter θ, the first learning unit 11 finds an appropriate value for the bandwidth h in equation (4) above using cross-validation or an information criterion (KL divergence minimization criterion). Based on the optimal solution for parameter θ obtained by the KL density ratio estimation method, the first learning unit 11 uses equation (2) above to determine the density ratio r for any input x. θ An estimated value can be obtained. The density ratio estimated by the first learning unit 11 is passed to the third learning unit 13, which will be described later.

[0049] The second learning unit 12 treats each of the sequence of state transitions of multiple events as an observed value in the execution process of information processing, and treats each observed value as a conditionally independent discrete value. Based on the frequency of occurrence of each observed value, it estimates the parameters (second parameters) of the class-specific probability distribution model corresponding to the exemplary behavior class and the non-exemplary behavior class, respectively. The second learning unit 12 sets up a class-specific probability distribution model using a multinomial distribution for the data corresponding to the state transitions of each event related to exemplary behavior and the data corresponding to the state transitions of each event related to non-exemplary behavior, estimates its parameters, and calculates the log-likelihood of the sequence of observed values ​​based on the class-specific probability distribution model. The second learning unit 12 uses integer discrete values ​​as observed values ​​for each state transition of an event in the state transitions of multiple events.

[0050] The second learning unit 12 focuses on the fact that the simple Bayes method treats each observation as a conditionally independent discrete variable, and extends the simple Bayes method to a multinomial distribution model to set up a class-based probability distribution model. Here, as shown in equation (10) below, the observed value X is a vector representing the number of occurrences of L categories, which corresponds to the state transitions of each event. X = x = (x1, x2, ..., x L ) ···(10)

[0051] Furthermore, the class label to which the observed value X belongs, i.e., the event Y, is defined as a binary variable by the following equation (11).

number

[0052] If the conditional probabilities P(X|Y) for the occurrence of observed value X under the given conditions of event Y are independent, then the observed variable x under the given conditions of event Y is... i The variables are conditionally independent of each other, and the likelihood ρ(x|y) is given by the simple Bayesian estimation method in equation (12).

[0053]

number

[0054] In this embodiment, since we are dealing with a sequence of data for each state transition of an integer-value event, instead of a single observation value, we are dealing with a category-specific occurrence count vector x=(x1,x2,···,x L We consider ) as one sample and extend it to a multinomial distribution probability model while maintaining the naive assumption that the state transition data for each event are conditionally independent. The occurrence probabilities θ1,···,θ for each marginal probability P(X) in which the observed value X is observed are given. M Under the constraints that the variables are independent and the sum of their probabilities of occurrence is 1, the probability model of the multinomial distribution can be expressed by the following equation (13).

[0055]

number

[0056] As shown in equation (13) above, the observed value x is the count value of the data corresponding to the state transition of each event. i The sum of (x1 + x2 + ... + x L Once the state transition of the i-th event in the sequence is determined, the distribution becomes the product of equation (14), and the observed value x corresponds to the state transition of the i-th event in the sequence. i Each with an independent probability θ i xi It is possible to find this.

number

[0057] Therefore, it can be seen that the relationship is similar to that of the simple Bayesian method in (12). Here, the prior probability P(Y) is a binary problem of a class of exemplary behavior (Y=0) and a class of non-exemplary behavior (Y=1), and the unknown parameters of each are θ. 0 ,θ 1 Let's define D as the marginal probability (result), and the non-model (Y=0) observation value as D. 0 The observed value of the model (Y=1) is D 1 Let's assume that equation (14) above can be expressed as equation (15) by transforming the product form into a sum under the Naive Bayes independence assumption and decomposing the log-likelihood for the data set, the set of observed values ​​D, into class-specific forms.

[0058]

number

[0059] Here, the constraint is expressed by equation (16).

number

[0060] Furthermore, applying the Lagrange multiplier method, the parameter θ related to the class of non-exemplary behavior is obtained. 0 The maximum value of the log-likelihood for the i-th component is given by equation (17).

number

[0061] Parameter θ related to the class of exemplary behavior 1 Similarly, when we find the maximum value of the log-likelihood for this as well, under the constraint of equation (16) above, the parameter θ corresponds to the class of exemplary behavior and the class of non-exemplary behavior, respectively. 1 , θ 0 This can be expressed by the following equation (18).

number

[0062] Equation (18) above shows that the parameters of the multinomial distribution model are estimated based on the frequency of occurrence of each component of the feature vector included in the training data belonging to each class, aggregated by category. i Since x takes an integer value, i (n) To prevent the multinomial distribution from diverging when θ is 0, smoothing can be performed by specifying +α (e.g., α=1) smoothing. In this way, the second learning unit 12 considers the sequence of observed data corresponding to the state transitions of multiple events to follow a multinomial distribution, estimates the parameters of the class-based probability distribution model by counting the frequency of occurrence of each category, and evaluates the degree of exemplary and non-exemplary behavior using the likelihood based on the model. The parameters θ estimated by the second learning unit 12 1 , θ 0 It will be handed over to the 3rd Learning Department, 13.

[0063] The third learning unit 13 includes a generator 131 that generates pseudo-exemplary behaviors statistically similar to true exemplary behaviors, with each of the sequence of state transitions of multiple events corresponding to exemplary behavior being treated as a true exemplary behavior, and a discriminator 132 that distinguishes between the pseudo-exemplary behaviors generated by the generator 131 and true exemplary behaviors. It learns multiple generative models, each having an objective function set with the probability distribution of exemplary behaviors and the probability distribution of non-exemplary behaviors determined from the density ratio estimated by the first learning unit 11, and parameters (second parameters) estimated by the second learning unit 12.

[0064] As shown in Figure 3, the third learning unit 13 performs adversarial learning on a Generative Adversarial Network (GAN) having a generator 131 and a discriminator 132. As shown in Figure 3, the third learning unit 13 performs adversarial learning on each of the multiple generative adversarial networks (GANs) provided for each observation value i (i=1,...,M) corresponding to the state transition of an event. Through the learning of the third learning unit 13, a total of M trained generators 131' are constructed, each corresponding to an observation value 1 to M that corresponds to the state transition of each event.

[0065] Figures 4 and 5 schematically represent the neural network configuration of the GAN generator 131 and discriminator 132 used by the first learning unit 11. As shown in Figure 4, the generator 131 consists of a neural network having an input layer, a hidden layer, and an output layer. The generator 131 is a model that generates pseudo-exemplary behavior from random noise. For example, m randomly sampled Gaussian noise vectors are input to the input nodes of the generator 131 (z1~z m ).

[0066] The generator 131 outputs an output G(z) after performing a sum-of-products operation on the input and weight parameters, followed by thresholding using an activation function. The output G(z) from the generator 131 is a pseudo-exemplary behavior that is similar to the observed value corresponding to the state transition of the i-th event corresponding to the exemplary behavior. A CNN or ResNet can be used as the neural network that constitutes the generator 131.

[0067] The classifier 132 shown in Figure 5 consists of a neural network having an input layer, a hidden layer, and an output layer. In the example in Figure 5, the training data input is an observed value x corresponding to the state transition of the i-th event corresponding to the exemplary behavior, which is acquired by the acquisition unit 10.

[0068] The classifier 132 outputs a probability value in the range of 1 to 0 after performing a sum-of-products operation on the input and weight parameters and thresholding using an activation function. When the classifier 132 correctly identifies the input training data relating to true exemplary behavior as true exemplary behavior, it outputs a probability value close to y=1. On the other hand, when it correctly identifies the input training data relating to pseudo-exemplary behavior as pseudo-exemplary behavior, it outputs a probability value close to y=0. In this way, the classifier 132 is a model that distinguishes the model distribution generated by the generator 131 from the data distribution of the training data, which is the true distribution. A CNN can be used as the neural network that constitutes the classifier 132.

[0069] Figure 3 is a block diagram illustrating adversarial learning of the GAN by the third learning unit 13. The GAN generator 131 adopted by the first learning unit 11 is denoted as function G, and the discriminator 132 as function D. The true model behavior is denoted as x, the predicted value output by the discriminator 132 is denoted as y, and the correct label is denoted as t. The correct label t is set to 1 for the true model behavior and 0 for the pseudo-model behavior generated by the generator 131. In this case, the discriminator 132 is a binary classification problem with the cross-entropy E given by equation (19) below. CE It can be expressed as follows.

[0070]

number

[0071] The first term inside the brace in equation (19) above represents t n lny n In this case, the predicted value y of the classifier 132 n However, the correct label for true exemplary behavior is t nIt is desirable to approach the value of = 1. On the other hand, in the second term within the braces, (1 - t n )ln(1 - y n ), for the predicted value y n of the discriminator 132, it is desirable to approach the value of the correct label that discriminates the pseudo-reference behavior as (1 - t n ) = 0. Thus, the cross-entropy E CE becomes the maximum value when the predicted value matches the value of the correct label.

[0072] Here, the generator 131 that constitutes the GAN has parameters w G , θ G and is represented by the function G(w G , θ G ). Also, the discriminator 132 has parameters w D , θ D and is represented by the function D(w D , θ D ). The objective function E of the GAN equipped with the generator 131 and the discriminator 132 based on the cross-entropy E CE in the above equation (19) can be represented by the following equation (20).

Equation

[0073] The E D(x)=1 lnD(w D , θ D ) represented by the first term in the above equation (20) is the expected value that the discriminator 132 discriminates the true reference behavior as the true reference behavior. The E D(x)=0 ln(1 - D(G(w G , θ G ), w D , θ D )) represented by the second term in the above equation (20) is the expected value that the discriminator 132 discriminates the pseudo-reference behavior generated by the generator 131 as the pseudo-reference behavior. Here, the expected value in the above equation (20) is expressed as the following equation (21) when using the probability distribution.

[0074]

Equation

[0075] Here, for the probability density function p(x) of exemplary behavior and the probability density function p'(x) of non-exemplary behavior in equation (1) above, we set p(x) ≡ ρ(x|y=1) and p'(x) ≡ ρ(x|y=0) in order to make them into a probabilistic labeled classification problem for min-max learning of GANs. The density ratio r estimated by the first learning unit 11 θ (x) is defined by the following equation (22).

number

[0076] The probability density function ρ(x|y=1), which is the conditional probability distribution of exemplary behavior in equation (22) above, can be calculated using maximum likelihood estimation or the like from M-dimensional state transition data of multiple events of exemplary behavior that have been stored in the device beforehand. Using the calculated probability density function ρ(x|y=1) of exemplary behavior, the probability density function ρ(x|y=0), which is the conditional probability distribution of non-exemplary behavior, can be expressed by the following equation (23).

number

[0077] Substitute the probability density function ρ(x|y=1) for exemplary behavior and the probability density function ρ(x|y=0) for non-exemplary behavior from equation (23) into the objective function E in equation (21), and set the prior probability ρ(y=1) for the class of exemplary behavior (y=1) and the prior probability ρ(y=0) for the class of non-exemplary behavior (y=0) to the parameter θ corresponding to the observed value x corresponding to the state transition of the i-th event corresponding to the class of exemplary behavior and the class of non-exemplary behavior estimated by the second learning unit 12. 1 , θ 0 We use the following. Furthermore, the posterior probability ρ(y=1|x) which is unique to the observed value x corresponding to the state transition of the i-th event is given by D(w) in the optimal solution of equation (22) above. D ,θ Dcorresponds to (0), and the posterior probability ρ(y = 0|x) of non-normative behavior with respect to the observed value x corresponding to the state transition of the i-th event is 1 - D(G(w G , θ G ), w D , θ D ) in the optimal solution of the above formula (21). Each posterior probability can be obtained from the density ratio r θ and the prior probability.

[0078] In the learning of the GAN, the generator 131 and the discriminator 132 are adversarially learned by the min-max optimization of the objective function E. Therefore, the generator 131 is learned so as to generate pseudo-normative behavior that can deceive the discriminator 132, and the discriminator 132 is learned so as to identify the pseudo-normative behavior generated by the generator 131 as pseudo-normative behavior.

[0079] In the learning of the discriminator 132, when the true normative behavior is given, the discriminator 132 outputs an output close to y = 1 to maximize the first term of the objective function E of the above formula (21). On the other hand, when the pseudo-normative behavior is given, the learning is performed so as to maximize the second term of the objective function E by the discriminator 132 outputting an output close to y = 0.

[0080] In the learning of the generator 131,(D(G(w G , θ G ), w D , θ D )(D(G(z)) in FIG. 3) of the above formula (21) outputs G(w G , θ G )(G(z) in FIG. 3) close to 1 to minimize the objective function E. The third learning unit 13 uses a learning procedure that alternately updates the parameters of the generator 131 and the parameters of the discriminator 132. The details of the learning procedures of the generator 131 and the discriminator 132 by the third learning unit 13 will be described later.

[0081] When the objective function E of the GAN is optimized, the third learning unit 13 passes the learned generator 131' constructed for each observed value i corresponding to the state transition of the event to the generation unit 14. Therefore, finally, M learned generators 131' are constructed.

[0082] The generation unit 14 generates pseudo-exemplary behaviors using each of the trained generators 131' obtained through training by the third learning unit 13. Therefore, the generation unit 14 generates M sets of pseudo-exemplary behaviors.

[0083] The component 15 constructs an entire sequence of state transitions for multiple events corresponding to information processing, based on the pseudo-exemplary behavior generated by the generation unit 14 using each of the multiple trained generators 131', and obtains a description of the information processing. The description of the information processing includes, for example, source code, scripts, configuration files, or equivalent processing descriptions.

[0084] Component 15 can, for example, use a Hidden Markov Model (HMM) and estimate the parameters of the state transition probability distribution and the event output probability distribution using the Baum-Welch algorithm, which is a type of EM algorithm, to construct the entire sequence of event state transitions.

[0085] In this case, component 15 uses the simulated exemplary behavior as an observation sequence and executes the following Baum-Welch algorithm learning steps from the first to the fifth step to estimate the state transition probability distribution and the event output probability distribution. The state transition probability distribution is a probability distribution that defines the probability of transitioning from one state to the next for each set of states, and represents the trend of state transitions in the execution process of information processing. In this embodiment, the symbol output probability distribution in the hidden Markov model corresponds to a distribution that represents the probability of events occurring observed in the execution process of information processing.

[0086] [Step 1] First, component 15 sets the initial values ​​for the state transition probability distribution and the event output probability. Component 15 can use any value as the initial value.

[0087] [Step 2] Next, component 15 analyzes the observed event sequence O based on the sequence of state transitions of the event to be learned.t ={O1,O2,…,O m Determine}. Here, each event O t This represents the type of event that occurred during the execution of information processing (e.g., click, input, save, execute, error notification, etc.).

[0088] Specifically, component 15 refers to a sequence of event state transitions corresponding to a pseudo-exemplary action generated by the generation unit 14 or an exemplary action collected in advance, and acquires the event that occurred at each time t as an observed symbol. Furthermore, component 15 identifies a state S corresponding to the processing status immediately after the event occurred. i (For example, edit state, run state, error display state, etc.) are mapped as hidden states.

[0089] [Step 3] Next, component 15 sets the value of grid, which is a variable of the Forward algorithm, to α t (i) is defined by the following equation (24), and each α t Perform the calculation.

number

[0090] [Step 4] Next, component 15 sets the value of grid, which is a variable of the backward algorithm, to β t (t) is defined by the following equation (25), and each β t Perform the calculation.

number

[0091] [Step 5] Next, component 15 is in state S at time t. i From state S j Transition probability Γ tCalculate (i,j) and the state transition probability a ij and event output probability b j Recalculate (O).

number

[0092] Subsequently, component 15 repeats steps 3 through 5 described above, and the point at which the parameters do not change or the likelihood no longer changes is taken as the convergence point, and the state transition probability distribution and event output probability distribution at that point are adopted as estimated values. In the Baum-Welch algorithm, the transition probability Γ is determined in step 5. t (i,j) State transition probability a ij and event output probability b j The procedure calculated from (O) corresponds to the expected value (E) step of the EM algorithm. Also, in the fifth step, the event output probability b j (O) Transition probability Γ t The procedure of recalculating from (i,j) corresponds to the maximization (M) step of the EM algorithm.

[0093] The component 15 estimates a sequence of state transitions for multiple events from the starting state to the goal state, based on the estimated state transition probability distribution and event output probability distribution. Furthermore, the component 15 constructs a description of information processing by converting the estimated sequence of event state transitions into processing content associated with each event.

[0094] The presentation unit 16 presents the entire state transitions of multiple events obtained by the component unit 15. The presentation unit 16 can display the entire state transitions of multiple events on, for example, the display device 107. The presentation unit 16 can transmit the entire state transitions of multiple events to an external server as support information to assist processing in other information processing devices.

[0095] The memory unit 17 stores the parameters and density ratios estimated by the first learning unit 11, as well as the parameters estimated by the second learning unit 12. The memory unit 17 also stores M trained generators 131'.

[0096] [Hardware configuration of the behavior generation device] Next, an example of a hardware configuration for realizing the behavior generation device 1 having the functions described above will be explained using Figure 6.

[0097] As shown in Figure 6, the behavior generation device 1 can be realized, for example, by a computer equipped with a processor 102 connected via a bus 101, a main memory 103, a communication interface 104, an auxiliary storage device 105, and an input / output I / O 106, and a program that controls these hardware resources. Furthermore, the behavior generation device 1 includes a display device 107.

[0098] The processor 102 is a circuit or device that performs arithmetic processing, and can be implemented by, for example, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc. Alternatively, some or all of these may be combined to form the processor.

[0099] The main memory 103 is composed of, for example, volatile random access memory (RAM), and pre-stores programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory 103 work together to realize each function of the behavior generation device 1, such as the acquisition unit 10, the first learning unit 11, the second learning unit 12, the third learning unit 13, the generation unit 14, and the configuration unit 15 shown in Figure 1.

[0100] The communication interface 104 is an interface circuit for networking the behavior generation device 1 with various external electronic devices.

[0101] The auxiliary storage device 105 consists of a read / write storage medium and a drive device for reading and writing various information such as programs and data to the storage medium. The auxiliary storage device 105 can use non-volatile storage such as a hard disk or flash memory as the storage medium.

[0102] The auxiliary storage device 105 has a program storage area for storing the communication management program. The auxiliary storage device 105 also has a program storage area for storing parameters representing the density ratio of the probability density functions of exemplary and non-exemplary behaviors executed by the behavior generation device 1, and a first learning program for estimating the density ratio. The auxiliary storage device 105 also has a program storage area for storing a second learning program for estimating parameters using polynomial Bayes, executed by the second learning unit 12. The auxiliary storage device 105 also has a program storage area for storing a third learning program for adversarial learning of the generative model, executed by the third learning unit 13. The auxiliary storage device 105 also has a program storage area for storing a configuration program that executes the Baum-Welch algorithm, executed by the configuration unit 15.

[0103] The auxiliary storage device 105 enables the storage unit 17 described in Figure 1. Furthermore, it may have, for example, a backup area for backing up the aforementioned data and programs.

[0104] The I / O106 is an input / output device that accepts signals from external devices and outputs signals to external devices.

[0105] The display device 107 is composed of an organic EL display, a liquid crystal display, or the like. The display device 107 can realize the presentation unit 16.

[0106] [Operation of the Behavior Generator] Next, the operation of the behavior generation device having the above-described configuration will be explained with reference to the flowcharts in Figures 7 to 10.

[0107] As shown in Figure 7, first, the acquisition unit 10 acquires a sequence of state transitions for multiple events corresponding to exemplary behavior and a sequence of state transitions for multiple events corresponding to non-exemplary behavior from a sequence of state transitions for multiple events representing the execution process of information processing that is stored in advance in the device (step S1).

[0108] Next, the first learning unit 11 performs the first learning process (step S2). Figure 8 is a flowchart illustrating the first learning process in step S2. As shown in Figure 8, the first learning unit 11 calculates the probability density ratio r between the probability density function of the exemplary behavior and the probability density function of the non-exemplary behavior. θ The first learning unit 11 learns the parameter θ that represents (step S20). The first learning unit 11 updates the parameter θ using the gradient descent method or the like with equations (7) to (9) above and finds the optimal solution for the parameter θ.

[0109] Next, the first learning unit 11 calculates the density ratio r from the learned parameter θ obtained in step S20, based on equation (2) above. θ We estimate (step S21).

[0110] Returning to Figure 7, the second learning unit 12 executes the second learning process (step S3). Figure 9 is a flowchart illustrating the second learning process in step S3. As shown in Figure 9, the second learning unit 12 calculates the parameter θ for the observed values ​​corresponding to the state transitions of events related to the exemplary behavior class and the non-exemplary behavior class. 1 , θ 0 The second learning unit 12 sets the following (step S31). Next, the second learning unit 12 defines the log-likelihood according to equation (13) above (step S32). In step S32, the second learning unit 12 changes the objective function for optimization from product form (equation (14)) to sum form. Next, the second learning unit 12 uses the Lagrange multiplier method of equation (17) above to set the parameter θ corresponding to the class of exemplary behavior that maximizes the log-likelihood (equation (15)) under the constraints (equation (16)).1 , and the parameter θ corresponding to the class of non-exemplary behavior 0 Estimate (Equation (18)) (Step S33). Estimated parameter θ 1 , θ 0 The data is stored in the memory unit 17, and the process proceeds to step S4 in Figure 7.

[0111] Next, the third learning unit 13 executes the third learning process (step S4). In step S4, the third learning unit 13 learns multiple generative models, each having an objective function set by the probability density functions of the exemplary behavior and the probability density functions of the non-exemplary behavior determined from the density ratio estimated by the first learning unit 11, and the parameters estimated by the second learning unit 12.

[0112] Figure 10 is a flowchart illustrating the third learning process in step S4. As shown in Figure 10, first, the third learning unit processes the density ratio r estimated in the first learning process in step S2 of Figure 7. θ The probability density function ρ(x|y=1) for exemplary behavior and the probability density function ρ(x|y=0) for non-exemplary behavior (equation (21) above), determined from the above, are set as the objective function E of the GAN (equation (21)) (step S50). More specifically, the third learning unit 13 calculates the probability density function ρ(x|y=1), which is the conditional probability distribution of exemplary behavior, from the data of exemplary behavior acquired by the acquisition unit 10 using the maximum likelihood estimation method or the like. The third learning unit 13 also determines the probability density function ρ(x|y=0), which is the conditional probability distribution of non-exemplary behavior expressed in equation (23) above, from the calculated probability density function ρ(x|y=1) of exemplary behavior.

[0113] Next, the third learning unit 13 calculates the parameters θ, which correspond to the exemplary behavior class and the non-exemplary behavior class, respectively, that were estimated in the second learning process of step S3 in Figure 7. 1 ,θ 0These are set as the prior probability ρ(y=1) for the exemplary and prior probability ρ(y=0) for the non-exemplary in the GAN's objective function E (Equation (22)) (Step S51). Furthermore, in the optimal solution of the objective function E in Equation (21) above, D(w D ,θ D ) becomes the exemplary posterior probability ρ(y=1|x) for the observed value x corresponding to the state transition of the event, and 1-D(G(w G ,θ G ),w D ,θ D ) is the posterior probability ρ(y=0|x) that an observed value x corresponding to the state transition of an event is non-model. These posterior probabilities ρ(y=1|x) and ρ(y=0|x) are given by the estimated density ratio r θ It can also be calculated from the prior probabilities ρ(y=1) and ρ(y=0).

[0114] Next, the third learning unit 13 acquires each of the event state transitions corresponding to the exemplary behavior obtained in step S1 of Figure 7 as the true exemplary behavior (step S52). In step S52, the observed state transition of the event i=1, which corresponds to one of the observed state transitions of the M events corresponding to the exemplary behavior (i=1), is taken as the true exemplary behavior.

[0115] Next, the third learning unit 13 inputs the true model behavior, which is the observed state transition of the event i=1 related to the model behavior acquired in step S52, as training data 134 into the classifier 132, and sets the parameter w of the classifier 132 so that it can distinguish the true model behavior from the true model behavior (y=1). D ,θ D Learn and update (step S53). As shown in the block diagram of the third learning unit 13 in Figure 3, true exemplary behavior is used as training data 134 input when training the classifier 132.

[0116] In step S53, the third learning unit 13 can train the classifier 132 to learn true exemplary behavior using, for example, backpropagation. Step S53 pre-constructs a classifier 132 that can distinguish true exemplary behavior from true exemplary behavior.

[0117] Next, the third learning unit 13 generates Gaussian noise and provides a random vector of the generated Gaussian noise as input to the generator 131 (step S54). Subsequently, the generator 131 uses the input z and weight parameter w based on the given Gaussian noise. G ,θ G A pseudo-exemplary behavior G(z) is generated by performing sum-of-products operations and threshold processing using an activation function (step S55).

[0118] Next, the third learning unit 13 trains the classifier 132. The training of the classifier 132 involves the parameters w of the generator 131. D ,θ D This is done with fixed parameters. First, the third learning unit 13 provides the true model behavior acquired in step S52 as training data 134 to the classifier 132 as input. Then, the third learning unit 13 uses backpropagation or the like to maximize the objective function E in equation (22) above, and the parameters w D ,θ D Update (step S56). Note that the label for training data 134 is set to 1 (true exemplary behavior).

[0119] Next, the third learning unit 13 provides the pseudo-model behavior generated by the generator 131 in step S55 as input to the classifier 132, and processes the parameters w using backpropagation or the like so that the objective function E in equation (21) above is maximized. D ,θ D Update (step S57). That is, in steps S56 and S57, in order to maximize the objective function E in equation (21) above, the first term is D(w D ,θ D )=1 is output, and the second term is D(G(w G ,θ G ),w D ,θ D The optimization is performed so that ) = 0. Note that the training data 134 is assigned the label 0 (pseudo-model behavior).

[0120] The learning of the classifier 132 in steps S56 and S57 corresponds to the dashed arrows in the block diagram of the third learning unit 13 shown in Figure 3, which indicate that the classifier error is calculated in block 135 of the objective function E based on the output 133 from the classifier 132, and then the error is backpropagated to the classifier 132.

[0121] Next, the third learning unit 13 trains the generator 131. During the training of the generator 131, the parameters of the discriminator 132 are fixed. The third learning unit 13 trains the generator 131 so that when random Gaussian noise is supplied to the generator 131, pseudo-exemplary behavior is generated. Specifically, the first learning unit 11 uses methods such as backpropagation to minimize the objective function E in equation (21) above, thereby adjusting the parameters w G ,θ G Update (Step S58).

[0122] The learning in step S58 corresponds to the dashed arrow flow in the block diagram of the third learning unit 13 in Figure 3, which indicates that the error is backpropagated to the generator 131. That is, in step S58, the pseudo-executive behavior generated by the generator 131 in Figure 3 is input to the discriminator 132, the generator error is calculated from its output 133 in block 135 of the objective function E, and then the dashed arrow flow indicates that the error is backpropagated to the generator 131.

[0123] Subsequently, the learning of the discriminator 132 and generator 131 from steps S55 to S58 is repeated until the value of the objective function E reaches a Nash equilibrium and converges (step S59: NO). On the other hand, if the value of the objective function E converges (step S59: YES), the processing from steps S53 to S59 is repeated until the learning of the generator 131 and discriminator 132 is performed using all true exemplary actions (step S60: NO). Note that the processing from steps S52 to S58 and from steps S59 to S60 can be performed in batches.

[0124] Subsequently, if the generator 131 and discriminator 132 have been trained using all true exemplary behaviors (step S60: YES), the third learning unit 13 stores the trained generator 131' corresponding to the observed state transition of event i=1 in the storage unit 17 (step S61). Further adversarial learning is then performed on each of the generative models constructed for each of the observed state transitions of events i=2, ..., M, as shown in connector a (step S61: NO), and after storing M trained generators 131' in the storage unit 17 (step S61: YES), the process moves to step S5 in Figure 7.

[0125] Next, the generation unit 14 generates pseudo-exemplary actions corresponding to the observed state transitions of each event using each of the trained generators 131' constructed by the third learning unit 13 (step S5). Subsequently, the configuration unit 15 constructs the entire sequence of state transitions for multiple events based on each of the pseudo-exemplary actions generated in step S5 (step S6). In step S6, the configuration unit 15 uses the pseudo-exemplary actions as the observation sequence and executes the learning step of the Baum-Welch algorithm to estimate the state transition probability distribution and the event output probability distribution.

[0126] In step S6, the component 15 estimates a sequence of state transitions for multiple events from the starting state to the goal state based on the estimated state transition probability distribution and event output probability distribution. Furthermore, the component 15 constructs a description of information processing by converting the estimated sequence of event state transitions into processing content associated with each event.

[0127] Subsequently, the presentation unit 16 presents a description of the information processing obtained in step S6 (step S7). In step S7, the presentation unit 16 can present a description of the information processing that reflects the characteristics of an exemplary information processing execution process as support information to assist in the design, implementation, or debugging of the information processing.

[0128] As described above, according to the behavior generation system of this embodiment, the probability density function and parameters obtained by the first learning unit 11 and the second learning unit 12 are set as the objective function of the GAN, adversarial learning of the GAN is performed with each of the observed values ​​corresponding to the state transitions of multiple events corresponding to the exemplary behavior as the true exemplary behavior, and pseudo-exemplary behaviors are generated using each of the generated trained generators 131'. Furthermore, descriptive information of information processing is obtained using the pseudo-exemplary behaviors that are similar to the true exemplary behaviors generated by each of the trained generators 131'. Thus, the characteristics of the information processing execution process by an expert can be fully utilized.

[0129] According to the behavior generation device 1 of this embodiment, a generation model is constructed for each observed value corresponding to the state transition of M events, and the objective function for each generation model corresponding to each observed value is set to parameters for exemplary behavior and non-exemplary behavior corresponding to the observed value corresponding to the state transition of the event estimated by the second learning unit 12. Therefore, the learning accuracy of the generation model can be improved.

[0130] In the embodiment described, component 15 uses a Hidden Markov Model (HMM) and estimates the parameters of the state transition probability distribution and the event output probability distribution using the Baum-Welch algorithm, a type of EM algorithm, to construct the entire sequence of event state transitions. However, the method for constructing the entire sequence of event state transitions is not limited to using an HMM. For example, methods such as determining the maximum likelihood sequence based on the estimated probability distribution, generating the sequence by probabilistic sampling, searching for the sequence while evaluating multiple candidates, or sequentially generating the event sequence using a sequence generation model can be used.

[0131] Although embodiments of the behavior generation device and behavior generation method of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications that a person skilled in the art can envision are possible within the scope of the invention described in the claims. [Explanation of Symbols]

[0132] 1...Action generation device, 10...Acquisition unit, 11...First learning unit, 12...Second learning unit, 13...Third learning unit, 14...Generation unit, 15...Configuration unit, 16...Presentation unit, 17...Storage unit, 101...Bus, 102...Processor, 103...Main memory, 104...Communication interface, 105...Auxiliary memory, 106...Input / output I / O, 107...Display device, 131...Generator, 132...Discriminator, 133...Output, 134...Training data, 135...Block of objective function E.

Claims

1. A first learning unit is configured to learn a first parameter that represents the ratio between a probability distribution of exemplary behavior, which is a sequence of state transitions of multiple events corresponding to an exemplary execution process of the information processing, and a probability distribution of non-executive behavior, which is a sequence of state transitions of multiple events corresponding to a non-executive execution process of the information processing that deviates from the range of exemplary behavior, and to estimate the ratio based on the learned first parameter. The second learning unit is configured to perform the aforementioned information processing by treating each of the sequence of state transitions of multiple events as an observed value, treating each observed value as a conditionally independent discrete value, and estimating the second parameter of a class-specific probability distribution model corresponding to the exemplary behavior class and the non-executive behavior class, respectively, based on the frequency of occurrence of each observed value. A third learning unit is configured to learn multiple generative models, each having an objective function set by the probability distribution of the exemplary behavior and the probability distribution of the non-exemplary behavior determined from the ratio estimated by the first learning unit, and the second parameter estimated by the second learning unit. The third learning unit includes a generator that generates pseudo-exemplary behaviors statistically similar to the true exemplary behavior, with each of the sequence of state transitions of the multiple events corresponding to the exemplary behavior being the true exemplary behavior, and a discriminator that distinguishes between the pseudo-exemplary behavior generated by the generator and the true exemplary behavior. A generation unit configured to generate the simulated exemplary behavior using each of the multiple trained generators obtained through learning by the third learning unit, A behavior-generating device equipped with the following features.

2. In the behavior generation device according to claim 1, Furthermore, the generation unit includes a component configured to construct an entire sequence of state transitions for multiple events based on the pseudo-model behavior generated using each of the multiple trained generators, and to obtain a description of the information processing. A behavior generation device characterized by the following features.

3. In the behavior generation device according to claim 2, Furthermore, it includes a presentation unit configured to present a description of the information processing obtained by the aforementioned components. A behavior generation device characterized by the following features.

4. In the behavior generation device according to claim 1, The probability distribution of the exemplary behavior is the probability density function of the exemplary behavior. The probability distribution of the non-exemplary behavior is the probability density function of the non-exemplary behavior, The ratio is the probability density ratio between the probability density function of the exemplary behavior and the probability density function of the non-exemplary behavior. A behavior generation device characterized by the following features.

5. A first learning step involves learning a first parameter that represents the ratio between the probability distribution of exemplary behavior, which is a sequence of state transitions of multiple events corresponding to an exemplary execution process of the information processing, and the probability distribution of non-executive behavior, which is a sequence of state transitions of multiple events corresponding to a non-executive execution process of the information processing that deviates from the range of exemplary behavior, and estimating the ratio based on the learned first parameter. The execution process of the aforementioned information processing is performed by treating each of the sequence of state transitions of multiple events as an observed value, treating each observed value as a conditionally independent discrete value, and estimating the second parameter of the class-specific probability distribution model corresponding to the class of exemplary behavior and the class of non-exemplary behavior, respectively, based on the frequency of occurrence of each observed value. A third learning step involves learning multiple generative models, each having an objective function set by the following parameters: a generator that generates pseudo-exemplary behaviors statistically similar to the true exemplary behavior, with each of the sequence of state transitions of the multiple events corresponding to the exemplary behavior being the true exemplary behavior; and a discriminator that distinguishes between the pseudo-exemplary behaviors generated by the generator and the true exemplary behavior. A generation step is performed to generate the simulated model behavior using each of the multiple trained generators obtained through the learning in the third learning step. A method for generating actions that includes the following features.

6. In the behavior generation method described in claim 5, Furthermore, the generation step includes a configuration step in which, based on the pseudo-model behavior generated using each of the multiple trained generators, the entire sequence of state transitions of multiple events is constructed, and a description of the information processing is obtained. A method for generating behavior characterized by the following features.

7. In the behavior generation method described in claim 6, Furthermore, the system includes a presentation step for presenting the description of the information processing obtained in the configuration step. A method for generating behavior characterized by the following features.

Images