Training device, estimating device, training method, and program

Abstract

This training device includes: an acquisition unit that acquires training data including a past action sequence for each customer, a measure condition related to a measure executed for the customer, and a measure effect related to the measure executed for the customer, the measure effect being composed of a combination of whether or not the customer has performed one or more actions and one or more evaluation values; and a training unit that uses the training data to train an action estimation model that receives input of the action sequence and the measure condition to estimate the measure effect.

Description

Learning device, estimation device, learning method, and program

[0001] The technology of the present disclosure relates to a learning device, an estimation device, a learning method, and a program.

[0002] Companies are accumulating customer behavioral history at various touchpoints and analyzing the customer journey, the process of customer behavior from becoming aware of a service, signing a contract, and continuing to use the service. They are then using the results of this analysis to improve their marketing and customer support measures. For example, they are analyzing the timing at which customers are likely to open an advertisement for a new service based on their behavioral history at touchpoints from app delivery to web access, or analyzing the procedural steps that make customers more likely to need support at a call center or in-store based on their behavioral history at touchpoints from web access to a call center.

[0003] Transformer is known as a technology for analyzing customer behavior processes. When modeling with Transformer, customer behavior history is first represented as a table consisting of records containing at least three items: customer, time, and behavior label representing the behavior content. Then, a series of behavior labels arranged in chronological order for each customer is prepared as training data. This training data is loaded into Transformer, and model parameters are estimated using self-supervised learning. In self-supervised learning, the behavior label for the next time or a series in which randomly selected behavior labels are masked is input into Transformer, and the task of assigning the masked behavior label from the output using multi-class classification is repeatedly performed. BERT4Rec (Non-Patent Document 1) is an example of analyzing customer behavior history with Transformer in this way and using it for recommendations.

[0004] Fei Sun, et al. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. CIKM, 2019. https: / / arxiv.org / abs / 1904.06690

[0005] Generally, in marketing strategies, evaluation figures such as sales and profit margins are used as KPIs (Key Performance Indicators), and strategies must be designed to maximize these KPIs. The relationship between strategy conditions, the customer behaviors promoted by the strategy conditions, and the KPIs is a causal relationship of strategy conditions → customer behavior → KPI. Finding the strategy conditions that maximize KPIs and understanding the customer behaviors promoted as a result can improve the effectiveness and efficiency of marketing strategies. However, while conventional technology can make fragmented predictions such as strategy conditions → KPIs or strategy conditions → customer behaviors, it cannot predict strategy conditions → customer behavior → KPIs in a seamless manner. Specifically, BERT4Rec realizes recommendations using a Transformer encoder, which inputs a behavior sequence with a masked future period and outputs a behavior sequence with the masked portions estimated and filled in. This technology alone can predict what behaviors are likely to occur in the future, but it is difficult to predict future behaviors that may occur in parallel with evaluation figures such as sales figures as the effects of measures, or to predict future behaviors that may occur as a result of sales being at their highest using this technology alone.

[0006] Furthermore, it was difficult to flexibly design the conditions assumed as policy conditions according to the behavioral sequence using this technology alone. To achieve this, BERT4Rec and a separate machine learning model designed for each policy effect and policy condition were required, making it difficult to achieve simply.

[0007] As such, with conventional technologies, it was difficult to predict the entire process from marketing policy conditions → customer behavior prompted by the policy → KPIs, and it was also difficult to flexibly design policy conditions. Attempting to achieve this by combining conventional technologies resulted in high computational costs.

[0008] Specifically, when a campaign triggers multiple KPIs or behavioral promotions that are expected to have a cascading effect, the computational cost of verifying the effectiveness is high. For example, when it is necessary to predict the probability that a customer will open an ad after an ad distribution campaign, the products that a customer is likely to purchase in the future, or the customer's LTV (Life Time Value), individual models are used for learning and inference for each. As a result, it was not possible to concisely predict the continuous behavioral process from opening an ad, purchasing a product, and increasing LTV in a single, consistent manner.

[0009] Furthermore, when there were multiple options for measures, the computational cost of verifying effectiveness was high. For example, when trying to encourage customers to sign up for a new service, there were options for the channel (email, app, letter, outbound telemarketing, etc.), timing (morning, afternoon, evening, specific behavioral triggers, etc.), and creative (appealing to empathy, appealing to the superiority of a feature, etc.), and it was not possible to make a concise prediction for all of these options using a single model.

[0010] The disclosed technology has been made in consideration of the above points, and aims to provide a learning device, estimation device, learning method, and program that can reduce calculation costs and estimate both the presence or absence of customer behavior and evaluation values ​​as the effects of a policy.

[0011] A first aspect of the present disclosure is a learning device that includes: an acquisition unit that acquires learning data including a past behavioral sequence for each customer, policy conditions related to policies implemented for the customer, and policy effects related to policies implemented for the customer, the policy effects being configured by a combination of the presence or absence of one or more behavioral details of the customer and one or more evaluation numerical values; and a learning unit that, based on the learning data, uses the behavioral sequence and the policy conditions as inputs and learns a behavior estimation model that estimates the policy effects.

[0012] A second aspect of the present disclosure is an estimation device that includes: an acquisition unit that acquires a customer's past behavioral sequence and policy conditions related to a policy to be implemented on the customer; and an estimation unit that, when the acquired behavioral sequence and policy conditions are input, estimates a policy effect related to a policy to be implemented on the customer using a trained behavior estimation model that estimates a policy effect when the acquired behavioral sequence and policy conditions are input, the policy effect is related to the policy to be implemented on the customer and is configured by a combination of the presence or absence of one or more behavioral contents of the customer and one or more evaluation numerical values.

[0013] A third aspect of the present disclosure is a learning method, which acquires learning data including a past behavioral sequence for each customer, policy conditions related to policies implemented for the customer, and policy effects related to policies implemented for the customer, the policy effects being configured by a combination of the presence or absence of one or more behavioral details of the customer and one or more evaluation values, and learns a behavior estimation model that estimates the policy effects based on the learning data, using the behavioral sequence and the policy conditions as inputs.

[0014] A fourth aspect of the present disclosure is a program for causing a computer to function as the learning device of the first aspect or the estimation device of the second aspect.

[0015] According to the disclosed technology, it is possible to reduce calculation costs and estimate both the presence or absence of customer behavior and the evaluation value as the effect of a policy.

[0016] 1 is a schematic block diagram of an example of a computer that functions as a learning device and an estimation device of this embodiment. FIG. 2 is a diagram showing an example of a past behavior sequence. FIG. 3 is a diagram showing an example of policy conditions related to policies implemented on customers, and policy effects related to policies implemented on customers. FIG. 4 is a block diagram showing the functional configuration of the learning device of this embodiment. FIG. 5 is a diagram showing the configuration of a behavior estimation model. FIG. 6 is an image diagram of the behavior estimation model used in the example of FIG. 3. FIG. 7 is a diagram showing an example of layer design of the behavior estimation model. FIG. 8 is a block diagram showing the functional configuration of the estimation device of this embodiment. FIG. 9 is a flowchart showing the flow of learning processing of this embodiment. FIG. 10 is a flowchart showing the flow of estimation processing of this embodiment.

[0017] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that the same or equivalent components and parts in each drawing are given the same reference numerals. Also, the dimensional proportions in the drawings are exaggerated for the convenience of explanation and may differ from the actual proportions.

[0018] <Configuration of Learning Device According to This Embodiment> FIG. 1 is a block diagram showing the hardware configuration of a learning device 10 according to this embodiment.

[0019] 1, the learning device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0020] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads programs from the ROM 12 or the storage 14 and executes the programs using the RAM 13 as a work area. The CPU 11 controls the above components and performs various arithmetic processing in accordance with the programs stored in the ROM 12 or the storage 14. In this embodiment, a learning program is stored in the ROM 12 or the storage 14. The learning program may be a single program, or a group of programs consisting of multiple programs or modules.

[0021] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0022] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to input various types of data, including learning data such as past behavioral patterns for each customer, policy conditions related to policies implemented for the customer, and policy effects related to policies implemented for the customer, which are policy effects configured by combining the presence or absence of one or more customer behaviors with one or more evaluation values. The policy conditions are expressed by any one of channel, timing, and creative, or a combination of channel, timing, and creative.

[0023] For example, as shown in FIG. 2, a past behavior sequence manually prepared for each customer is input to the input unit 15. FIG. 2 shows an example in which a past behavior sequence is configured with records including at least three items: customer, time, and behavior label. In this example, the first record indicates that the behavior of "Web browsing - pricing plan explanation site" was observed for customer A at 11:32 on January 24, 2023. The example also shows that the behavior of "Call center inquiry - confirmation of this month's payment amount" was observed for customer A at 11:40 on January 24, 2023, for customer B at 12:00 on January 24, 2023, and for customer C at 10:44 on January 24, 2023.

[0024] Furthermore, as shown in FIG. 3, the input unit 15 receives input of policy conditions related to the policies implemented for each customer, which are manually prepared for each customer, and the policy effects related to the policies implemented for the customer. FIG. 3 shows an example in which the policy conditions are whether to deliver the advertisement this time, next time, or not. An example is shown in which the presence or absence of behavioral content of the policy effect is whether or not there was a response one week after delivery, and whether or not there was a response one month after delivery. Here, the presence or absence of a response one week after delivery indicates whether or not there was a response to the advertisement within one week from the starting date, and the presence or absence of a response one month after delivery indicates whether or not there was a response to the advertisement within one month from the starting date. Also shown is an example in which the evaluation value of the policy effect is sales.

[0025] This example shows that ad delivery and response to the ad were observed at different dates and times for each customer. In the example on the left, delivery was made on the most recent date of 7 / 14, but not on the next delivery opportunity of 7 / 24, so the action condition = "Delivered this time." In the example in the center, delivery was not made on the most recent date of 7 / 14, but on the next delivery opportunity of 7 / 24, so the action condition = "Delivered next time." In the example on the right, delivery was not made on either the most recent date of 7 / 14 or the next delivery opportunity of 7 / 24, so the action condition = "No delivery."

[0026] The display unit 16 is, for example, a liquid crystal display, and displays various information including the processing results. The display unit 16 may be a touch panel type and function as the input unit 15.

[0027] The communication interface 17 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0028] Next, a description will be given of the functional configuration of the learning device 10. Fig. 4 is a block diagram showing an example of the functional configuration of the learning device 10.

[0029] As shown in FIG. 4, the learning device 10 functionally comprises an acquisition unit 20, a learning unit 22, and a model storage unit 24.

[0030] The acquisition unit 20 acquires the input learning data.

[0031] The learning unit 22 uses a plurality of learning data sets, inputs a behavioral sequence and a policy condition, and learns a behavior estimation model that estimates the policy effect.

[0032] Specifically, as shown in FIG. 5, the behavior estimation model 100 has a Transformer encoder 100A and a decoder 100B. The encoder 100A receives an action sequence for an arbitrary period as input, and the decoder 100B receives the output of the encoder 100A and the policy conditions as input, and outputs the policy effects.

[0033] The behavior inference model 100 is not limited to a Transformer, but may be any model that can accept past behavior sequences and policy conditions and output future policy effects, such as a multi-layer neural network or gradient boosting.

[0034] Fig. 6 shows an image diagram of the behavior estimation model 100 used in the example of Fig. 3. The campaign conditions are set to select from three options regarding the timing of advertisement distribution: no distribution, current distribution, and next distribution, and the campaign effects are set to three options: response one week after distribution, response one month after distribution, and sales.

[0035] 7 shows an example of the layer design of the behavior inference model 100. The encoder 100A includes a Self-Attension layer, a Layer Norm layer, a Feed Forward layer, and a Layer Norm layer. The decoder 100B includes a Self-Attension layer, a Layer Norm layer, a Cross-Attension layer, a Layer Norm layer, a Feed Forward layer, and a Layer Norm layer.

[0036] The learning unit 22 learns parameters relating to the part of the behavior estimation model that receives behavior sequences as input by self-supervised learning using the behavior sequences.

[0037] For example, in self-supervised learning for the encoder 100A, the behavioral sequence input to the encoder 100A is masked, and the parameters of the encoder 100A are repeatedly updated so that the masked portions can be output from the encoder 100A, thereby further improving the accuracy of the policy effect estimation.

[0038] Furthermore, in the self-supervised learning of the encoder 100A, learning is performed so that the result obtained through the Output layer for the output of the encoder 100A when a masked behavior sequence is input matches the output when the same behavior sequence is input without being masked.

[0039] The learning of the behavior inference model 100 in the learning unit 22 is performed in two stages: in the first stage, self-supervised learning of the encoder 100A is performed using only the behavior sequences, and in the second stage, learning of the entire behavior inference model 100 is performed using the behavior sequences, policy conditions, and policy effects. In the second stage, learning of only the decoder 100B may be performed.

[0040] The learning unit 22 stores the learned behavior estimation model 100 in the model storage unit 24 .

[0041] <Configuration of Estimation Apparatus According to This Embodiment> FIG. 1 is a block diagram showing the hardware configuration of an estimation apparatus 50 according to this embodiment.

[0042] 1 , the estimation device 50, like the learning device 10, has a CPU 11, a ROM 12, a RAM 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other so as to be able to communicate with each other via a bus 19. An estimation program is stored in the ROM 12 or the storage 14.

[0043] The input unit 15 is used to input various information including the past behavioral sequence of the customer to be estimated and the policy conditions related to the policy to be implemented for the customer.

[0044] Next, a description will be given of the functional configuration of the estimation device 50. Fig. 8 is a block diagram showing an example of the functional configuration of the estimation device 50.

[0045] As shown in FIG. 8, the estimation device 50 functionally includes a model storage unit 52, an acquisition unit 54, and an estimation unit 56.

[0046] The model storage unit 52 stores a trained behavior estimation model 100 similar to that of the model storage unit 24 of the learning device 10 .

[0047] The acquisition unit 54 acquires the input past behavioral sequence of the customer to be estimated and the policy conditions related to the policy to be implemented on the customer.

[0048] The estimation unit 56 uses the trained behavior estimation model 100 to estimate the effect of a measure when the acquired behavioral sequence and policy conditions are input, by estimating a combination of the presence or absence of one or more behavioral contents of the target customer and one or more evaluation numerical values.

[0049] For example, the estimation unit 56 inputs the behavioral sequence of the customer to be estimated into the encoder 100A, and inputs the policy conditions regarding the timing of advertisement delivery, which are either no delivery, delivery this time, or next delivery, into the decoder 100B, and uses the output of the decoder 100B to estimate the policy effects, such as whether or not there is a response one week after delivery, whether or not there is a response one month after delivery, and sales.

[0050] <Operation of the Learning Device According to the Present Embodiment> Next, the operation of the learning device 10 according to the present embodiment will be described.

[0051] 9 is a flowchart showing the flow of the learning process by the learning device 10. The learning process is performed by the CPU 11 reading out a learning program from the ROM 12 or storage 14, expanding it into the RAM 13, and executing it. Also, it is assumed that learning data including past behavioral patterns for each customer, policy conditions related to policies implemented for the customer, and policy effects related to the policies implemented for the customer has been input to the learning device 10. The learning process is an example of a learning method.

[0052] In step S100 , the CPU 11 functions as the acquisition unit 20 to acquire a plurality of pieces of learning data received by the input unit 15 .

[0053] In step S102, the CPU 11 functions as the learning unit 22 to perform self-supervised learning on the encoder 100A of the behavior estimation model 100 using behavior sequences included in a plurality of pieces of training data.

[0054] In step S104, the CPU 11, as the learning unit 22, performs supervised learning on the entire behavior estimation model 100 using multiple pieces of learning data, stores the learned behavior estimation model 100 in the model storage unit 24, and ends the learning process.

[0055] <Operation of the Estimation Device According to the Present Embodiment> Next, the operation of the estimation device 50 according to the present embodiment will be described.

[0056] 10 is a flowchart showing the flow of estimation processing by the estimation device 50. The estimation processing is performed by the CPU 11 reading out an estimation program from the ROM 12 or the storage 14, deploying it in the RAM 13, and executing it. The model storage unit 52 of the estimation device 50 stores a trained behavior estimation model 100 trained by the learning device 10. The estimation device 50 is also assumed to have input thereto a past behavior sequence of a customer to be estimated and policy conditions related to policies to be implemented on the customer.

[0057] In step S110, the CPU 11 functions as the acquisition unit 54 to acquire the past behavioral sequence of the customer to be estimated and the policy conditions related to the policy to be implemented on the customer.

[0058] In step S112, the CPU 11, as the estimation unit 56, uses the trained behavior estimation model 100 to estimate a combination of the presence or absence of one or more behavioral contents of the target customer and one or more evaluation numerical values ​​as the effect of the measure when the acquired behavior sequence and measure conditions are input, and then terminates the estimation process.

[0059] As described above, the learning device according to this embodiment acquires learning data including each customer's past behavioral patterns, policy conditions related to policies implemented for the customer, and policy effects that are configured by combining the presence or absence of one or more customer behavioral details with one or more numerical evaluation values, and learns a behavior estimation model that estimates policy effects based on the learning data and inputs the behavioral patterns and policy conditions. This makes it possible to reduce computational costs and estimate both the presence or absence of customer behavioral details and numerical evaluation values ​​as policy effects.

[0060] Furthermore, the estimation device according to this embodiment acquires a customer's past behavioral patterns and the conditions for measures to be implemented on the customer, and estimates the effectiveness of the measures when the acquired behavioral patterns and the conditions for the measures are input using a trained behavior estimation model. This reduces calculation costs and makes it possible to estimate both the presence or absence of the customer's behavior and the evaluation value as the effectiveness of the measures.

[0061] In addition, the output of the behavior estimation model for estimating the effectiveness of a policy is a combination of the behavior targeted by the policy and an evaluation value. This makes it possible to train a behavior estimation model that combines behavior and evaluation values. By using this behavior estimation model for estimation, it is possible to obtain estimation results that suppress contradictions between the behavior that is the effect of the policy and the evaluation value.

[0062] Furthermore, when the number of items for policy conditions and policy effects increases, the computational cost increases only for the input / output layers of the Transformer decoder, while the computational costs of other intermediate layers and the Transformer encoder remain unchanged. This allows the computational cost of estimating policy effects to be reduced.

[0063] Furthermore, when changing, adding, or deleting the policy conditions or the policy effects to be predicted, it is only necessary to update the Transformer decoder, and it is not necessary to change the Transformer encoder. This allows the calculation cost to be reduced even when updating the policy effects to be estimated.

[0064] Furthermore, the present invention is not limited to the device configuration and operation of the above-described embodiment, and various modifications and applications are possible within the scope of the gist of the present invention.

[0065] For example, although the above description has been given taking the case where the estimation device and the learning device are configured separately, the present invention is not limited to this and the estimation device and the learning device may be configured as a single device.

[0066] In addition, the various processes executed by the CPU after reading the software (program) in the above embodiment may be executed by various processors other than the CPU. Examples of processors in this case include dedicated electrical circuits, such as programmable logic devices (PLDs) whose circuit configuration can be changed after manufacture, such as field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs), which are processors having a circuit configuration designed specifically for executing specific processes. Furthermore, the learning process and estimation process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, etc.). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

[0067] In addition, in each of the above embodiments, the learning program and the estimation program are described as being pre-stored (installed) in the storage 14, but the present invention is not limited to this. The programs may be provided in a form stored on a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The programs may also be downloaded from an external device via a network.

[0068] The following additional notes are provided regarding the above-described embodiments.

[0069] (Supplementary Item 1) A learning device comprising: a memory; and at least one processor connected to the memory, wherein the processor is configured to acquire learning data including a past behavioral sequence for each customer, policy conditions related to policies implemented for the customer, and policy effects related to policies implemented for the customer, the policy effects being configured as a combination of the presence or absence of one or more behavioral details of the customer and one or more evaluation numerical values, and to learn a behavior estimation model that estimates the policy effects based on the learning data, using the behavioral sequence and the policy conditions as inputs.

[0070] (Supplementary clause 2) A non-transitory storage medium storing a program executable by a computer to execute a learning process, wherein the learning process is configured to: acquire learning data including past behavioral patterns for each customer, policy conditions related to policies implemented for the customer, and policy effects related to policies implemented for the customer, the policy effects being configured as a combination of the presence or absence of one or more behavioral details of the customer and one or more evaluation numerical values; and learn a behavior estimation model that estimates the policy effects based on the learning data, using the behavioral patterns and the policy conditions as inputs.

[0071] (Supplementary Item 3) An estimation device including: a memory; and at least one processor connected to the memory, wherein the processor is configured to acquire a customer's past behavioral sequence and policy conditions related to a policy to be implemented on the customer, and estimate a policy effect when the acquired behavioral sequence and policy conditions are input using a trained behavior estimation model that estimates a policy effect related to a policy to be implemented on the customer, the policy effect being composed of a combination of the presence or absence of one or more behavioral contents of the customer and one or more evaluation numerical values.

[0072] (Supplementary Item 4) A non-transitory storage medium storing a program executable by a computer to execute an estimation process, wherein the estimation process is configured to: acquire a customer's past behavioral sequence and policy conditions related to a policy to be implemented on the customer; and estimate the policy effect when the acquired behavioral sequence and policy conditions are input using a trained behavior estimation model that estimates the policy effect related to the policy to be implemented on the customer, the policy effect being composed of a combination of the presence or absence of one or more behavioral contents of the customer and one or more evaluation numerical values.

[0073] REFERENCE SIGNS LIST 10 Learning device 11 CPU 13 RAM 14 Storage 15 Input unit 16 Display unit 20, 54 Acquisition unit 22 Learning unit 24, 52 Model storage unit 50 Estimation device 56 Estimation unit 100 Behavior estimation model 100A Encoder 100B Decoder

Claims

1. An acquisition unit that acquires learning data including the past action sequence for each customer, the policy conditions regarding the policies implemented for the customer, and the policy effects regarding the policies implemented for the customer, and including the policy effects composed of combinations of the presence or absence of one or more action contents of the customer and one or more evaluation values; and a learning unit that learns an action estimation model that estimates the policy effects with the action sequence and the policy conditions as inputs based on the learning data. A learning device including the above.

2. The learning device according to claim 1, wherein the learning unit learns the parameters regarding the part of the action estimation model that takes the action sequence as an input by self-supervised learning using the action sequence.

3. The action estimation model has an encoder and a decoder of Transformer. The encoder takes the action sequence as an input. The decoder takes the output of the encoder and the policy conditions as inputs and outputs the policy effects. The learning device according to claim 1.

4. An acquisition unit that acquires the past action sequence of a customer and the policy conditions regarding the policy to be implemented for the customer; and an estimation unit that estimates the policy effects regarding the policy to be implemented for the customer, which are composed of combinations of the presence or absence of one or more action contents of the customer and one or more evaluation values, using a learned action estimation model with the action sequence and the policy conditions as inputs, when the acquired action sequence and the policy conditions are used as inputs. An estimation device including the above.

5. The action estimation model has an encoder and a decoder of Transformer. The encoder takes the action sequence as an input. The decoder takes the output of the encoder and the policy conditions as inputs and outputs the policy effects. The estimation device according to claim 4.

6. A learning method of acquiring learning data including the past action sequence for each customer, the policy conditions regarding the policies implemented for the customer, and the policy effects regarding the policies implemented for the customer, and including the policy effects composed of combinations of the presence or absence of one or more action contents of the customer and one or more evaluation values, and learning an action estimation model that estimates the policy effects with the action sequence and the policy conditions as inputs based on the learning data.

7. A program for causing a computer to function as the learning device according to any one of claims 1 to 3, or the estimation device according to claim 4 or 5.

Images