Information processing method, information processing device, and information processing program

Abstract

It estimates user behavior based on a series of user actions. [Solution] The information processing method includes: acquiring a device log output from a device used by a user; issuing a token for the device log; inputting the target time series token containing the issued series of tokens into a language model pre-trained to output a feature vector of a time series token containing a series of tokens, thereby calculating a target feature vector which is the feature vector of the target time series token; inputting the target feature vector into an output model pre-trained to output an estimated result of the action corresponding to the feature vector, thereby estimating the user's action; and outputting the estimated result of the action.

Description

[Technical Field]

[0001] The present disclosure relates to a technology for estimating a user's behavior. [Background Art]

[0002] Patent Literature 1 discloses a technology for detecting the state of a device used by a consumer, searching for a reference action entry corresponding to the detected state of the device by referring to a reference action entry that represents the correspondence between the consumer's behavior and the state of the device used by the consumer when performing the behavior, and estimating the consumer's behavior based on the searched reference action entry. [Prior Art Literature] [Patent Literature]

[0003] [Patent Literature 1] Japanese Unexamined Patent Publication No. 2009-26305 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, in Patent Literature 1, since the consumer's behavior corresponding to the state of the device is merely estimated from the state of the device, it is difficult to estimate the user's behavior grasped from a series of the user's motions.

[0005] An object of the present disclosure is to provide a technology for estimating a user's behavior grasped from a series of the user's motions. [Means for Solving the Problem]

[0006] An information processing method in one aspect of the present disclosure is an information processing method performed by a computer, comprising: acquiring a device log output from a device used by a user; issuing tokens for the device log; calculating a target feature vector which is the feature vector of the target time series token by inputting a target time series token containing the issued series of tokens into a language model pre-trained to output a feature vector of a time series token containing a series of tokens; estimating the user's behavior by inputting the target feature vector into an output model pre-trained to output an estimation result of the behavior corresponding to the feature vector; and outputting the estimation result of the behavior. [Effects of the Invention]

[0007] According to this disclosure, it is possible to estimate a user's behavior based on a series of actions taken by the user. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the configuration of an information processing system in an embodiment of the present disclosure. [Figure 2] This figure shows the first example of a correspondence table. [Figure 3] This figure shows a second example of a correspondence table. [Figure 4] This is a schematic diagram illustrating the structure of the inference model. [Figure 5] This flowchart shows an example of processing in the estimation phase of the information processing device in the embodiment of the present disclosure. [Figure 6] This flowchart shows an example of processing in the learning phase of the information processing device in this embodiment. [Figure 7] This figure shows an example of a target time-series token. [Figure 8] This figure shows an example of a target time-series token that includes a dummy token indicating the time and day of the week of occurrence. [Figure 9]This figure shows an example of a target time series token that includes a dummy token indicating a blank period. [Modes for carrying out the invention]

[0009] (Knowledge forming the basis of this disclosure) The advancement of IoT (Internet of Things) technology has made it easy to obtain device logs output from various devices installed in homes. These device logs can be used to estimate user behavior and understand their lifestyle patterns. Understanding these lifestyle patterns allows for, for example, the development of targeted marketing strategies. Therefore, estimating user behavior from device logs is a valuable tool.

[0010] For example, Patent Document 1 discloses a technology that detects the state of equipment used by a user and estimates the user's behavior by comparing the detected state of the equipment with a database in which the state of the equipment and the user's actions are pre-associated.

[0011] However, Patent Document 1 only estimates user behavior corresponding to the state of the device. Therefore, Patent Document 1 cannot estimate behaviors (such as cooking) that can only be understood by analyzing a series of actions, such as opening a refrigerator or heating something in a microwave oven.

[0012] Therefore, the inventors have come up with the idea of ​​this disclosure based on the realization that by issuing tokens for device logs and inputting a time-series token consisting of a series of issued tokens into a language model, it is possible to estimate a user's actions, which consist of a series of user actions.

[0013] (1) An information processing method according to an aspect of the present disclosure is an information processing method executed by a computer, comprising: acquiring a device log output from a device used by a user; issuing a token for said device log; inputting a target time-series token including the issued series of said tokens into a language model pre-trained to output a feature vector of time-series tokens including a series of said tokens, thereby calculating a target feature vector that is said feature vector of said target time-series token; inputting said target feature vector into an output model pre-trained to output an estimation result of an action corresponding to the feature vector, thereby estimating the action of said user; and outputting the estimation result of said action.

[0014] According to this configuration, tokens are issued from the device log, a target time-series token including the issued series of tokens is input to the language model, the feature vector output from the language model is input to the output model, and the user's action is estimated. Thereby, this configuration can estimate the user's action consisting of a series of operations.

[0015] This configuration issues tokens for a device log, and inputs time-series tokens obtained by arranging the tokenized device logs in chronological order into the language model. Thereby, the device log is converted into an input sequence similar to natural language, and can be handled in chronological order. As a result, the user's action can be estimated from the device log.

[0016] Furthermore, since the output model is pre-trained to estimate an action from the feature vector output from the language model, the output model can be configured with a small number of parameters, and the function of the computer can be improved.

[0017] (2) In the information processing method according to (1) above, the device log may include time information when the user uses the device and operation details of the device.

[0018] According to this configuration, since the device log includes time information and operation content, it is possible to grasp when the estimated user behavior occurred.

[0019] (3) In the information processing method according to (1) or (2) above, the device log may include time information when the user uses the device and operation content of the device.

[0020] According to this configuration, since the device log includes device information, it is possible to grasp which device the user operated.

[0021] (4) In the information processing method according to any one of (1) to (3) above, the time information may include a use time and a day of the week when the device is used.

[0022] According to this configuration, since the time information includes the use time and the day of the week, it is possible to grasp when and on which day of the week the estimated user behavior occurred.

[0023] (5) In the information processing method according to any one of (1) to (4) above, issuing the token may include generating a dummy token indicating an occurrence time of the behavior of the user based on time information included in the device log, and including the dummy token in the target time-series token.

[0024] According to this configuration, since a dummy token indicating an occurrence time of a behavior is included in the time-series token, it is possible to specify when the estimated behavior occurred.

[0025] (6) In the information processing method according to any one of (1) to (5) above, issuing the token may include issuing the token using correspondence information in which a component of the device log and a token ID corresponding to the component are associated in advance.

[0026] In this configuration, tokens are issued using correspondence information that pre-associates the components of the device log with the tokens corresponding to those components. This reduces the processing steps required to issue tokens and improves the computer's functionality.

[0027] (7) In the information processing method described in any of (1) to (6) above, the corresponding information may be composed of community units to which the user belongs.

[0028] With this configuration, tokens are issued using correspondence information common to the community, so it is possible to issue tokens using correspondence information optimized for each community.

[0029] (8) In the information processing method described in any of (1) to (7) above, the result of estimating the action may be a probability distribution that shows the probability that the user took each of the multiple actions.

[0030] With this configuration, the probability of each action is shown as an estimated result, allowing us to understand the validity of the estimated action from its probability.

[0031] (9) In the information processing method described in any of (1) to (8) above, the language model may be a neural network model that has been pre-trained by self-supervised learning of the time-series tokens.

[0032] This configuration allows us to construct a language model that can accurately calculate feature vectors from time-series tokens.

[0033] (10) In the information processing method described in any of (1) to (9) above, the output model may be a neural network model that has been supervised and trained to minimize the error between the estimated behavior output from the output model and the correct label indicating the behavior corresponding to the time series token when the feature vector output from the language model is input to the output model when the time series token is input to the language model.

[0034] This configuration allows for the generation of an output model that can accurately estimate user behavior from feature vectors.

[0035] (11) In the information processing method described in any of (1) to (10) above, the device log may include the transmission history of SNS (Social Networking Service).

[0036] This configuration allows us to estimate user behavior on social media.

[0037] (12) In the information processing method described in any of (1) to (11) above, the actions to be estimated may include at least one of the following: getting up, going to bed, cleaning, going out, returning home, cooking, eating, bathing, doing laundry, and transmission history.

[0038] This configuration allows us to estimate the user's daily behavior.

[0039] (13) In the information processing method described in any of (1) to (12) above, issuing the token may include generating a dummy token that indicates a gap in user activity based on the time interval of the device logs that are in the order of the time series, and including the dummy token in the target time series token.

[0040] This configuration allows for a more accurate estimation of user behavior by taking into account the time gaps in user actions that occur when a user performs a particular action.

[0041] This disclosure can also be implemented as an information processing program that causes a computer to execute each of the characteristic configurations included in such information processing methods, or as an information processing system that operates using this information processing program. It goes without saying that such a computer program can be distributed via computer-readable non-temporary recording media such as CD-ROMs or via communication networks such as the Internet.

[0042] (14) An information processing device in another aspect of the present disclosure is an information processing device having a processor, the processor acquires a device log output from a device used by a user, issues tokens for the device log, inputs a target time series token including the issued tokens to a language model pre-trained to output a feature vector of a time series token including a series of tokens, calculates a target feature vector which is the feature vector of the target time series token, inputs the target feature vector to an output model pre-trained to output an estimated result of an action corresponding to the feature vector, estimates the user's action, and outputs the estimated result of the action.

[0043] (15) An information processing program in another aspect of the present disclosure causes a computer to perform the following: acquire a device log output from a device used by a user; issue tokens for the device log; input a target time series token containing the issued tokens into a language model pre-trained to output a feature vector of a time series token containing a series of tokens, thereby calculating a target feature vector which is the feature vector of the target time series token; input the target feature vector into an output model pre-trained to output an estimated result of an action corresponding to the feature vector, thereby estimating the user's action; and outputting the estimated result of the action.

[0044] The embodiments described below are all specific examples of this disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept will be described as optional components. In addition, the contents of each embodiment can be combined.

[0045] (Embodiment) Figure 1 is a block diagram showing an example of the configuration of an information processing system 1 in an embodiment of the present disclosure. The information processing system 1 includes an information processing device 10, a device 20, a display device 30, and a user terminal 40.

[0046] The information processing device 10 is composed of computers such as cloud servers and edge servers. Device 20 is a device that generates operation history and other data resulting from user actions. For example, device 20 is an electrical appliance installed in the user's house and used by the user. Examples of electrical appliances in device 20 include refrigerators, air conditioners, washing machines, microwave ovens, ovens, televisions, and audio equipment. Device 20 is an IoT device equipped with communication functions, but it also includes devices installed outdoors, such as delivery boxes and intercoms. Device 20 is connected to the information processing device 10 via a network, which is, for example, the internet. Device 20 sends device logs to the information processing device 10 when a predetermined event occurs and / or periodically thereafter. A predetermined event occurs, for example, when an operation is input from the user, or when device 20 autonomously changes its operating mode or settings. The device log is an operation history showing the operating status of device 20.

[0047] The device log includes a timestamp and details of the device's operation. That is, the device log has a table-like data structure where multiple items are associated with corresponding values. The timestamp includes information about when the user used the device and when the device's settings were changed. The time information includes the time of use and day of the week when the user used the device.

[0048] The device log may include the transmission history of the user terminal 40. The transmission history includes, for example, the transmission history of SNS (Social Networking Service) messages and emails sent by the user using the user terminal 40. SNS messages include not only messages posted to SNS, but also so-called "like" messages that indicate the user has given a favorable evaluation to the content.

[0049] The display device 30 is, for example, a personal computer and is connected to the information processing device 10 via a network for communication. The display device 30 obtains the estimated results of user behavior from the information processing device 10 and displays the estimated results. The display device 30 is owned, for example, by the operator of the information processing device 10 or by a marketing planner who plans marketing tailored to the user based on the estimated results of user behavior.

[0050] The user terminal 40 is a device owned by the user whose actions are to be estimated, and is connected to the information processing device 10 via a network in a communicable manner. The user terminal 40 consists of, for example, a personal computer, a smartphone, a tablet computer, etc. When the user terminal 40 sends an SNS message or email, it generates a transmission history and sends it to the information processing device 10. The transmission history includes, for example, a timestamp and the content of the transmission. If the transmission history is included in the device log, the user terminal 40 is also included in the concept of a device.

[0051] The information processing device 10 includes a processor 100 and a memory 200. The processor 100 is composed of a central processing unit and includes an acquisition unit 101, a token issuing unit 102, an estimation unit 103, an output unit 104, and a learning unit 105. The acquisition unit 101 to the learning unit 105 are realized, for example, by the CPU executing an information processing program stored in the memory 200. However, this is just an example, and the acquisition unit 101 to the learning unit 105 may be composed of dedicated hardware circuits such as ASICs.

[0052] The acquisition unit 101 acquires a device log from the device 20 via the communication circuit shown in the diagram. The acquisition unit 101 stores the acquired device log in the device log storage unit 201. The acquisition unit 101 acquires the transmission history from the user terminal 40 via the communication circuit shown in the diagram. The acquisition unit 101 stores the acquired transmission history as a device log in the device log storage unit 201.

[0053] The token issuing unit 102 acquires device logs from the device log storage unit 201 or the acquisition unit 101, and issues tokens for the acquired device logs.

[0054] A token is the smallest unit used when parsing text. In this embodiment, a token consists of a token ID that uniquely identifies a component included in the device log. The component included in the device log is text that describes the operation of device 20, or a plurality of words that make up that text.

[0055] The token issuing unit 102 issues a token by referring to the correspondence table 202 stored in memory 200. The correspondence table 202 is an example of correspondence information. Figure 2 shows a first example of the correspondence table 202. The correspondence table 202 includes items for components and token IDs. The correspondence table 202 stores components in association with the token IDs corresponding to those components. The correspondence table 202 is created in advance and stored in memory 200. The components are components of the device log. In the first example of the correspondence table 202, the components are sets of objects (or subjects) and verbs that constitute the text indicating the operation content included in the device log (for example, "open the refrigerator"). The token ID is an identifier that uniquely identifies the component.

[0056] Figure 3 shows a second example of correspondence table 202. In the second example of correspondence table 202, the words that make up the text indicating the operation content included in the device log are considered as constituent elements. For example, if the operation content is "open the refrigerator," then the object "refrigerator" and the verb "open" are both considered as constituent elements.

[0057] Let's refer back to Figure 1. The estimation unit 103 groups the series of tokens issued by the token issuance unit 102 together as time-series tokens. Hereafter, these grouped time-series tokens will be referred to as target time-series tokens. The estimation unit 103 inputs the target time-series tokens into the language model to calculate the target feature vector.

[0058] The language model is a component of the inference model 203. As shown in Figure 4, the inference model 203 includes the language model 231 and the output layer 232. The language model 231 consists of a neural network model pre-trained to output feature vectors of time-series tokens containing a set of tokens. Self-supervised learning using a large number of time-series tokens can be applied as pre-training. The feature vector is a vector of a predetermined dimension that represents the features of the time-series token.

[0059] The estimation unit 103 estimates user behavior by inputting the calculated target feature vector to the output layer 232. The output layer 232 is pre-trained to output behavior estimation results corresponding to a large number of feature vectors. In detail, the output layer 232 is a supervised-trained neural network model that minimizes the error between the behavior estimation result output from the output layer 232 and the correct label indicating the behavior corresponding to the time-series token when the feature vector output from the language model 231 is input to the language model. The output layer 232 is an example of an output model. The user behaviors to be estimated are predetermined. The user behaviors to be estimated include, for example, at least one of waking up, going to bed (sleeping), cleaning, going out, returning home, cooking, eating, bathing, and doing laundry. The estimation unit 103 generates a probability distribution as an estimation result that shows the probability of each behavior to be estimated. The estimation unit 103 may also use the behavior with the highest probability as the estimated user behavior.

[0060] Refer back to Figure 1. The output unit 104 outputs the estimation result generated by the estimation unit 103 to the display device 30. The output unit 104 may also output the estimation result to the user terminal 40.

[0061] The learning unit 105 trains the inference model 203.

[0062] The memory 200 is composed of a non-volatile, rewritable storage device, such as a solid-state drive. The memory 200 includes a device log storage unit 201. The memory 200 stores the correspondence table 202 and the inference model 203.

[0063] Figure 4 is a schematic diagram showing the structure of the inference model 203. The inference model 203 includes a language model 231 and an output layer 232.

[0064] The language model 231 is composed of a large-scale language model with a structure similar to, for example, BART (Bidirectional Encoder Representations from Transformers). The language model 231 is trained by self-supervised learning using MLM (Masked Language Modeling) and NSP (Next Sentence Prediction). MLM is a method of training the language model to predict the masked words from a text by masking some of the words contained in the text. NSP is a method of training the language model to correctly determine whether two texts are consecutive sentences or not. In this embodiment, the learning unit 105 masks some of the tokens that make up the time-series tokens and trains the language model 231 to correctly predict the masked tokens. The learning unit 105 also trains the language model 231 to correctly determine whether two time-series tokens are consecutive or not.

[0065] The output layer 232 consists of multiple output nodes connected to multiple nodes that constitute the final layer of the language model 231. The output layer 232 is trained using training data in which time-series tokens are associated with ground truth labels indicating the actions corresponding to those time-series tokens. The learning unit 105 inputs the time-series tokens to be trained into the language model 231. The learning unit 105 inputs the feature vectors output from the language model 231 into the output layer 232 and adjusts the parameters of the output layer 232 so that the error between the estimation result output from the output layer 232 and the ground truth label indicating the action corresponding to the time-series token is minimized. The parameters are the weight values ​​and bias values ​​of each node that constitutes the output layer 232.

[0066] Figure 5 is a flowchart illustrating an example of processing in the estimation phase of the information processing device 10 in an embodiment of the present disclosure. The flowchart in Figure 5 is performed, for example, at predetermined intervals. The predetermined interval is, for example, one day, one week, etc. The following processing will be explained using the example of processing device logs output from a device 20 installed in a house. The processor 100 executes the processing shown in Figure 5 for all device logs acquired at the predetermined time.

[0067] (Step S1) The token issuing unit 102 acquires the device log from the device log storage unit 201 or the acquisition unit 101.

[0068] (Step S2) The token issuing unit 102 issues tokens for the acquired device logs. The following describes an example of token issuance when using the correspondence table 202 in Figure 2. The token issuing unit 102 extracts objects, subjects, and verbs from the text by applying morphological analysis and syntactic analysis to the text stored in the operation details section of the device log. For example, if the text contains the object "the refrigerator" and the verb "open", the token issuing unit 102 extracts "open the refrigerator" as a component from the device log. The token issuing unit 102 searches for the token ID corresponding to the extracted component "open the refrigerator" in the correspondence table 202 shown in Figure 2. In the correspondence table 202 in Figure 2, token ID "101" is associated with the component "open the refrigerator". Therefore, the token issuing unit 102 issues token "101" for the component "open the refrigerator".

[0069] On the other hand, the following is an example of token issuance when using the correspondence table 202 in Figure 3. The token issuance unit 102 segments the text into words by applying morphological analysis to the text stored in the operation details item of the device log. The token issuance unit 102 searches for the token ID for the segmented words in the correspondence table 202 in Figure 3. For example, if the text is "open the refrigerator", the words "refrigerator" and "open" are extracted from the text. In the correspondence table 202 in Figure 3, token IDs "101" and "301" are associated with the components "refrigerator" and "open", respectively. Therefore, the token issuance unit 102 issues token "101" for the word "refrigerator" and token "301" for the word "open".

[0070] The correspondence table 202 may be created separately for each user. If the device log contains the user ID of the user who operated the device 70, the token issuing unit 102 may refer to the correspondence table 202 of the user indicated by the user ID and issue a token. For example, if a device log is obtained indicating that the person who performed the action "open the refrigerator" was the mother, the token issuing unit 102 will refer to the mother's correspondence table 202. If "101" is registered as the token ID for "open the refrigerator" in the mother's correspondence table 202, the token issuing unit 102 will issue a token of "101" for "open the refrigerator". On the other hand, if "102" is registered as the token ID for "open the refrigerator" in the father's correspondence table 202, the token issuing unit 102 should issue a token of 102 for "open the refrigerator".

[0071] The correspondence table 202 may be created on a community basis. A community is, for example, a family or a specific group. The token issuing unit 102 may issue a token by referring to the user-specific correspondence table 202 if the device log contains a user ID, or by referring to the community-based correspondence table 202 if the device log does not contain a user ID. The token issuing unit 102 may also issue tokens using only the community-based correspondence table 202. In this case, the effort of managing the correspondence table 202 for each user is eliminated, and the storage capacity of memory 200 is saved.

[0072] (Step S3) The estimation unit 103 groups the tokens issued in step S2 into fixed units to generate a target time-series token. For example, the estimation unit 103 can generate a single target time-series token from a series of tokens issued from a series of device logs where the time interval between consecutive device logs is within a fixed time interval T1. The time interval T1 can be any appropriate value, such as 30 minutes or 1 hour.

[0073] Figure 7 shows an example of a target time-series token. In the example in Figure 7, the time intervals of the series of device logs corresponding to "open refrigerator," "rice cooking complete," "open freezer," and "heat in microwave" were within time interval T1. The correspondence table 202 shown in Figure 2 stored the token IDs "103," "203," "101," and "307" for "open refrigerator," "rice cooking complete," "open refrigerator," and "heat in microwave," respectively. Therefore, the estimation unit 103 assigns "103," "203," "101," and "307" to "open refrigerator," "rice cooking complete," "open freezer," and "heat in microwave," and combines these series of tokens to generate a single target time-series token "103 / 203 / 101 / 307." The symbol " / " indicates a token separator.

[0074] As things stand, the time of occurrence of the time-series token cannot be determined. Therefore, the estimation unit 103 may issue a dummy token indicating the time of occurrence of the time-series token and include the issued dummy token in the target time-series token.

[0075] The dummy token may be a token indicating the time of occurrence. The dummy token may also be a token indicating the day of the week. This allows us to determine when the time-series tokens occurred and when the estimated actions occurred.

[0076] Figure 8 shows an example of a target time-series token that includes dummy tokens indicating the time and day of the week of occurrence. In the example in Figure 8, similar to Figure 7, the target time-series tokens "103 / 203 / 101 / 307" are shown for "open refrigerator / rice cooking finished / open freezer / heat in microwave." The estimation unit 103 issues a dummy token indicating the time of occurrence for this target time-series token and includes the issued dummy token in the time-series token.

[0077] In this example, the occurrence time of the target time series token "103 / 203 / 101 / 307" was "16:00" on "Wednesday". Therefore, the estimation unit 103 issues a dummy token "1003" for "Wednesday" and a dummy token "16" for "16:00", and includes the dummy token "1003 / 16" in the time series token. As a result, the target time series token becomes "1003 / 16 / 103 / 203 / 101 / 307". Here, the occurrence time of the target time series token can be the time indicated by the timestamp of any one of the series of device logs corresponding to the target time series token. For example, the occurrence time of the target time series token can be the time indicated by the timestamp of the oldest device log among the series of device logs. When the method of including a dummy token indicating the occurrence time in the target time series token is adopted, the estimation unit 103 should calculate an estimation result that includes the probability distribution of the action and the occurrence time indicated by the time series token. The insertion position of the dummy token is, for example, the beginning position of the original time-series token. The token IDs corresponding to the days of the week are predetermined in correspondence table 202. For tokens corresponding to times, a numerical value indicating the time can be used (for example, 1305 for 13:50).

[0078] A dummy token may be a token that indicates a gap in time during which user actions cannot be determined from the device log. Figure 9 shows an example of a target time series token that includes a dummy token indicating a gap in time. In the example in Figure 9, similar to Figure 7, the target time series tokens "103 / 203 / 101 / 307" are shown for "open refrigerator / rice cooking complete / open freezer / heat in microwave." In this example, there was a 30-minute gap between "rice cooking complete" and "open refrigerator." Therefore, the estimation unit 103 inserts a dummy token "401" indicating a gap in time between the token "203" corresponding to "rice cooking complete" and the token "101" corresponding to "open refrigerator" in this target time series token. The gap in time is, for example, a period of a predetermined time TO (e.g., 10 minutes) or more and less than the above-mentioned time interval T1 (e.g., 1 hour). The estimation unit 103 calculates the time interval Tx of the preceding and succeeding tokens among the multiple tokens that constitute the target time series token from the timestamp of the device log. The estimation unit 103 inserts a dummy token indicating a gap in time between the corresponding preceding and succeeding tokens if the time interval Tx is greater than or equal to a predetermined time TO and less than the time interval T1. The dummy token indicating a gap in time may be a token whose length can be identified. For example, if the gap in time is 30 minutes, the estimation unit 103 may issue a dummy token whose length can be identified as 30 minutes.

[0079] (Step S4) The estimation unit 103 calculates the feature vector (target feature vector) of the target time series token by inputting the target time series token into the language model 231. Specifically, the estimation unit 103 represents the target time series token as a 1-hot vector of, for example, several hundred (e.g., 400) dimensions and inputs that 1-hot vector into the language model 231.

[0080] (Step S5) The estimation unit 103 estimates user behavior by inputting the target feature vector calculated in step S4 to the output layer 232. As a result, the estimation unit 103 calculates probability distributions showing the probability of each of the following actions as the behavior estimation result: waking up, going to bed, cleaning, going out, returning home, cooking, eating, bathing, and doing laundry. If a dummy token containing the time of action is included in the target time series token, the estimation unit 103 may include the time of action in the estimation result. In this case, the estimation unit 103 may aggregate the behavior estimation results calculated for all device logs throughout the day to calculate the daily behavior estimation result. The daily behavior estimation result shows what actions were taken at what time of day. For example, the daily behavior estimation result can use data showing actions at specific time periods, such as cooking between 5 PM and 6 PM, and eating between 7 PM and 8 PM. Furthermore, if a user-specific correspondence table 202 is used, the estimation unit 103 may estimate the user who is the primary performer of the action. In this case, the estimation unit 103 only needs to include the user ID of the user who performed the action in the action estimation result.

[0081] Furthermore, if the device log includes the aforementioned transmission history, the presumed actions include, for example, posting messages on social media, sending emails, or posting "like" messages.

[0082] (Step S6) The output unit 104 outputs the behavior estimation result calculated in step S5 to the display device 30. The display device 30 then displays the behavior estimation result. This allows the marketing planner to understand the user's lifestyle patterns from the estimation result and formulate a marketing strategy for that user. For example, if the behavior estimation result shows that the user cooks between 5 PM and 6 PM, the marketing planner can formulate a marketing strategy such as advertising home-delivered meals or ingredients to the user during this time.

[0083] The output unit 104 may output the behavior estimation result calculated in step S5 to the user terminal 40. The user who has seen the behavior estimation result may input a feedback instruction to the user terminal 40 to point out an error in the behavior estimation result. For example, if the behavior estimation result is that the time period from 17:00 to 18:00 is cooking, but the correct answer is eating, the user will input a feedback instruction to the user terminal 40 indicating that the time period from 17:00 to 18:00 was eating, not cooking. The user terminal 40 will send the feedback instruction to the information processing device 10. The feedback instruction is acquired by the learning unit 105. The learning unit 105 can then retrain the inference model 203 with the correct label of the target time series token corresponding to the behavior estimation result for which feedback was given set to "eating". In addition, the user who has seen the behavior estimation result can reflect on their own actions.

[0084] Figure 6 is a flowchart showing an example of processing in the learning phase of the information processing device 10 in this embodiment. This flowchart is performed, for example, at predetermined intervals. The predetermined time interval for the learning phase may be the same as or longer than the execution interval of the estimation phase. For example, if the estimation phase is executed once a day, the learning phase may be performed once every 1 month, 3 months, or 6 months.

[0085] (Step S11) The learning unit 105 trains the language model 231 by self-supervised learning of time-series tokens. For example, in the initial state, the learning unit 105 trains the language model 231 using pre-prepared time-series tokens, and in subsequent instances, it can retrain the language model 231 using target time-series tokens issued by the token issuing unit 102.

[0086] (Step S12) The learning unit 105 supervises the output layer 232 using training data that includes time-series tokens and correct labels for actions corresponding to those time-series tokens. For example, in the initial state, the learning unit 105 trains the output layer 232 using pre-prepared time-series tokens, and from the next time onward, it trains the output layer 232 using target time-series tokens issued by the token issuing unit 102 that correspond to the estimated results of actions for which feedback instructions have been input. In this case, the learning unit 105 trains the output layer 232 by specifying the correct action indicated by the feedback instruction as the correct label for the target time-series tokens used as training data.

[0087] Thus, according to this embodiment, tokens are issued from the device log, the target time-series tokens including the issued series of tokens are input to the language model, the feature vectors output from the language model are input to the output model, and the user's behavior is estimated. As a result, this configuration can estimate the user's behavior consisting of a series of actions.

[0088] For example, in conventional user behavior estimation using device logs, if three device logs corresponding to (i) opening the freezer, (ii) taking out frozen food, and (iii) heating in the microwave were obtained, user behavior was estimated for each of the three device logs. Therefore, it was difficult to estimate user behavior that took into account the user's intention arising from the series of actions (i) → (ii) → (iii). In contrast, this configuration estimates user behavior from time-series tokens corresponding to the series of actions consisting of (i) → (ii) → (iii). Therefore, this configuration can estimate user behavior that can be grasped from a series of actions.

[0089] Language model 231 excels at contextual understanding of time-series data and is adept at understanding natural language. Language model 231 was developed to understand natural language, and its input typically uses tokens, where numbers are assigned to natural language elements. However, since device logs are not natural language, they cannot be directly applied to the language model.

[0090] The information processing device 10 issues tokens for the device logs and inputs the time-series tokens obtained by arranging the tokenized device logs in chronological order into the language model 231. As a result, the device logs are converted into an input sequence similar to natural language and can be handled in chronological order. Consequently, user behavior can be estimated from the device logs.

[0091] Furthermore, since the output layer 232 (output model) is pre-trained to estimate behavior from feature vectors output from the language model, the output layer 232 can be constructed with a small number of parameters, thereby improving the computer's capabilities.

[0092] The following modifications may be adopted for this disclosure.

[0093] (1) When the transmission history is used as the device log, the correspondence table 202 shown in Figure 2 registers elements such as posting SNS messages and sending emails, and the correspondence table 202 shown in Figure 3 registers elements such as "email," "SNS message," and "send."

[0094] (2) In Figure 1, the information processing device 10 is described as a cloud server, but it may also be installed in a home appliance.

[0095] (3) In Figure 1, the information processing device 10 is equipped with a learning unit 105, but the learning unit 105 may be provided by an external computer.

[0096] (4) In Figure 1, the inference model 203 is provided by the information processing device 10, but it may also be provided by an external computer.

[0097] (5) The operation details of the device log are not limited to text, but may also be in binary data format.

[0098] (6) The device log may include identification information for device 20.

[0099] (7) The estimation unit 103 may calculate the estimated behavior for each day of the week by aggregating the estimated behavior results calculated for each day of the week.

[0100] (8) The acquisition unit 101 may acquire device logs from an external server that stores device logs from the device 20. [Industrial applicability]

[0101] According to this disclosure, it is possible to estimate user behavior based on a series of user actions, which is useful when providing various services to users. [Explanation of Symbols]

[0102] 1: Information Processing System 10: Information Processing Device 20:Equipment 30:Display device 40: User terminal 100: Processor 101: Acquisition Department 102: Token Issuance Department 103:Estimation part 104: Output section 105: Learning Department 200: Memory 201: Equipment log storage unit 202: Correspondence Table 203: Inference Models 231: Language Model 232: Output layer

Claims

1. A method of information processing performed by a computer, Obtaining device logs output from devices used by the user, To issue a token for the aforementioned device log, By inputting a target time series token containing the issued series of tokens into a language model pre-trained to output a feature vector of a time series token containing a series of tokens, the target feature vector, which is the feature vector of the target time series token, is calculated. The user's behavior is estimated by inputting the target feature vector into an output model that has been pre-trained to output the estimated behavior corresponding to the feature vector, This includes outputting the estimated results of the aforementioned actions, Information processing methods.

2. The aforementioned device log includes time information and details of the device's operation when the user used the device. The information processing method according to claim 1.

3. The device log further includes identification information of the device used by the user. The information processing method according to claim 2.

4. The aforementioned time information includes the time of use and day of the week when the equipment was used. The information processing method according to claim 2.

5. Issuing the aforementioned token includes generating a dummy token indicating the time of occurrence of the user's action based on the time information contained in the device log, and including the dummy token in the target time series token. The information processing method according to claim 1.

6. Issuing the aforementioned token includes issuing the token using correspondence information in which the components of the device log and the token ID corresponding to the components are pre-associated. The information processing method according to claim 1.

7. The aforementioned correspondence information is organized on a community-by-community basis to which the user belongs. The information processing method according to claim 6.

8. The estimated result of the aforementioned actions is a probability distribution that shows the probability that the user took each of the multiple actions. The information processing method according to claim 1.

9. The language model is a neural network model that has been pre-trained by self-supervised learning of the time-series tokens. The information processing method according to claim 1.

10. The output model is a supervised-trained neural network model such that, when the feature vector output from the language model is input to the language model when the time-series tokens are input to the language model, the error between the estimated action output from the output model and the correct label indicating the action corresponding to the time-series tokens is minimized. The information processing method according to claim 1.

11. The aforementioned device log includes the transmission history of SNS (Social Networking Service), The information processing method according to claim 1.

12. The actions to be estimated include at least one of the following: waking up, going to bed, cleaning, going out, returning home, cooking, eating, bathing, doing laundry, and sending messages. The information processing method according to claim 1.

13. Issuing the aforementioned tokens includes generating dummy tokens that indicate periods of inactivity for the user based on the time intervals of device logs in a time series, and including the dummy tokens in the target time series tokens. The information processing method according to claim 1.

14. An information processing device having a processor, The aforementioned processor, We obtain device logs output from devices used by the user. A token is issued for the aforementioned device log, By inputting a target time series token containing the issued series of tokens into a language model pre-trained to output a feature vector of a time series token containing a series of the aforementioned tokens, the target feature vector, which is the feature vector of the target time series token, is calculated. The user's behavior is estimated by inputting the target feature vector into an output model that has been pre-trained to output the estimated behavior corresponding to the feature vector. Output the estimated result of the aforementioned action. Information processing device.

15. On the computer, Obtaining device logs output from devices used by the user, To issue a token for the aforementioned device log, By inputting a target time series token containing the issued series of tokens into a language model pre-trained to output a feature vector of a time series token containing a series of tokens, the target feature vector, which is the feature vector of the target time series token, is calculated. The user's behavior is estimated by inputting the target feature vector into an output model that has been pre-trained to output the estimated behavior corresponding to the feature vector, To output the estimated result of the aforementioned action and to perform the following: Information processing program.

Images