Log data analysis device, log data analysis method, and log data analysis program

Abstract

This log data analysis device 10 comprises: an acquisition unit 101 that acquires log data in which an action of a user and one or more attributes associated with the action are written; a selection unit 102 that, on the basis of the action in the log data and the number of items of unique data in a combination of attributes associated with the action, selects a combination of attributes of the log data acquired by the acquisition unit 101; and an analysis unit 103 that analyzes the log data through use of a prescribed language model by using the combination of attributes of the log data selected by the selection unit 102.

Description

[Technical Field]

[0001] The disclosed technology relates to a log data analysis device, a log data analysis method, and a log data analysis program. [Background technology]

[0002] In the field of natural language processing, language modeling techniques (such as BERT) that acquire the meaning of each word in context from a large number of documents are widespread. Language models have also been deployed in other fields, and are used, for example, in recommendation systems (such as BERT4Rec) that acquire the meaning of each action in context from a large number of action logs and suggest the next action. Non-Patent Document 1, for example, is an example of a document disclosing BERT4Rec. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, Peng Jiang BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, Internet<URL:https: / / arxiv.org / abs / 1904.06690> Summary of the Invention [Problem to be solved by the invention]

[0004] Previously, attempts to analyze user behavior from behavior logs collected across services have involved treating combinations of log attributes as vocabulary and applying language models. Identifiers that correspond to words in natural language become identifiers in behavior logs that consist of combinations of one or more attributes. Attributes vary depending on the type of behavior, such as product or service names, channels, and classification categories, and when applying language model technology, it is necessary to design how attributes are combined.

[0005] If we treat all attribute combinations in a user's log as a vocabulary, applying a language model involves treating each attribute combination as a single word in the language model, assigning a unique ID to each combination of attribute values, and then processing the data to represent that single word. This requires a huge amount of dictionary data, which indicates the correspondence between each attribute value combination and its ID. When the dictionary data becomes too large, the primary storage (memory) used during processing becomes insufficient, resulting in insufficient processing. While this can be avoided by using secondary storage, such as a hard disk drive (HDD) or solid-state drive (SSD), instead of primary storage, the computational cost increases because the dictionary must be searched for and updated each time it is referenced, making it impossible to process the data within a realistic timeframe. Selecting specific attributes from a user's log to narrow the vocabulary requires identifying user behavior at an appropriate granularity, since meaningful attributes vary depending on the type of log. If one attribute is selected and used as an identifier for the behavior log, the granularity of the behavior will be coarse, making it difficult to distinguish the characteristic trends of behavior that can be inferred from the language model.If all attributes are selected and used as identifiers for the behavior log, the granularity of the behavior will be too fine, making it impossible to obtain the characteristic trends of behavior that can be inferred from the language model.

[0006] The disclosed technology has been made in consideration of the above points, and aims to provide a log data analysis device, a log data analysis method, and a log data analysis program that analyze log data using combinations that include many behavioral characteristics while reducing computational costs. [Means for solving the problem]

[0007] A first aspect of the present disclosure is a log data analysis device that includes an acquisition unit that acquires log data that describes user behavior and one or more attributes associated with the behavior; a selection unit that selects a combination of attributes of the log data acquired by the acquisition unit based on the behavior of the log data and the number of unique data in the combination of attributes associated with the behavior; and an analysis unit that analyzes the log data using a predetermined language model using the combination of attributes of the log data selected by the selection unit.

[0008] A second aspect of the present disclosure is a log data analysis method, in which a processor acquires log data describing a user's behavior and one or more attributes associated with the behavior, selects a combination of attributes for the acquired log data based on the unique number of combinations of the behavior and the attributes associated with the behavior in the log data, and performs a process of analyzing the log data using a predetermined language model using the selected combination of attributes for the log data.

[0009] A third aspect of the present disclosure is a log data analysis program that causes a computer to function as the log data analysis device of the first aspect of the present disclosure. [Effects of the Invention]

[0010] According to the disclosed technology, by selecting a combination of attributes that is an appropriate number for the model used for analysis, it is possible to provide a log data analysis device, a log data analysis method, and a log data analysis program that analyze log data using combinations that include many behavioral characteristics while reducing computational costs. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a hardware configuration of a log data analysis device according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of a log data analysis device. [Figure 3] FIG. 2 is a diagram illustrating an example of log data to be analyzed by the log data analysis device of the present embodiment. [Figure 4A] FIG. 10 is a diagram showing an example of expressing a user's behavior using only the attribute "channel." [Figure 4B] FIG. 10 is a diagram showing an example of expressing a user's behavior using all of the attributes "channel," "action," and each event. [Figure 4C] FIG. 10 is a diagram showing an example of expressing a user's behavior using attributes "event 1" and "event 2." [Figure 5] FIG. 10 is a diagram illustrating a selection process performed by a selection unit. [Figure 6] FIG. 10 is a diagram illustrating a selection process performed by a selection unit. [Figure 7] FIG. 10 is a diagram illustrating a method for estimating cardinality by a selection unit. [Figure 8] FIG. 10 is a diagram illustrating an example in which the target vocabulary size is divided equally for each data source. [Figure 9] FIG. 10 is a diagram illustrating an example in which a target vocabulary size is assigned in proportion to the maximum vocabulary size for each data source. [Figure 10] 10 is a flowchart showing the flow of a log data analysis process performed by the log data analysis device. [Figure 11] 10A and 10B are diagrams showing the structure of log data and the number of vocabulary words in a single attribute used to explain a specific example of a log data analysis process by a log data analysis device. [Figure 12] 6 is a diagram for explaining a comparison between a case where log data is used as is, a case where the simple method shown in FIG. 5 is used, and a case where the method described in this embodiment is used. DETAILED DESCRIPTION OF THE INVENTION

[0012] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that the same reference numerals are used to designate identical or equivalent components and parts in each drawing. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0013] FIG. 1 is a block diagram showing the hardware configuration of a log data analysis device 10 according to this embodiment. The log data analysis device 10 according to this embodiment acquires log data containing user behavior and one or more attributes associated with the behavior, performs an analysis process on the acquired log data, and outputs the analysis results. The analysis process performed by the log data analysis device 10 analyzes the log data using a language model to analyze characteristic trends in user behavior. In this embodiment, the language model may include both a model using statistics and a model using a neural network. The log data analysis device 10 according to this embodiment uses, for example, word2vec, doc2vec, Transformer, BERT, etc. as the language model. By analyzing the characteristic trends in user behavior using the language model, the log data analysis device 10 can suggest the next action to the user.

[0014] Furthermore, the log data analysis device 10 of this embodiment selects an appropriate combination of attributes when applying log data to a language model. Then, the log data analysis device 10 of this embodiment applies log data having the selected combination to the language model. By applying log data having the selected combination to the language model, the log data analysis device 10 of this embodiment can analyze log data using a combination that includes many behavioral features while reducing the amount of calculation. The specific configuration and operation of the log data analysis device 10 will be described in detail later.

[0015] The log data analysis device 10 according to this embodiment may be implemented as a general-purpose computer such as a server computer or a personal computer (PC).

[0016] 1, the log data analysis device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0017] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls the above components and performs various arithmetic processing in accordance with the program stored in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a log data analysis program that analyzes log data.

[0018] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0019] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0020] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may also function as the input unit 15 by adopting a touch panel system.

[0021] The communication interface 17 is an interface for communicating with other devices. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.

[0022] Next, the functional configuration of the log data analysis device 10 will be described.

[0023] FIG. 2 is a block diagram showing an example of the functional configuration of the log data analysis device 10. As shown in FIG.

[0024] 2, the log data analysis device 10 has, as functional components, an acquisition unit 101, a selection unit 102, and an analysis unit 103. Each functional component is realized by the CPU 11 reading out a log data analysis program stored in the ROM 12 or the storage 14, expanding it in the RAM 13, and executing it.

[0025] The acquiring unit 101 acquires log data to be analyzed by the log data analysis device 10. The acquiring unit 101 sends the acquired log data to the selecting unit .

[0026] The selection unit 102 performs a selection process on the log data sent from the acquisition unit 101. By performing the selection process on the log data sent from the acquisition unit 101, the selection unit 102 can suppress the amount of calculation in the analysis unit 103 and cause the analysis unit 103 to analyze the log data using a combination that includes many behavioral features.

[0027] The selection process performed by the selection unit 102 will now be described in detail.

[0028] 3 is a diagram showing an example of log data to be analyzed by the log data analysis device 10 of this embodiment. The log data analysis device 10 treats one record of the log data as one word in natural language and performs an analysis process of the log data using a language model.

[0029] Taking the actions of User A in the log data of Figure 3 as an example, the log data describes User A opening a campaign email (record 1), browsing a website by selecting a URL in the email (record 3), and clicking on content with content ID 1 on the viewed website (record 4). Also, taking record number 2 in the log data of Figure 3 as an example, the log data describes User B opening a campaign email (record 2) and browsing a website by selecting a URL in the email (record 5).

[0030] When analyzing log data such as that shown in Figure 3, it is desirable to select attributes to be used so that the content of user behavior can be identified at an appropriate granularity in accordance with the language model used for analysis. In the following explanation, the number of unique values ​​in the entire data when the value of a combination of attributes is considered as one word is referred to as the vocabulary size. The vocabulary size is an index of granularity. When analyzing log data, it is possible to use only one attribute or all attributes.

[0031] Figure 4A is a diagram showing an example of expressing user behavior using only "channel" as an attribute. When using only one attribute, say "channel" from the log data in Figure 3, the user behavior will be aggregated into "email" or "web," as shown in Figure 4A. However, when using only one attribute, the resolution of user behavior is insufficient, making it impossible to identify the user's characteristic behavior, and even if an analysis is performed, meaningful analysis results may not be obtained.

[0032] 4B is a diagram showing an example of expressing user behavior using all of the attributes "channel," "action," and each event. When all attributes are used, as shown in FIG. 4B, each user behavior can be expressed in detail, but the resolution of user behavior may be excessive, and many actions may be unique, making it impossible to obtain statistical trends. Furthermore, when all attributes are used, the calculation cost for the analysis process in the analysis unit 103 becomes enormous.

[0033] Therefore, in this embodiment, the selection unit 102 selects attributes to be used so that the granularity (number of vocabulary words) is appropriate for the language model used in the analysis using the language model in the analysis unit 103. FIG. 4C is a diagram showing an example of a representation of user behavior using the attributes "event 1" and "event 2." By selecting the attributes to be used for analysis in this way, the statistical trends of user behavior are appropriately reflected, making it suitable for analysis using a language model. Furthermore, by selecting the attributes to be used for analysis, the amount of calculation in the analysis process in the analysis unit 103 can be reduced compared to when all attributes are used.

[0034] A specific example of the selection process of the selection unit 102 is shown below. Fig. 5 is a diagram illustrating the selection process by the selection unit 102. The selection process described using Fig. 5 is also referred to as a simple method in the following description. In the log data shown in Fig. 3, there are seven candidate combinations of attributes that identify the behavioral content of the data source "email opening". (Event 1), (Event 1, Event 2), (Event 1, Event 3), (Event 1, Event 2, Event 3), (Event 2), (Event 2, Event 3), (Event 3)

[0035] The selection unit 102 performs a processing process to create a correspondence dictionary between the attribute values ​​that appear and the IDs for each combination of attributes in the log data. The selection unit 102 performs a similar processing process for other data sources. Figure 5 also shows the log data in which each attribute value has been replaced with an ID.

[0036] The selection unit 102 then selects a combination of attributes that is close to the target vocabulary size. The target vocabulary size may depend on the analysis method or analysis policy using the language model in the analysis unit 103. For example, when BERT4Rec is used as the language model, the target vocabulary size is set to 30,000. Of course, a larger or smaller target vocabulary size may be set in order for the analysis unit 103 to perform an analysis that captures user behavior in detail.

[0037] However, with the simple method shown in Figure 5, if the string length of each item in the log data or the pattern of attribute combinations is large, the dictionary information may become huge. In other words, with this method, the dictionary information may increase each time data is added. Furthermore, if the dictionary information cannot be stored in RAM 13, the dictionary information must be saved in storage 14 and full scans must be repeated, significantly reducing the processing performance.

[0038] Therefore, in this embodiment, the selection unit 102 updates the cardinality estimation object with the activity details that appeared for each combination of attributes in the log data. The cardinality estimation object refers to a data structure and a group of functions generated using technology such as HyperLogLog (HLL) and an improved version of HLL. By updating the cardinality estimation object with the activity details that appeared for each combination of attributes in the log data, the selection unit 102 can estimate the number of unique data items for the combination of attributes. In this embodiment, the log data is analyzed using a language model, so the number of unique data items can also be said to be the vocabulary size.

[0039] 6 is a diagram illustrating the selection process by the selection unit 102. The selection unit 102 performs a processing process to update the cardinality estimation object with the action content that has appeared for each combination of attributes. The selection unit 102 also performs a processing process to create a data structure for other data sources in the same way.

[0040] The selection unit 102 then estimates the vocabulary size using the cardinality estimation object for each combination. The selection unit 102 also uses a hash function to generate an ID for each attribute value, and also uses the hash value to update the HLL. Figure 6 also shows the log data in which each attribute value has been generated as an ID using the hash function.

[0041] When a cardinality estimation object is given a hash value of an attribute value of log data, such as a character string, the cardinality estimation object does not store the hash value itself, but instead uses a function to calculate the characteristics of the hash value and stores or updates the calculated statistics. When a hash value is considered as a bit string, HLL uses a statistical value that indicates how many consecutive zeros there are from the beginning. In this embodiment, since only the statistical value, not the hash value itself, is stored or updated, it is possible to limit memory consumption to a certain level, even for hash value patterns.

[0042] Furthermore, since the cardinality estimation object can be created for each fragment of user log data and then merged, it is easy to parallelize the vocabulary size estimation process for each attribute combination pattern.

[0043] FIG. 7 is a diagram illustrating a cardinality estimation method by the selection unit 102. Cardinality can also be expressed as the number of variations or the number of vocabulary words. For each combination of attributes, the selection unit 102 generates a hash value by assigning an ID to the behavior attribute value and using a hash function for cardinality estimation. The behavior attribute value refers to the value in the log data stored for each attribute. In this embodiment, FarmHash is used as the hash function, but other hash functions may also be used.

[0044] After generating the hash value, the selection unit 102 generates a cardinality estimation object for each combination of attributes.

[0045] For example, when estimating cardinality for only the behavior attribute value of one attribute, the selection unit 102 generates a hash value for the behavior attribute value using a hash function and performs cardinality estimation for the generated hash value. Also, when estimating cardinality for a combination of behavior attribute values ​​of two attributes, the selection unit 102 generates a hash value for the combined behavior attribute values ​​using a hash function and performs cardinality estimation for the generated hash value.

[0046] The selection unit 102 generates a cardinality estimation object for each combination of attributes, thereby obtaining an approximate cardinality value for each combination of attributes. Here, the selection unit 102 divides the input log data by row, thereby enabling distributed processing of hash value generation and cardinality estimation object updates. Then, the selection unit 102 merges the cardinality estimation objects after distributed processing, thereby enabling estimation of the cardinality of the entire log data.

[0047] The selection unit 102 assigns all hash values ​​of attribute values ​​obtained from the log data to the cardinality estimation object for each data source and for each enumerated attribute combination, and then calculates the estimated cardinality value from the statistical values ​​accumulated in each cardinality estimation object. When combining multiple attributes, the attribute values ​​are combined into a single string using a separator (e.g., a comma).

[0048] The total number of vocabulary words for all data sources is required to be equal to or less than the target vocabulary word count. Therefore, the selection unit 102 may divide the target vocabulary word count equally for each data source. For example, when the importance of each data source of the user behavior log or the content of each attribute value is unknown and analysis is carried out assuming that all data sources have equal value for the time being, the target vocabulary word count is divided equally for each data source. Figure 8 is a diagram showing an example of dividing the target vocabulary word count equally for each data source. Figure 8 shows the estimated vocabulary word count for each attribute combination.

[0049] For example, if the overall target vocabulary size is 30,000, and there are two types of data sources, the target vocabulary size for each is set to 15,000. Then, the selection unit 102 selects the largest attribute combination that does not exceed the target vocabulary size. In the example of Figure 8, the attribute combination of the record from the data source "email open" is selected as (event 1, event 2), and the attribute combination of the record from the data source "web browsing" is selected as (event 1, event 3).

[0050] Note that the selection unit 102 may allocate the vocabulary size arbitrarily rather than equally dividing it for each data source. For example, if the importance of each data source or the content of each attribute is known, the selection unit 102 may allocate the vocabulary size arbitrarily when adjusting the data source to express the user's behavioral characteristics. In other words, if the overall target vocabulary size is 30,000, and there are two data sources, the target vocabulary size for each data source may not be 15,000, but may be 10,000 for one data source and 20,000 for the other.

[0051] Furthermore, if the number of vocabulary words for a single attribute is close to the number of total records, the selection unit 102 may determine that most of the records are unique and likely have no statistical significance, and may ignore that attribute. Specifically, the selection unit 102 may ignore records where the ratio of the number of vocabulary words for a single attribute to the total records exceeds a threshold, and then allocate the target vocabulary word count based on the ratio of the maximum vocabulary word count for each data source (hereinafter also referred to as the "maximum vocabulary word count"). For example, if the importance of each data source or the content of each attribute value is unknown, but it is assumed that a data source with a larger vocabulary word count indicates user behavioral characteristics in more detail, the target vocabulary word count is allocated based on the ratio of the maximum vocabulary word count for each data source. Figure 9 is a diagram showing an example of allocating the target vocabulary word count based on the ratio of the maximum vocabulary word count for each data source. Figure 9 shows the estimated vocabulary word count for each combination of attributes.

[0052] For example, if the total number of records is 1 million and the threshold is 99%, records with a vocabulary count (unique count) for a single attribute that exceeds 99% of the total number of records, i.e., 990,000, are ignored. The strikethrough in Figure 9 indicates that the vocabulary count for a single attribute for "Event 4" is the same (100%) as the total number of records, so attribute combinations including Event 4 are not included in the comparison of maximum vocabulary counts. Furthermore, if the overall target vocabulary count is 30,000, the maximum vocabulary count for the data source "email open" that does not exceed the threshold is 180,000, and the maximum vocabulary count for the data source "web browsing" that does not exceed the threshold is 430,000. As a result of the selection unit 102 allocating based on the maximum vocabulary count ratio (180,000:430,000), the allocated vocabulary counts for each data source, calculated based on the ratio of the maximum vocabulary counts for "email open" and "web browsing," are 8,852 and 21,148, respectively. The selection unit 102 then selects, for each data source, the attribute combination that results in the maximum vocabulary count that does not exceed this maximum vocabulary count. In the example shown in Figure 9, for the data source "email opened", the combination of (event 1, event 2) is selected, and for the data source "web browsing" record attribute combination, the combination of (event 1, event 2, event 3) is selected.

[0053] The analysis unit 103 analyzes the log data based on the combination of attributes selected by the selection unit 102. As described above, the analysis unit 103 analyzes the log data using a language model such as word2vec, doc2vec, Transformer, or BERT. The analysis unit 103 uses the language model to analyze the log data based on the combination of attributes selected by the selection unit 102, thereby being able to analyze trends in user behavior from the log data.

[0054] By having such a configuration, the log data analysis device 10 can reduce calculation costs compared to when all attributes are used, and can analyze log data using combinations that include many behavioral characteristics.

[0055] Next, the operation of the log data analysis device 10 will be described.

[0056] 10 is a flowchart showing the flow of the log data analysis process by the log data analysis device 10. The CPU 11 reads out the log data analysis process program from the ROM 12 or the storage 14, loads it into the RAM 13, and executes it, thereby performing the log data analysis process.

[0057] First, in step S101, the CPU 11 acquires log data to be analyzed from the outside.

[0058] Following step S101, in step S102, CPU 11 selects a combination of attributes from the acquired log data. CPU 11 selects the attributes to be used so that the granularity (vocabulary size) is appropriate for the language model used for analysis using the language model. More specifically, CPU 11 updates the cardinality estimation object with the behavior content that appeared for each combination of attributes in the log data. Then, based on the result of updating the cardinality estimation object, CPU 11 selects a combination of attributes that is close to the target vocabulary size.

[0059] Following step S102, in step S103, the CPU 11 analyzes the log data using a language model for the selected combination of attributes.

[0060] Next, a specific example of the log data analysis process performed by the log data analysis device 10 will be described.

[0061] 11 is a diagram showing the structure of log data and the number of vocabulary words in a single attribute used to explain a specific example of the log data analysis process by the log data analysis device 10. Here, it is assumed that log data with a total of 1 million records is used.

[0062] 12 is a diagram for explaining a comparison between the case where log data is used as is, the case where the simple method shown in FIG. 5 is used, and the case where the method described in this embodiment is used. Note that the explanation will be given assuming that the target vocabulary size is 100,000 words.

[0063] If log data of 1 million records is used as is, the applicability (vocabulary size) of the language model will be 1 million records, i.e., 1 million words, and the data size will be 20 megabytes (MB).

[0064] When using a simple method, the applicability of the language model (vocabulary size) is 99,991 words, the data size is 8.8 MB, and the intermediate data size retaining the dictionary and vocabulary size information during processing is 98 MB. In other words, when using a simple method, the size may be larger than before processing. Furthermore, when using a simple method, all dictionary objects must be stored in memory. If dictionary objects cannot be stored in memory, it is inefficient, but one method is to store them on a hard disk (storage) instead of memory. However, this requires multiple searches of the data in storage, and maintaining performance becomes difficult as the data becomes large. Distributed processing could be considered to improve performance, but it would require merging the dictionary objects created for each data fragment into one, making it difficult to maintain processing performance.

[0065] In contrast, when the method described in this embodiment is used, the applicability of the language model (vocabulary size) is 99,990 words, and the data size is 16 MB, but the intermediate data size holding the dictionary and vocabulary size information during processing is 10 MB. When the method described in this embodiment is used, the intermediate data size is constant regardless of the size before processing. Furthermore, when the method described in this embodiment is used, a maximum of two objects must be held in memory, but the data size is smaller than that of the simple method. Furthermore, when the method described in this embodiment is used, even if cardinality estimation objects are generated in a distributed manner, they can be easily merged using low memory, and further, common objects are not required for distributed ID generation.

[0066] As described above, the method described in this embodiment can reduce the amount of data to be stored compared to when log data is used as is and when the simple method shown in Fig. 5 is used. Furthermore, by reducing the amount of data to be stored, the method described in this embodiment can reduce the amount of calculation compared to when log data is used as is and when the simple method shown in Fig. 5 is used. Furthermore, the method described in this embodiment can analyze log data using combinations that include many behavioral features by selecting a combination of attributes that is close to the number of vocabulary words suitable for the language model used for analysis.

[0067] In the above embodiments, the log data analysis process executed by the CPU after reading the software (program) may be executed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) such as field-programmable gate arrays (FPGAs), whose circuit configuration can be changed after fabrication, and dedicated electrical circuits such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to execute specific processes. The log data analysis process may be executed by one of these processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0068] In addition, in each of the above embodiments, the log data analysis program is described as being pre-stored (installed) in the storage 14, but the present invention is not limited to this. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0069] The following additional notes are provided regarding the above-described embodiments.

[0070] (Additional note 1) Memory and at least one processor coupled to said memory; Including, The processor: Acquire log data that describes user actions and one or more attributes associated with the actions; selecting the combination of attributes of the acquired log data based on the behavior of the log data and the unique number of combinations of attributes associated with the behavior; The selected combination of attributes of the log data is used to analyze the log data with a predetermined language model. A log data analysis device configured as follows.

[0071] (Additional note 2) A non-transitory storage medium storing a program executable by a computer to perform a log data analysis process, Log data analysis process: Acquire log data that describes user actions and one or more attributes associated with the actions; selecting the combination of attributes of the acquired log data based on the behavior of the log data and the unique number of combinations of attributes associated with the behavior; The selected combination of attributes of the log data is used to analyze the log data with a predetermined language model. Non-transitory storage medium. [Explanation of symbols]

[0072] 10 Log data analysis device 101 Acquisition Department 102 Selection section 103 Analysis Department

Claims

1. an acquisition unit that acquires log data in which user actions and one or more attributes associated with the actions are recorded; a selection unit that selects the combination of attributes of the log data acquired by the acquisition unit based on the behavior of the log data and the number of unique data in the combination of attributes associated with the behavior; an analysis unit that analyzes the log data using a predetermined language model by using the combination of the attributes of the log data selected by the selection unit; A log data analysis device comprising:

2. The log data analysis device according to claim 1 , wherein the selection unit estimates the number of unique data items for each combination of attributes using the content of the behavior that occurred, and selects the combination of attributes based on the result of the estimation.

3. 2. The log data analysis device according to claim 1, wherein the selection unit assigns a target number of records for analysis by the analysis unit for each of the actions based on a predetermined rule.

4. 4. The log data analysis device according to claim 3, wherein the selection unit equally divides the number of target records for analysis by the analysis unit for each of the actions.

5. 4. The log data analysis device according to claim 3, wherein the selection unit assigns a target number of records for analysis by the analysis unit based on a ratio of the maximum number of unique data for each of the actions.

6. The log data analysis device according to claim 1 , wherein the selection unit estimates the number of unique data items in a state where attribute values ​​of the log data items are hashed.

7. The processor: Acquire log data that describes user actions and one or more attributes associated with the actions; selecting the combination of attributes of the acquired log data based on the behavior of the log data and the unique number of combinations of attributes associated with the behavior; The selected combination of attributes of the log data is used to analyze the log data with a predetermined language model. A method for analyzing log data that performs processing.

8. A log data analysis program for causing a computer to function as the log data analysis device according to any one of claims 1 to 6.

Images