Analysis system, analysis method, and program
The analysis system uses a large-scale language model to generate explanatory texts and graphs, addressing the challenge of analyzing consumer purchasing behavior for marketing, enhancing understanding and providing actionable insights for inexperienced analysts.
Patent Information
- Application Number
- JP2024035814
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-03-08
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-03-08
AI Technical Summary
Analyzing consumer purchasing behavior for marketing purposes is challenging for inexperienced personnel due to the requirement of knowledge and experience, making it difficult to perform appropriate analysis.
An analysis system and method that utilizes a large-scale language model to compare customer groups based on attribute data, generating explanatory texts and graphs to assist in understanding the differences between groups, supported by a data generation application, an insight summary application, and an analysis chatbot.
Facilitates effective analysis of customer groups by providing concise explanatory texts and visual graphs, aiding analysts in understanding complex data without requiring extensive expertise, and offering actionable insights for marketing strategies.
Smart Images

Figure 0007579998000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an analysis system, an analysis method, and a program. [Background technology]
[0002] When analyzing consumer purchasing behavior for marketing purposes, clustering of consumers may be performed based on information including purchasing history. For example, Patent Document 1 discloses a clustering device that can group multiple consumers into multiple purchasing groups that show similar changes in purchasing behavior. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2014-106910 A Summary of the Invention [Problem to be solved by the invention]
[0004] Marketing personnel analyze the purchasing behavior of each of multiple purchasing groups. Such analysis requires knowledge and experience, so it can be difficult for inexperienced personnel to perform an appropriate analysis.
[0005] The present disclosure aims to provide an analysis system, an analysis method, and a program that can assist in analyzing customer groups. [Means for solving the problem]
[0006] An analysis system according to one embodiment of the present disclosure includes at least one memory configured to store a program and at least one processor configured to execute one or more processes based on content of the program, the one or more processes including a process of acquiring comparison data between a first group and a second group, the first group and the second group being each extracted from a plurality of customers included in customer data based on a user's instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution degree of each of the important attributes in the comparison; a process of inputting one or more instruction sentences for contrasting the first group with the second group into a large-scale language model together with the comparison data; and a process of outputting an explanatory sentence describing a comparison result of the first group with the second group based on an output of the large-scale language model.
[0007] An analysis system according to another aspect of the present disclosure comprises at least one memory configured to store a program and at least one processor configured to execute one or more processes based on the contents of the program, the one or more processes including: a process of extracting a first group and a second group from a plurality of customers included in customer data based on a user's instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; a process of generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes that are effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; a process of generating one or more graphs for the comparison data; a process of generating an explanatory text for the one or more graphs; and a process of outputting the explanatory text together with the one or more graphs.
[0008] An analysis method according to one embodiment of the present disclosure includes executing a process of acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user's instructions, the customer data including attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution degree of each of the important attributes in the comparison; a process of inputting one or more instruction sentences for contrasting the first group with the second group into a large-scale language model together with the comparison data; and a process of outputting an explanatory sentence describing a comparison result of the first group with the second group based on an output of the large-scale language model.
[0009] An analysis method according to another aspect of the present disclosure includes having one or more computers execute the following processes: extracting a first group and a second group from a plurality of customers included in customer data based on a user's instructions, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes that are effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; generating one or more graphs for the comparison data; generating an explanatory text for the one or more graphs; and outputting the explanatory text together with the one or more graphs.
[0010] A program according to one embodiment of the present disclosure causes one or more computers to execute the following processes: a process of acquiring comparison data between a first group and a second group, where the first group and the second group are each extracted from a plurality of customers included in customer data based on a user's instructions, the customer data includes attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data includes a plurality of important attributes that are effective for comparing the first group with the second group and a contribution degree of each of the important attributes in the comparison; a process of inputting one or more instruction sentences for contrasting the first group with the second group into a large-scale language model together with the comparison data; and a process of outputting an explanatory sentence describing a comparison result of the first group with the second group based on an output of the large-scale language model.
[0011] A program according to another aspect of the present disclosure causes one or more computers to execute the following processes: extracting a first group and a second group from a plurality of customers included in customer data based on a user's instructions, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes that are effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; generating one or more graphs for the comparison data; generating an explanatory text for the one or more graphs; and outputting the explanatory text together with the one or more graphs. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a schematic diagram of an analysis system according to an embodiment. [Diagram 2] Figure 2 is a graph showing the importance of attributes output as SHAP values. [Diagram 3] FIG. 3 is a graph showing the cumulative composition ratios of multiple attributes. [Figure 4] FIG. 4 is a graph showing the SHAP value for one attribute. [Diagram 5] FIG. 5 is a diagram illustrating an example of an input screen of the data generation application. [Figure 6] FIG. 6 is a diagram illustrating an example of a selection field for "brand and genre" on the input screen of FIG. [Figure 7] FIG. 7 is a diagram showing an example of a selection box for "genre" on the input screen of FIG. [Figure 8] FIG. 8 is a flow chart showing a method of generating data by a data generating application. [Figure 9] FIG. 9 is a diagram illustrating a result screen relating to "general statistics" of the data generation application. [Figure 10] FIG. 10 is a diagram showing an example of a result screen related to "life" of the data generation application. [Figure 11] FIG. 11 is a diagram illustrating detailed information about model accuracy of a data generation application. [Figure 12] FIG. 12 is a diagram illustrating detailed information about important features of a data generation application. [Figure 13] FIG. 15 is a graph showing the influence of each of several important attributes. [Figure 14] FIG. 14 is a diagram illustrating an example of a display screen of the explanation application. [Figure 15] FIG. 15 shows a summary result produced by the explanation application. [Figure 16] FIG. 16 is a flowchart showing a method for generating answer insights. [Figure 17] FIG. 17 is a diagram illustrating an example of a chat screen of the analytical chatbot. [Figure 18] FIG. 18 is a flowchart showing a method for generating and acquiring comparative data via an analytical chatbot. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Examples of the analysis system 11, analysis method, and program of the present disclosure will be described with reference to Figures 1 to 18. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.
[0014] [Overall configuration of the analysis system] 1, the analysis system 11 includes one or more analysis devices 20. Each analysis device 20 may be realized as a computer including, for example, one or more processors 21, one or more memories 22, and a communication interface (IF) 23. When the one or more analysis devices 20 include a plurality of analysis devices 20, some or all of the configurations of the analysis devices 20 may be different from each other.
[0015] A program 24 is stored in the one or more memories 22. The program 24 includes program code for an application and an operating system. The one or more processors 21 execute one or more processes based on the program 24 to realize various functions. The memory 22 may store comparison data 25 generated by the execution of the program 24.
[0016] The communication interface 23 enables communication with other devices via the network 12. The network 12 includes, for example, the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, and the like.
[0017] The analysis system 11 may include one or more machine learning devices 30. Alternatively, the analysis device 20 may include the functionality of the machine learning device 30. The one or more analysis devices 20 may be capable of communicating with the one or more machine learning devices 30 via the network 12. Each machine learning device 30 may be realized as a computer including one or more processors 31, one or more memories 32, and a communication interface (IF) 33. The communication interface 33 enables communication with other devices via the network 12.
[0018] The one or more memories 32 store a program 34. The program 34 includes an application and an operating system. The one or more processors 31 execute processes based on the program 34 to realize various functions. The memory 32 may store one or more learning models 35 generated by executing the program 34. When generating comparison data, the analysis device 20 operates the machine learning device 30 to generate the learning model 35 and execute a prediction calculation.
[0019] The processors 21 and 31 are, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), a field-programmable gate array (FPGA), or other arithmetic devices. The processors 21 and 31 are, for example, processing circuits configured to execute various software processes. The processing circuits may include dedicated hardware circuits (e.g., ASICs, etc.) that process at least a part of the software processes. In other words, the software processes may be executed by a processing circuitry that includes at least one of one or more software processing circuits and one or more dedicated hardware circuits.
[0020] The memories 22 and 32 may be non-transitory computer-readable media. The memories 22 and 32 may include, for example, a random access memory (RAM) or other volatile memory. The memories 22 and 32 may be configured to temporarily store programs and data. The memories 22 and 32 may include storage that permanently stores data including programs. The storage may be, for example, a read-only memory (ROM), a hard disk drive, a flash memory, or other non-volatile storage device. The storage may be a removable storage device such as a memory card. The communication interfaces 23 and 33 may be realized, for example, as a LAN or other wired communication IF.
[0021] The analysis system 11 may include a database 13. The database 13 may be held in a server device (not shown). Alternatively, the analysis device 20 may use data in the database 13 that is not included in the analysis system 11 via the network 12. Alternatively, the analysis device 20 or the machine learning device 30 may include a part or all of the database 13.
[0022] The analysis system 11 may include a large-scale language model 14. Alternatively, the analysis device 20 may use a large-scale language model 14 that is not included in the analysis system 11 through the network 12. The large-scale language model 14 is a natural language processing model trained using a large amount of text data. The large-scale language model 14 is a general-purpose language model that can be adapted to various natural language processing tasks, such as information extraction, text summarization, text generation, or question answering.
[0023] The large-scale language model 14 is configured to generate a sentence according to the instruction when a prompt including some instruction is input, and output the sentence as a completion. In this example, the analysis device 20 generates a prompt and inputs it to the large-scale language model 14, and then the analysis device 20 acquires the completion generated by the large-scale language model 14.
[0024] The user terminal 15 may be capable of communicating with the analysis device 20 through the network 12. The user terminal 15 may be, for example, a computer or a mobile terminal such as a tablet. The user terminal 15 may include a display 16. A user can use the functions of one or more applications provided by the analysis device 20 through the user terminal 15. Screens of the one or more applications are displayed on the display 16.
[0025] The one or more applications may be installed in the user terminal 15, or some or all of the functions may be provided as a Web application from the analysis device 20. The one or more applications may include at least one of a data generation application (customer profiling), an insight summarization application (explanatory AI agent), or an analytical chatbot (AI analyst). The analysis system 11 may include three analysis devices 20 for providing the data generation application, the insight summarization application, and the analytical chatbot, respectively.
[0026] [Customer insight analysis] Customer insight (also called consumer insight) generally refers to the motivation, true feelings, or psychology of customers that underlie their purchasing behavior. Analyzing customer insight makes it possible to carry out various marketing activities, such as reviewing or improving strategies for providing existing products or services, developing new products or services, and cultivating new markets. Customer insight is generally obtained by analyzing various information, including customer attribute information and customer purchasing history.
[0027] One example of analysis is comparing the insights of your brand's customer group with those of your competitors' customer group. Comparing the insights of two customer groups can provide clues for differentiating your company from your competitors. If you can provide products or services that customers potentially want based on this analysis, it can lead to differentiation from your competitors or the development of new customers.
[0028] The analysis device 20, particularly the data generation application, is configured to provide contrast data 25 for use in analyzing customer insights. In this disclosure, a consumer of a product or service is referred to as a "customer," a person in charge of marketing the product or service is referred to as an "analyst," and a person using the application to provide the contrast data 25 to the analyst is referred to as a "user." The "user" may also be the "analyst." A customer may include a potential customer, i.e., a consumer who has not purchased the product or service in question but may purchase it. The analyst may be a business providing the product or service, or a marketer contracted by the business to perform marketing.
[0029] The data generation application is configured to generate comparison data 25 based on customer data. The comparison data 25 includes, for example, data comparing two customer groups (in this example, a first group and a second group) created based on a certain analysis theme. The two customer groups include a first group that is the subject of analysis, and a second group that is the subject of comparison with the subject of analysis. The multiple customers included in the first group and the multiple customers included in the second group are each extracted from multiple (a large number of) customers included in the customer data.
[0030] For example, if the first group is a group of purchasers of a certain product, the second group may be a group of potential customers who have not purchased the product. Alternatively, if the first group is a group of viewers who have viewed an advertisement for a certain product, the second group may be a group of viewers who have viewed an advertisement for a product other than the first product.
[0031] A user can set any analysis theme depending on the purpose of the analysis. The analysis theme may include the names of two objects to be compared, such as the names of two groups, two product names, or two service names. For example, when comparing a customer group that prefers "lager beer" with a customer group that prefers "ale beer," the analysis theme can be set as "lager beer vs. ale beer" or "How is lager beer different from ale beer?".
[0032] The customer data may be stored in database 13. The multiple (large number of) customers may include customers of multiple types of products or services, customers of multiple brands, and customers of multiple business operators. For example, the multiple customers may include a customer group that uses each of one or multiple EC sites, and multiple business operators may provide products or services at one EC site. Alternatively or in addition to this, the multiple customers may include a customer group that participates in a common point program.
[0033] The customer data includes attribute data of each of a plurality of customers. In the present disclosure, the term "attribute" may refer to information (e.g., "age") including a combination of an "attribute" as a data item (e.g., "age") and an "attribute value" that is a value for the attribute (e.g., "age: 40 years old"). The attribute data may include static data items and dynamic data items related to the attributes of the customers.
[0034] Static data items include, but are not limited to, for example, date of birth, place of origin, date of first purchase, and date of start of use of the service. Dynamic data items include, but are not limited to, for example, name, age, address, family structure, annual income, occupation, income, employment status, living situation, hobbies, behavioral history, and interests. Behavioral history includes, but is not limited to, service usage history, product or service transaction history (purchase history), and website browsing history or search history. Interests include, but are not limited to, information related to products or services in which each customer is interested, and fields or genres of interest.
[0035] [Calculation of contribution using learning model] The one or more learning models 35 may be a prediction model that predicts whether each of a plurality of customers belongs to a first group or a second group based on customer data. The prediction model may be, for example, a random forest classifier. The one or more learning models 35 may include a plurality of learning models 35 generated using different learning data.
[0036] The learning model 35 is generated, for example, as follows. First, the customer data of each of the first and second groups is extracted as learning data, and then the learning data is divided into training data and test data. Then, a prediction model is generated using the training data. Then, the accuracy of the prediction model may be confirmed by evaluating the prediction model created using the test data. The accuracy of the prediction model can be evaluated using, for example, a receiver operating characteristic curve (ROC curve) or a precision-recall curve (PR curve). A prediction model that has a certain level of accuracy or higher is stored in the memory 32 as a practical learning model 35.
[0037] Each of the multiple attributes contained in the customer data can be a feature that changes the prediction result. When dealing with a learning model, the word "feature" may be used as a term that includes an "attribute" and its "value for the attribute." However, in this example, as described above, "attribute" is used as a term that includes not only a simple data item but also an attribute and an attribute value. Therefore, "attribute" is used as a term with the same meaning as "feature."
[0038] In the random forest, the contribution (importance) of each of the multiple features used in the training can be displayed. The data generation application may be configured to use the training model 35 to output the contribution of each attribute as part of the comparison data 25.
[0039] The contribution of each attribute may be calculated as the impact or importance of the feature, as a SHAP (SHapley Additive exPlanations) value, which is an application of the Shapley Value to machine learning. The SHAP value is used to quantitatively express the degree of contribution of each attribute used as a variable in prediction. It can be said that outputting the quantitative contribution of such attributes (features) is an additional function of the learning model 35.
[0040] In the analysis of customer insights, the SHAP value, i.e., the contribution of each attribute in the prediction, is required, rather than the prediction result of "whether the customer belongs to the first group or the second group," among the calculation results of the learning model 35. Therefore, it is advisable to extract customer data for a desired analysis period (for example, the sales period of the product to be analyzed) as learning data.
[0041] In this case, prediction may be performed while adding attribute data related to one attribute so that the feature amount included in the learning data increases by one, and the degree of influence of the added attribute data on the prediction result may be calculated successively. For example, if adding attribute data related to a certain attribute results in a clearer classification result, it can be said that the attribute has a strong influence on the prediction result. In this case, the SHAP value of the attribute becomes large.
[0042] In this way, the SHAP value indicates the degree of impact on the model output. More specifically, a SHAP value greater than 0 indicates a higher contribution, and a SHAP value less than 0 indicates a lower contribution. Therefore, an attribute with a larger SHAP value is a unique feature of the corresponding customer group, that is, an important attribute. The data generation application or learning model 35 may be configured to output graphs 17 to 19 as shown in FIGS. 2 to 4 as part of the calculation results.
[0043] FIG. 2 shows an example of a graph 17 in which the contributions of multiple attributes are output as SHAP values. In graph 17, the top 20 features with the highest influence among multiple attributes are displayed with SHAP values on the horizontal axis. The top 20 attributes are arranged from top to bottom in order of the greatest contribution. The top N attributes (N is a natural number) extracted in this way are called "multiple important attributes" that are useful for comparing the first group with the second group. The inverted triangular scale 17a on the left side of graph 17 tapers downward from the wide top side, indicating that the influence of the attributes decreases the further down it goes.
[0044] Graph 17 is also called the "SHAP bee swarm plot," and one plot (point) shows the SHAP value of one customer for each attribute. The color of each plot may indicate the relative scale of the values, for example, the closer to red the larger the value, and the closer to blue the smaller the value. In Graphs 17, 86, and 88, dark colors indicate blue and light colors indicate red. The vertical axis 17b of the "Feature value" on the right side of Graph 17, from "High" to "Low," is shown as a gradation that changes color continuously, from the highest end being red (gray in the figure) to the lowest end being blue (black in the figure), to show the color scale of the plot.
[0045] The horizontal position of each plot indicates the SHAP value for each customer feature (attribute), where a larger SHAP value (located to the right) indicates a larger positive contribution, and a smaller SHAP value (located to the left) indicates a larger negative contribution.
[0046] FIG. 3 shows an example of a graph 18 showing the cumulative composition ratio of SHAP values for a plurality of feature amounts. The graph 18 includes a bar graph showing the composition ratio of each of the plurality of feature amounts, and a line graph showing the cumulative ratio. When displaying some feature amounts with high influence among the plurality of feature amounts, it is also possible to display feature amounts up to 50% in terms of cumulative ratio. For example, in the graph 18 of FIG. 3, the top four feature amounts are the top 50% of feature amounts, that is, important feature amounts (important attributes).
[0047] FIG. 4 shows an example of a graph 19 indicating one SHAP value among a plurality of feature quantities. The feature quantity shown in FIG. 4 is a member rank in a point program or an EC site. The member rank is indicated, for example, from 0 to 7, with the larger the numerical value, the higher the member rank. When the member rank is set according to the usage of the point program or the EC site, the higher the usage (e.g., the larger the purchase amount), the higher the member rank. Graph 19 shows that the attribute "high member rank (member rank 6, 7)" contributes in a positive direction, and the attribute "low or medium member rank (member rank 1 to 5)" contributes in a negative direction.
[0048] [Data generation application] The data generation application is configured to generate comparison data 25 in response to a user input. The comparison data 25 mainly includes numerical data that is a result of calculation using a learning model 35. A series of processes executed in response to one user input or data generated for the series of processes is called a "job." Furthermore, the comparison data 25 output by one job is called "one dataset." A name (dataset name) may be automatically given to each dataset. The dataset name may be the same as the analysis theme.
[0049] The data generation application is configured to generate a new job when a new data set is output. The generated job and the data set generated by the job are stored in memory 22. A user can read one or more data sets generated in the past as existing jobs. When multiple users share one data generation application, a data set viewing authority may be set for each user so that the user can read jobs generated in the past by logging in to the data generation application.
[0050] 5 is an exemplary input screen 40 of a data generation application. The input screen 40 is an example, and the design can be changed as desired. The input screen 40 may include a sidebar 41 and a main column 42.
[0051] The sidebar 41 may include, for example, at least one of an account input field 43 for login, an action selection field 44, and a settings button 45. The sidebar 41 may include other input fields, selection fields, or buttons. In the data generation application, the action selection field 44 may be configured to display options of "Create new job" and "View existing jobs."
[0052] The data generation application may operate the machine learning device 30 to generate, as a default setting, a plurality of learning models 35 for each job. The plurality of learning models 35 may be generated using attribute data including a combination of prescribed attributes relating to each of "General Statistics", "Life", "Shopping", and "Attitude" as learning data, for example.
[0053] Attributes related to "general statistics" include, for example, gender, age, and transaction amount per specified period on a specified EC site (eg, GMS: Gross Merchandise Sales, total distribution transaction amount), but are not limited to these.
[0054] Examples of attributes related to "lifestyle" include, but are not limited to, educational background, employment status, residential status, whether or not a person has children, whether or not they are married, whether or not they own a vehicle and a license, and whether or not they play various sports. "Shopping" may include attributes related to the customer's transaction history (purchase history) of products, including, but not limited to, attributes related to the genre, brand, and purchase amount of products purchased in the past.
[0055] Examples of the "attitude" attribute may include various attitudes of customers in various areas including life and shopping. Examples of attitudes include, but are not limited to, "I prefer cashless shopping to cash," "I think online shopping saves time," "I like to earn points," "I think I have a sociable personality," "I don't want to use the same brand that other people have," and "I pay attention to my daily diet."
[0056] When the setting button 45 is selected, the data generation application may display one or more option setting fields 46-48. The setting field 46 may be a field for adding features. The setting field 46 may be a multiple selection type, and it is possible to add a combination of specified attributes such as "General Demography," "Interest," "Shopping Behavior," and "Service Usage," for example.
[0057] "Interest" is a combination of attributes related to what interests multiple customers have. Attributes related to interests may be, for example, the degree of interest in each category of products and services handled on the EC site, such as "interest in beauty and cosmetic products" or "interest in PCs and peripherals." Attribute data related to purchasing trends and service usage may be obtained from the transaction history of each category or brand of products and services handled on the EC site.
[0058] The setting field 46 may include, as an option, an "Add New" option that allows any combination of attributes to be freely added. When a combination of attributes is added through the setting field 46, the data generation application causes the machine learning device 30 to operate so as to generate a learning model 35 using learning data including the corresponding attribute data.
[0059] The setting field 47 may be a selection field related to the number of important attributes included in the output data, and may be, for example, a choice between "up to the top 20" and "within 50% cumulative composition ratio". Additionally or instead, the setting field 47 may allow the selection of an option to add detailed information about the important attributes. Furthermore, the setting field 47 may allow the setting of the number of important attributes for which detailed information is to be displayed. The setting field 48 may allow the setting of the display language, and may be, for example, a choice between "Japanese" and "English".
[0060] The main column 42 may include a setting area 50 for an analysis group, which is a first group, and a setting area 60 for a comparison group, which is a second group. The setting area 50 may include an analysis group name input field 51, a data selection field 52, a start date input field 53 for the start of the data, and an end date input field 54 for the end of the data. The setting area 50 may further include one or more selection buttons 55.
[0061] Similar to the setting area 50, the setting area 60 may include a comparison group name input field 61, a data selection field 62, a start date input field 63, an end date input field 64, and one or more selection buttons 65. The user can use the selection fields 52, 62 to select data for a desired number of customers from the entire customer data.
[0062] The selection field 62, input fields 63, 64, and selection button 65 of the setting area 60 may be displayed or made inputtable when input into the selection field 52 and input fields 53, 54 is completed in the setting area 50. When input into the input fields 53, 54 is completed, the same date as that in the input fields 53, 54 may be automatically input into the input fields 63, 64 as a default value.
[0063] The one or more selection buttons 55 may be input fields for arbitrarily selectable options. For example, the one or more selection buttons 55 may be radio buttons for alternatively selecting one of "Brand & Genre", "Brand", or "Genre". The one or more selection buttons 65 may be radio buttons for alternatively selecting one of "Brand & Genre", "Brand", and "Genre", including an option of "Random". When the option of "Random" is selected, data for a specified number of people is randomly extracted. The selection field 66 shown by the dashed line in FIG. 5 indicates that the selection field 66 is in an inoperable state due to the selection of "Random" by the selection button 65.
[0064] 5 shows all of the multiple input fields, selection fields, and buttons for ease of explanation, but the input fields, selection fields, or buttons may be displayed in sequence according to the processing order of the data generation application. For example, the action selection field 44 may be displayed after logging in by entering an account in the input field 43. Also, when "Create new job" is selected in the action selection field 44, name input fields 51 and 61 may be displayed. Furthermore, when the name input fields 51 and 61 are filled in, the selection field 52 and input fields 53 and 54 may be displayed.
[0065] 6 and 7 are display examples when the additional selection field 56, 66 is operated. When "Brand & Genre" is selected with the selection button 55, 65, options as shown in FIG. 6 may be displayed. When "Genre" is selected with the selection button 55, 65, options as shown in FIG. 7 may be displayed. When "Brand" is selected with the selection button 55, 65, a list of brand names may be displayed.
[0066] The genre options may include, but are not limited to, for example, "TV / Audio / Camera," "Sports / Outdoor," "Smartphone / Tablet," "Diet / Health," "PC / Peripherals," "Medicine / Contact Lenses / Nursing," and "Home Appliances." The genres may be the same as the categories of products or services handled on the EC site.
[0067] When one of multiple genres or brands is selected in selection field 56, customers included in the first group are extracted from multiple customers who have a transaction history of products or services of the selected designated genre or brand. Similarly, when one genre or brand is selected in selection field 66, customers included in the second group are extracted from multiple customers who have a transaction history of products or services of the selected designated genre or brand.
[0068] When "Brand & Genre" is selected with selection buttons 55, 65, a combination of brand and genre becomes selectable in selection fields 56, 66. For example, a brand name ("Brand Y" in FIG. 6) may first be selectable in selection fields 56, 66, and when one brand is selected, multiple genres belonging to the specified brand (multiple genres related to furniture in FIG. 6) may become selectable. Brand and genre may be selectable with separate selection buttons.
[0069] It may be possible to select multiple combinations of brands and genres. For example, in FIG. 6, "Brand X & Bookshelf" has been selected as the first brand and genre, and "Brand Y" has been selected as the second brand and genre, and multiple genres belonging to Brand Y are displayed as selectable. By selecting "Brand & Genre", customers included in the first group or the second group are extracted from multiple customers who have transaction history of the specified genre of the specified brand.
[0070] Genres and brands may be selectable from classifications across multiple hierarchies (e.g., three hierarchies). For example, in the case of brands, any hierarchy may be selectable from among the first hierarchy of company name (corporate brand), the second hierarchy of product category name (family brand), and the third hierarchy of product name (product brand). The hierarchy of genres may be the same as that of an EC site, for example.
[0071] 5, the main column 42 may include a confirmation button 59 and a decision button 69. When the confirmation button 59 is operated, a confirmation screen (not shown) showing the contents input on the input screen 40 may be displayed. The confirmation screen (not shown) may include a decision button 69. When the decision button 69 is operated, a process of generating data including the comparison data 25 is executed based on the instructions input by the user.
[0072] With reference to FIG. 8, a method of generating data via a data generation application will now be described. In step S1, processor 21 executes a process of extracting a plurality of customers belonging to a first group and a plurality of customers belonging to a second group from a plurality of customers included in the customer data. At this time, first customer data for a plurality of customers belonging to the first group and second customer data for a plurality of customers belonging to the second group are extracted from the entire customer data.
[0073] The customer data includes attribute data related to a plurality of attributes possessed by each customer. In step S1, the processor 21 extracts, as learning data, customer data corresponding to each of the plurality of learning models 35 to be generated. For example, the processor 21 extracts, from the customer data, learning data related to a combination of attributes additionally input by the user, in addition to learning data related to a combination of default attributes.
[0074] In step S2, the processor 21 causes the machine learning device 30 (processor 31) to generate a plurality of learning models 35 using the learning data. In step S3, the processor 21 causes the machine learning device 30 (processor 31) to execute a calculation (prediction) process using the learning model 35. This calculation includes a calculation for evaluating the accuracy of the learning model 35 and a calculation for calculating a SHAP value for each of a plurality of attributes included in the learning data. Steps S2 to S7 are executed for each of the plurality of learning models 35.
[0075] In step S4, the processor 21 obtains the calculation result of the learning model 35. In the following step S5, the processor 21 generates comparison data based on the calculation result. More specifically, from among the multiple attributes used in the calculation, the top N attributes are extracted as important attributes based on the SHAP value of each attribute. The number N is specified by default or is specified by user input. Therefore, the comparison data includes multiple important attributes effective for comparing the first group with the second group and a SHAP value indicating the contribution of each important attribute in the comparison.
[0076] In step S6, processor 21 generates explanatory text for the defined items and, if necessary, items specified by the user, based on the comparison data. In addition, in step S7, processor 21 generates one or more graphs to be paired with the explanatory text, based on the comparison data used when generating the explanatory text. In other words, the explanatory text is generated so as to describe an explanation for the corresponding graph. Steps S6 and S7 may be reversed in order, or may be executed simultaneously. In step S8, processor 21 outputs generated data including the generated explanatory text and graphs.
[0077] 9 illustrates a result screen 70 of a data generation application for displaying generated data. The result screen 70 may display a title 70a of the corresponding job. The title 70a may be an analysis theme including a first group name and a second group name. The result screen 70 may also include a selection field 70b for selecting a job to display. Each of the lists displayed in the selection field 70b may display the name of the job and the creation date and time of the job.
[0078] The result screen 70 may include a navigation column 71 at the top. The navigation column 71 may include multiple tabs, such as, for example, "General Statistics," "Life," "Shopping," "Attitude," "More," and "Take Action." The "Life," "Shopping," "Attitude," and "More" tabs may display the calculation results of multiple corresponding learning models 35.
[0079] "Life" may include contrast data related to a customer's life stage. "Shopping" may include contrast data related to each of a customer's search history, page browsing history, transaction history, and campaign participation history, for example. "Attitude" is contrast data related to a customer's attitude. "Other" may include contrast data including all attributes, for example, or may include contrast data related to an attribute selected by the user. Since contrast data corresponding to multiple tabs are displayed in a similar interface, FIG. 10 illustrates an example of contrast data related to "Life," and omits the display of contrast data corresponding to other tabs.
[0080] Operating each tab may jump to an area (shown in Figs. 9 and 10) displaying the corresponding results. Also, scrolling down the results screen 70 may sequentially display the results corresponding to each of the multiple tabs.
[0081] When the "General Statistics" tab is operated, a display area 72 comparing the first group and the second group may be displayed. The display area 72 may include, for example, but is not limited to, a graph 72a comparing the number of customers in both groups, a graph 72b comparing gender distribution, a graph 72c comparing age distribution, and a graph 72d comparing usage of the EC site. For example, the graph 72d may be displayed when the customer data includes data on customers who use the EC site.
[0082] Graph 72d shows the distribution of GMS (total merchandise transaction amount) during the data acquisition period as data on the transaction amounts of customers constituting each of groups 1 and 2. The legend of graph 72d is, for example, "no purchase," "low GMS: less than 10,000 yen," "medium GMS: between 10,000 yen and 100,000 yen," and "high GMS: over 100,000 yen."
[0083] As shown in Fig. 10, the result screen 70 may include buttons 75, 76, and 77 at the bottom for displaying detailed information related to the learning model 35. Operating button 75 displays detailed information about the model accuracy. Operating button 76 displays detailed information indicating important features in the model. Operating button 77 displays detailed information about the influence that top features have on the model results.
[0084] Operating a tab in the navigation column 71 may display a display field 70c that displays information related to the model accuracy of the corresponding learning model 35. "Model Confidence" shown in the display field 70c indicates the reliability of the model. The reliability may be determined, for example, by the value of the AUC (area under ROC curve), with AUC>75 being high (HIGH), AUC>0.6 being medium (MEDIUM), and other values being low (LOW). Operating the button 75 displays detailed information about the model accuracy (see FIG. 11).
[0085] As shown in FIG. 11 , the detailed information about the model accuracy may include a graph 75a showing an ROC curve. The area under the ROC curve (AUC) is used as an evaluation index of the corresponding learning model 35. The detailed information about the model accuracy may include a confusion matrix 75b of the learning model 35. The confusion matrix provides information showing the agreement and disagreement between the predicted values and the actual values of the model. The confusion matrix includes four values: True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN).
[0086] 10, the screen showing the comparative data for "Lifestyle," "Shopping," "Attitude," and "Other" may include a graph display area 73 and an insight display area 74. The graph display area 73 may include a selection button 73a for selecting the display format of the graph, and a graph 73b displayed in the selected format. The selection button 73a may include a choice between a "SHAP bar plot" showing SHAP values as a bar graph, and a "SHAP bee swarm plot" which is a graph plotting SHAP values as points.
[0087] In the "SHAP bar plot" graph 73b, the SHAP values of each customer are converted into a single summary value for the top 20 influential features (important attributes) among multiple features, and are displayed as a bar graph. This makes it clear to what extent each feature contributes in either a positive or negative direction. The "SHAP bee swarm plot" graph 73b is a graph in the format shown in Figure 2, and important attributes (for example, the top 20 features) are displayed in order of their influence.
[0088] The insight display area 74 displays a character string outlining the results shown in the graph 73b. As an example, the insight display area 74 may display explanatory text for the comparison data, particularly the top 5 important attributes among the top 20 important attributes shown in the graph 73b. That is, the explanatory text may describe a smaller set number of important attributes among the multiple important attributes. This set number may be specified by a user input in the setting field 47, or if not specified, a default set number may be used.
[0089] The character string displayed in the insight display area 74 may be automatically generated from the data of the analysis result and a prepared template. For example, the processor 21 may generate a sentence such as "It is highly likely that they have children in high school" by combining the name of the attribute (e.g., "They have children in high school") with a character string indicating the result "It is highly likely that they have children in high school" for the top five feature amounts with the highest SHAP values.
[0090] FIG. 12 illustrates a graph 76a showing the cumulative composition ratio of the SHAP value, which is displayed when the button 76 is operated. The graph 76a has the same format as the graph 18 shown in FIG. 3. In the graph 76a, the top 20 feature amounts (important attributes) are arranged from top to bottom in order of the highest composition ratio. Therefore, the graph 76a shows how important each feature amount is, that is, which are the important attributes. For example, the top five feature amounts can be said to be particularly important attributes among the important attributes.
[0091] Operating button 77 may display one or more graphs 78 (see FIG. 13 ) showing how each of the most important attributes affects the model results. Here, the number of top features to display may be set in setting field 47.
[0092] 13 shows graphs 78a, 78b, 78d, and 78e that show the influence on the model results of the top five feature amounts mentioned in the insight display area 74. Each of the graphs 78a to 78e includes a plurality of plots (points) corresponding to a plurality of customers, with the horizontal axis representing the first to fifth ranked feature amounts and the vertical axis representing the SHAP value for each feature amount.
[0093] The "Action" tab can be used to perform additional analysis for marketing based on the above-mentioned contrast data. In the additional analysis, for example, an additional learning model 35 may be generated to extract customers who have attributes common to customers in the target group from among customers outside the target group. The additional learning model 35 is constructed to predict customers who have similar attributes similar to the important attributes of the first group. Such additional learning model 35 can extract a group of potential customers to be targeted for marketing.
[0094] [Description application] Although a simple explanation of the comparison data is displayed in the insight display area 74 of the data generation application, the rest is displayed as numerical data or graphs. Therefore, the analyst must interpret the numerical data and graphs to perform a detailed analysis.
[0095] Since such analysis requires knowledge and experience, it is difficult for an inexperienced analyst to perform an appropriate analysis. Even for an analyst with some experience, it is time-consuming to interpret the results from multiple graphs. Therefore, the explanation application is configured to summarize the comparison data and answer questions about the comparison data.
[0096] 14 is an exemplary display screen 80 of an explanation application. The display screen 80 is an example, and the design can be changed as desired. The display screen 80 may include a sidebar 81 and a main column 82.
[0097] The sidebar 81 may include, for example, a login user name input field 83 and a dataset selection field 84. The dataset selection field 84 lists the names of datasets that each user has previously created. When a user logs in by inputting a user name in the input field 83, the user can select, in the dataset selection field 84, a dataset of a job that the user has previously created.
[0098] The main column 82 may display the name of the data set selected by the user. The main column 82 may include a "Q&A" tab 82a and a "Summary" tab 82b. When the "Q&A" tab 82a is selected, a question input field 85 is displayed in the main column 82. When a question is input in the question input field 85, an answer insight 87 that is an answer to the question is displayed below the question input field 85 together with a graph 86. The answer insight 87 and the graph 86 are generated based on comparison data related to the same important attribute. That is, the explanation application is configured to output the answer to the question in the form of a sentence (character string) when a question related to the comparison data is input.
[0099] When the "Summary" tab 82b is selected, the main column 82 displays summary insights 89 (see FIG. 15) summarizing the results of the data set (e.g., "Lager vs. Ale"), i.e., the explanation application is configured to generate an explanatory text summarizing the results of the contrast data.
[0100] As shown in Figure 15, summary insights 89 may be displayed in the main column 82 along with a corresponding graph 88. The summary insights 89 and graph 88 are generated based on contrast data relating to the same important attributes. Graph 88 is an example of a "SHAP bee swarm plot" graph, but may also be a "SHAP bar plot" graph, or multiple graphs may be displayed.
[0101] Summary insights 89 may be generated for each of the items of “General Statistics,” “Lifestyle,” “Shopping,” and “Attitude” included in the comparison data 25. Although summary insights 89 for “Lifestyle” are illustrated in FIG. 15, summary insights 89 for other items may also be included in the main column 82.
[0102] [How Insights are Generated] The insights 87 and 89 may be generated using a general-purpose large-scale language model 14. The insights 87 and 89 may be described separately for each of the first and second groups, or may be described together to describe the differences between the two groups.
[0103] The memory 22 of the analysis device 20 may store one or more templates of prompts for input to the large-scale language model 14. Here, a prompt for obtaining answer insights 87 is referred to as a question prompt, and a prompt for obtaining summary insights 89 is referred to as a summary prompt. The one or more templates may include a question template used to generate a question prompt and a summary template used to generate a summary prompt.
[0104] The prompt preferably includes at least one of a setting, an instruction, and / or a source of data to enable the large-scale language model 14 to generate an appropriate completion. Thus, a template may include at least one of one or more setting statements, one or more data specification statements, and one or more instruction statements.
[0105] The prompt and template may be written in a language (e.g., English) that is often included in the training data in the large-scale language model 14. In this case, the prompt and template may include an instruction to generate insights 87, 89 in the description language of the question sentence entered in the question input field 85 or the language set in the setting field 48, and to translate the generated sentence into the target language.
[0106] The insights 87 and 89 can be said to be documents that should be generated by an analyst by looking at the numerical data included in the comparison data or various graphs (e.g., graphs 86 and 88) generated from the numerical data. Therefore, the prompt may include a setting sentence for setting a character as an analyst to the large-scale language model 14. For example, the one or more setting sentences may include a setting sentence for causing the analyst to respond as an analyst, such as "You are a data scientist."
[0107] The one or more data specification statements may include a statement specifying the data to be analyzed, such as "Please analyze [data name]," and the prompt may include data corresponding to the data name. The data included in the prompt may be numerical data such as a SHAP value, or a descriptive text (character string) generated based on the numerical data.
[0108] [Data Name] may be, for example, the name of a data set or the name of an analysis item such as "General Statistics," "Lifestyle," "Shopping," or "Attitude." The data name may correspond to the data in graphs 86 and 88 to be displayed together with insights 87 and 89. A sentence specifying the data may be included in the setting sentence, such as "You are the analyst who will analyze [Data Name]."
[0109] The one or more instruction statements may include multiple action instructions to be performed by the large-scale language model 14. In this case, the one or more instruction statements may include a general instruction statement such as "Your task is to perform the following actions" and multiple action instructions each corresponding to a multiple action.
[0110] The prompt or template may include, for example, a series of action instructions such as (1) to (4) below, but is not limited to these. (1) Please summarize the top three contributing features.
[0111] (2) Regarding the top three features with the highest contribution, please explain why those features have such a high contribution. (3) Based on the results of (1) and (2) above, please suggest some effective marketing methods.
[0112] (4) Please summarize the results of (1) to (3) above. In addition to a simple summary like (1), action instructions can provide more useful information to analysts by including an analysis of factors like (2) and even a marketing method suggestion like (3). Furthermore, as the number of action instructions increases, the text becomes longer and more complex, so adding a text summarizing the output content like (4) can help analysts better understand the results.
[0113] Before generating a question prompt, the explanation application is configured to extract data or attributes to be used as an information source based on a question sentence entered in the question input field 85. Hereinafter, a method for generating an insight 87 that is an answer to the question sentence will be described with reference to Fig. 16.
[0114] In step S11, the processor 21 acquires a question sentence relating to a comparison between the first group and the second group input by the user. In the following step S12, the processor 21 selects one or more related attributes related to the question sentence from among a plurality of attributes. In step S12, for example, a natural language processing model such as the large-scale language model 14 may be used to select attributes that have a high similarity to the question sentence or words contained in the question sentence as the related attributes.
[0115] For example, the processor 21 may input a prompt to the large-scale language model 14 to instruct it to select the top few (e.g., 20) attributes related to the question sentence. In this case, the prompt may include an explanatory text for each of the multiple attributes as an information source for determining the relevance. The explanatory text may be generated from the analysis result data and a pre-prepared template text, such as the character string displayed in the insight display area 74.
[0116] In this case, the large-scale language model 14 selects and outputs multiple top attributes that are highly related to the question sentence without considering the SHAP value. The processor 21 may then exclude attributes with a SHAP value of 0 from the selected multiple top attributes and set the remaining multiple top attributes as related attributes, because attributes with a SHAP value of 0 do not contribute to the comparison of the target group with the comparison group.
[0117] In step S13, the processor 21 obtains the SHAP value (degree of contribution) of the selected related attribute. When there are multiple related attributes, in step S14, the processor 21 selects a prescribed number of related attributes with high degrees of contribution as important attributes. For example, the processor 21 may select the top N related attributes with high positive correlation and the top M related attributes with high negative correlation as important attributes. The prescribed numbers (the values of N and M) can be set arbitrarily, and may be, for example, a default value prescribed in the explanation application or a specified value designated by the user. N and M may be the same value (for example, 1, 2, 3, or 5) or may be different values. For example, the value of M may be larger or smaller than the value of L.
[0118] In step S15, the processor 21 creates a question prompt. At this time, the processor 21 may generate the prompt using a question template including the setting sentence as described above. Here, the data specification sentence may include, as information sources, one or more important attributes selected in step S14 and the contribution degree (which may be a SHAP value) of each important attribute. The information source may be an explanatory sentence for one or more important attributes instead of the contribution degree (SHAP value). This explanatory sentence may be a character string generated from the data of the analysis result and a fixed sentence, such as input to the large-scale language model 14 in order to select the related attribute in step S12. In addition, the instruction sentence includes an action instruction sentence for answering the question, which is generated using, for example, a natural language processing model.
[0119] In step S16, the processor 21 inputs the generated prompt to the large scale language model (LLM) 14. Then, in step S17, the processor 21 acquires a completion output by the large scale language model 14. Furthermore, in step S18, the processor 21 may edit the completion to generate a reply sentence.
[0120] For example, if the completion is written in English, analysis device 20 may use another machine translation model to translate the completion into the same language as the question (e.g., Japanese). In addition, the answer may be generated by adding a note about the accuracy of the analysis result to the completion or adding the date and time when the answer was generated.
[0121] In step S19, processor 21 outputs an answer sentence generated based on the output of large-scale language model 14 as answer insight 87. This answer sentence is a sentence including an answer to the question, and is also an explanatory sentence describing the result of comparing the first group with the second group. In step S19, an answer sentence written in both the description language of the completion (e.g., English) and the description language of the question sentence (e.g., Japanese) may be output. In this case, insight 87 may be written in both the description language of the completion and the description language of the question sentence.
[0122] [Analysis chatbot] 17 illustrates an example of a chat screen 90 of the analytical chatbot. The analytical chatbot is configured so that a virtual analytical assistant, an "AI analyst," answers questions from a user in a chat format.
[0123] The analytical chatbot is configured to analyze a question sentence and generate an answer sentence by using the large-scale language model 14. When generating an answer sentence, the analytical chatbot may obtain contrast data related to the question as in steps S11 to S19, and include the contrast data in the prompt as an information source of the answer.
[0124] The chat screen 90 includes an input field 91 for the user to input a question, and a send button 92. The chat screen 90 further includes one or more bot speech bubbles 93 which are comment fields for the AI analyst, and one or more user speech bubbles 94 which reflect the text input in the input field 91. When the user inputs text in the input field 91 and operates the send button 92, a user speech bubble 94 including the input text is added to the chat screen 90. The analytical chatbot outputs a response to the question input by the user to the bot speech bubble 93 as a comment from the AI analyst.
[0125] As a first example, the analytical chatbot may generate an answer based on a dataset already generated by a data generation application. In the first example, the analytical chatbot may output a bot speech bubble 93 including a list of datasets on the chat screen 90 together with an instruction such as "Please select a dataset to be analyzed." Alternatively, the analytical chatbot may obtain corresponding comparative data as an information source in response to a question.
[0126] As a second example, the analytical chatbot may generate a new data set and then generate an answer for the data set. In this case, the analytical chatbot may instruct the user to input two customer groups corresponding to the first group and the second group, and generate comparative data for the two input customer groups. In the second example, the analytical chatbot may output a bot speech bubble 93 including a "Create new job" button along with a question such as "Do you want to create a new job?" to the chat screen 90.
[0127] Hereinafter, with reference to FIG. 18, a method for the analytical chatbot to acquire already created comparison data as in the first example, or generate new comparison data as in the second example, will be described. Note that the function (program code) for realizing the method described here may be the same as the function (program code) when the data generation application performs the process of acquiring a data set and creating a new job. Therefore, the analytical chatbot may share the program for executing these processes with the data generation application. In addition, the analytical chatbot may have the user sequentially input items to be input on the input screen 40 in a chat format.
[0128] As shown in FIG. 18, the processor 21 may determine whether to generate a new job in step S21 based on a user's response to a question such as "Do you want to generate a new job?".
[0129] If step S21 is a negative determination (NO), in step S22, the processor 21 may display options of created datasets together with an instruction such as "Please select a dataset to be analyzed." Next, in step S23, the processor 21 acquires the dataset (comparison data) selected by the user.
[0130] Instead of step S22, the processor 21 may display a question such as "What kind of data do you want to analyze?" and obtain a free-form answer from the user. In this case, the processor 21 may extract relevant attributes, important attributes, or a data set including them related to the question, as in steps S11 to S14.
[0131] If step S21 is answered in the affirmative (YES) manner, the processor 21 outputs an instruction such as "Please input the names of the analysis target and the comparison target" to thereby obtain the first group name and the second group name in step S24.
[0132] In step S25, processor 21 generates a new job. Furthermore, processor 21 causes the user to input conditions for data generation as necessary, and then generates comparison data 25 in step S26. The generation of comparison data 25 in step S26 can be performed in the same manner as steps S1 to S8 in FIG. 8.
[0133] In step S27, the processor 21 stores the generated comparison data in the memory 22. Thereafter, when a question about the generated comparison data is input from the user, the processor 21 acquires the corresponding comparison data in step S28 and sets it as an information source of the answer.
[0134] Thereafter, the analytical chatbot may output an answer in response to a request input by the user based on the acquired comparison data. For example, when a question about certain comparison data is input, the analytical chatbot may generate an answer sentence in the manner shown in Fig. 16. In this case, the analytical chatbot may share a program for executing the process shown in Fig. 16 with the explanation application.
[0135] More specifically, the analytical chatbot may select a feature with a high contribution from the features related to the question, and input a prompt to generate an explanation for the feature to the large-scale language model 14. Then, the analytical chatbot may output an answer sentence generated based on the completion through the bot speech bubble 93. The analytical chatbot may also output a graph showing the comparison data that was the source of the answer together with the answer sentence.
[0136] [Effects of the present disclosure] The analysis device 20 can generate an explanatory text based on the calculated numerical data. When the explanatory text is a concise description of important attributes as displayed in the insight display area 74, it is useful for understanding the outline of the comparison data. When the explanatory text includes an analysis of factors or a proposal of a marketing method, as in the summary insight 89, the analysis device 20 can help the analyst with his / her work in a more multifaceted manner. Furthermore, when the explanatory text is output as a response to an open-ended question, as in the answer insight 87 or an analytical chatbot, the analysis device 20 can provide information that meets the analyst's needs.
[0137] The analysis device 20 can display explanatory text for the comparison data together with a graph generated from the same data as the source. Therefore, the analyst can read the explanatory text and understand it linguistically while visually recognizing the graph. Although the cognitive characteristics of which is easier to recognize, visual information or linguistic information, differ depending on the analyst, many analysts have complex cognitive characteristics. Therefore, by displaying a set of visual information, i.e., a graph, and explanatory text, which is linguistic information, it is possible to help the analyst understand.
[0138] Such explanations can also be generated using a large-scale language model 14, such as an answer insight 87 or an analytical chatbot. However, there is a major problem in that the output of a generative AI using a general large-scale language model is often inaccurate. Therefore, the analysis device 20 can output a more accurate answer by including contrast data, which is the source of the answer, in the prompt.
[0139] Since there is a limit to the token length that can be entered as a prompt, it is not possible to include all of the customer data in the prompt. Also, it is difficult to base an answer on simple customer data alone. In this respect, the contrast data can be a concise yet useful source of information, since it includes one or more important attributes characteristic of the classification between the first group and the second group and the contribution of each important attribute.
[0140] [Effects of this disclosure] According to the present disclosure, the following effects can be achieved. (1) The analysis device 20 can input the comparison data together with the instruction sentence into the large-scale language model 14, and output an explanatory sentence describing the comparison result of the first group with the second group in addition to numerical data such as the contribution degree of multiple attributes. This can help an analyst who looks at the comparison data and analyzes the two customer groups, the first group and the second group.
[0141] (2) The summary insight 89 and the graph 88 are generated based on the same comparison data. Furthermore, since the summary insight 89 is displayed together with the corresponding graph 88, an analyst can more easily understand the data shown in the graph 88 by reading the summary insight 89.
[0142] (3) The answer insight87 of the explanation application or the analytical chatbot outputs customer insights as answers to the user's questions. Therefore, the user or analyst can understand the contents of the comparative data by reading the explanation text without having to interpret the numerical data and graphs included in the comparative data.
[0143] (4) The answer insight 87 and the graph 86 are generated based on the same comparison data. Furthermore, since the summary insight 89 is displayed together with the corresponding graph 88, the analyst can more easily understand the data shown in the graph 88 by reading the summary insight 89.
[0144] (5) The analysis device 20 generates comparative data of the first group with the second group, and outputs an explanatory text along with one or more graphs of the comparative data. The explanatory text can help an analyst to better understand the data shown in the one or more graphs, thereby assisting the analyst in analyzing the two customer groups, the first group and the second group, by looking at the comparative data.
[0145] (6) Graphs 73b, 86, and 88 show multiple important attributes and the contribution of each of the multiple important attributes, so that the analyst can know the important attributes of the customers that lead to the difference between the first group and the second group, and further understand the degree of influence of these important attributes.
[0146] (7) In the graphs 73b and 88, the important attributes are arranged in descending order of their contribution. Therefore, by looking at the order of the important attributes, the user can intuitively understand the relationship between them and the degree of their influence.
[0147] (8) A plurality of customers have transaction histories of at least one of products or services across a plurality of categories. The user can specify, via the input screen 40, to extract at least one of the first group and the second group from among a plurality of customers who have transaction histories of products or services in a specified category among the plurality of categories. In this case, comparison data for customers in the specified category can be output.
[0148] (9) At least some of the multiple customers have transaction histories of one or more brands that provide products or services across multiple categories. The user can then specify, via the input screen 40, to extract at least one of the first and second groups from the multiple customers who have transaction histories of products or services in one designated category provided by one designated brand. In this case, comparison data on customers in the designated category of the designated brand can be output.
[0149] (10) The attribute data includes the transaction amount of each customer's product or service. The graph 72d displays data on the transaction amount (e.g., GMS) of the customers in the first and second groups in comparison. By outputting such a graph 72d, the analyst can intuitively understand the consumption trends of the purchasers in each of the first and second groups.
[0150] (11) Based on the contribution of each attribute obtained as a calculation result of the learning model 35, it is possible to estimate, from among multiple attributes, an important attribute that separates the first group from the second group.
[0151] (12) The comparison data includes the calculation results of a plurality of learning models 35, and the plurality of learning models 35 are generated using attribute data relating to different combinations of attributes. The combination of attributes may be a set combination such as general statistics, lifestyle, shopping, and attitude, or may be a combination selected arbitrarily by the user. By selecting characteristic combinations of attributes or related combinations of attributes from a large number of customer attributes and using them as learning data, the first group and the second group can be compared from multiple perspectives.
[0152] (13) By using the SHAP value, the contribution can be quantitatively calculated. [Example of change] This embodiment can be modified as follows: This embodiment and the following modifications can be combined with each other to the extent that there is no technical contradiction.
[0153] [Change Example 1] Various processes for realizing the data generation application, the insight summarization application, and the analytical chatbot may be distributed and executed by multiple analysis devices 20. In this case, not only may the three analysis devices 20 individually realize the data generation application, the insight summarization application, and the analytical chatbot, but also a plurality of processes required for realizing the three applications may be distributed and executed by any number of analysis devices 20.
[0154] [Change Example 2] When all the processes for realizing the data generation application, the insight summarization application, and the analytical chatbot are realized by one analysis device 20, the analysis system 11 is realized by one device (e.g., a computer). In this case, the one analysis device 20 may include the large-scale language model 14 and the machine learning device 30.
[0155] [Change Example 3] The first group and the second group may each include the same customer group. In this case, by setting different data periods for the first group and the second group, comparative data on the trends of the same customer group in two periods can be obtained. For example, when a designated brand sells different new products every season or year, such data can be used to analyze the purchasing trends of the same customer group for those products.
[0156] [Change Example 4] The data generation application may have the function of an insight summarization application. For example, an explanatory text (summary insight) generated using the large-scale language model 14 in the same manner as the insight summarization application may be displayed in the insight display area 74 of the data generation application. Alternatively, the main column 82 may be displayed by operating the "action" tab on the result screen 70. According to this configuration, after generating contrast data, answer insights 87 and summary insights 89 for the contrast data can be obtained using the same data generation application. In addition, each of the data generation application, the insight summarization application, and the analytical chatbot may share some or all of the functions of each other.
[0157] [Change Example 5] The analysis device 20 may not include one or more of the functions of the data generation application, the insight summarization application, and the analytical chatbot. If the analysis device 20 does not include a data generation application, the insight summarization application and the analytical chatbot may acquire the comparison data already generated. In this case, the analysis system 11 may not include a machine learning device 30 for generating the learning model 35.
[0158] [Change Example 6] An insight summarization application may provide only answer insights 87 or summary insights 89 .
[0159] Aspects grasped based on the above embodiment and modified examples are listed below. [1] at least one memory configured to store a program; and at least one processor configured to execute one or more processes based on the content of the program, The one or more processes include a process of acquiring comparison data between a first group and a second group, the first group and the second group being extracted from a plurality of customers included in customer data based on a user's instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; outputting an explanatory text describing a comparison result between the first group and the second group based on an output of the large-scale language model; 16. An analysis system comprising:
[0160] [2] The one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text; The analysis system according to [1] above.
[0161] [3] The one or more processes are acquiring a question input by the user regarding a comparison between the first group and the second group; selecting one or more relevant attributes from the plurality of attributes that are related to the question; selecting a predetermined number of related attributes having a high contribution rate from the one or more related attributes; Including, In the process of acquiring the comparison data, the predetermined number of related attributes are the plurality of important attributes; In the process of inputting to the large-scale language model, the one or more instruction sentences include an instruction sentence for answering the question, In the process of outputting the explanatory text, the explanatory text includes an answer to the question. The analysis system according to [1] above.
[0162] [4] The one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text. The analysis system according to [3] above.
[0163] [5] at least one memory configured to store a program; and at least one processor configured to execute one or more processes based on the content of the program, The one or more processes include A process of extracting a first group and a second group from a plurality of customers included in customer data based on an instruction from a user, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution degree of each of the important attributes in the comparison; generating one or more graphs for the contrast data; generating a legend for the one or more graphs; outputting said legend together with said one or more graphs; 16. An analysis system comprising:
[0164] [6] The one or more graphs include a graph showing the plurality of important attributes and the contribution degree of each of the plurality of important attributes. An analysis system according to any one of the above [2][4][5].
[0165] [7] The plurality of important attributes are arranged in order of the degree of contribution. The analysis system according to [6] above. [8] The plurality of customers have transaction histories of at least one of products or services across a plurality of categories; The user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated genre among the plurality of genres. The analytical system according to any one of the above [1] to [7].
[0166] [9] At least some of the plurality of customers have transaction histories with a plurality of brands that offer products or services across a plurality of genres; The user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated category provided by one of a plurality of brands; The analysis system according to any one of the above [1] to [8].
[0167]
[10] The attribute data includes a transaction amount of each of the customers' products or services; The one or more graphs include a graph displaying a comparison of data on the transaction amounts of the customers constituting the first group and the second group, The analysis system according to any one of the above [1] to [9].
[0168]
[11] The comparison data includes results of one or more learning models; Each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; The calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction. The analysis system according to any one of the above [1] to
[10] .
[0169]
[12] the one or more learning models include a plurality of learning models; The learning data used to generate the plurality of learning models includes attribute data relating to combinations of attributes different from each other. The analysis system according to
[11] above.
[0170]
[13] The contribution of each attribute in the prediction is calculated as a SHAP (SHapley Additive exPlanations) value. The analysis system according to
[11] or
[12] above.
[0171]
[14] One or more computers, a process of acquiring comparison data between a first group and a second group, the first group and the second group being extracted from a plurality of customers included in customer data based on a user's instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; outputting an explanatory text describing a comparison result between the first group and the second group based on an output of the large-scale language model; 4. A method of analysis comprising:
[0172]
[15] On one or more computers, A process of extracting a first group and a second group from a plurality of customers included in customer data based on an instruction from a user, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution degree of each of the important attributes in the comparison; generating one or more graphs for the contrast data; generating a legend for the one or more graphs; outputting said legend together with said one or more graphs; 4. A method of analysis comprising:
[0173]
[16] One or more computers, a process of acquiring comparison data between a first group and a second group, the first group and the second group being extracted from a plurality of customers included in customer data based on a user's instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; outputting an explanatory text describing a comparison result between the first group and the second group based on an output of the large-scale language model; A program to execute.
[0174]
[17] One or more computers, A process of extracting a first group and a second group from a plurality of customers included in customer data based on an instruction from a user, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution degree of each of the important attributes in the comparison; generating one or more graphs for the contrast data; generating a legend for the one or more graphs; outputting said legend together with said one or more graphs; A program to execute. [Explanation of symbols]
[0175] 11...analysis system, 12...network, 13...database, 14...large-scale language model, 15...user terminal, 16...display, 17-19,72a-72d,73b,75a,76a,78,78a-78e,86,88...graph, 20...analysis device, 21,31...processor, 22,32...memory, 23,33...communication interface, 24,34...program, 25...contrast data, 30...machine learning device, 35...learning model, 40...input screen, 41,81...sidebar, 42,82...main column, 43...account input field, 44...action selection field, 45...setting button, 46-48...option setting field, 50...setting area, 51,61...name input field, 5 2, 56, 62, 66, 70b...selection field, 53, 63, 54, 64, 83, 91...input field, 55, 65, 73a...selection button, 59...confirmation button, 60...setting area, 69...decision button, 70...result screen, 70a...title, 70c...display field, 71...navigation column, 72...display area, 73...graph display area, 74...insight display area, 75~77...buttons, 75b...confusion matrix, 80...display screen, 82a, 82b...tabs, 83...user name input field, 84...dataset selection field, 85...question input field, 87...insight, 87...answer insight, 89...summary insight, 90...chat screen, 92...send button, 93...bot speech bubble, 94...user speech bubble.
Claims
1. at least one memory configured to store a program; and at least one processor configured to execute one or more processes based on the content of the program, The one or more processes include: A process of acquiring comparison data between a first group and a second group, the first group and the second group are each extracted from a plurality of customers included in the customer data based on an instruction from a user; the customer data includes attribute data relating to a plurality of attributes of each of the customers; the comparison data includes a plurality of important attributes effective for comparing the first group with the second group, and a contribution of each of the important attributes to the comparison; The comparison data includes results of one or more learning models; Each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction; The contribution degree quantitatively represents the degree of influence of each attribute used as a variable in the prediction on the prediction result, The plurality of important attributes are the top N attributes (N is a natural number equal to or greater than 2) having the highest contribution degree among the plurality of attributes; and inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; outputting an explanatory text describing a comparison result between the first group and the second group based on an output of the large-scale language model; 16. An analysis system comprising:
2. The one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text. The analytical system of claim 1 .
3. The one or more processes include: obtaining a question input by the user regarding a comparison of the first group with the second group; selecting one or more relevant attributes from the plurality of attributes that are relevant to the question; selecting a predetermined number of related attributes having a high contribution rate from the one or more related attributes; Including, In the process of acquiring the comparison data, the predetermined number of related attributes are the plurality of important attributes; In the process of inputting to the large-scale language model, the one or more instruction sentences include an instruction sentence for answering the question, In the process of outputting the explanatory text, the explanatory text includes an answer to the question. The analytical system of claim 1 .
4. The one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text. The analysis system according to claim 3 .
5. at least one memory configured to store a program; and at least one processor configured to execute one or more processes based on the content of the program, The one or more processes include: A process of extracting a first group and a second group from a plurality of customers included in customer data based on an instruction from a user, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data includes a plurality of important attributes effective for comparing the first group with the second group, and a contribution of each of the important attributes to the comparison; The comparison data includes results of one or more learning models; Each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction; The contribution degree quantitatively represents the degree of influence of each attribute used as a variable in the prediction on the prediction result, The plurality of important attributes are the top N attributes (N is a natural number equal to or greater than 2) having the highest contribution degree among the plurality of attributes; and generating one or more graphs for the contrast data; generating a legend for the one or more graphs; outputting said legend together with said one or more graphs; 16. An analysis system comprising:
6. the one or more graphs include a graph showing the plurality of important attributes and the contribution of each of the plurality of important attributes; 6. The analysis system according to claim 2, 4 or 5.
7. The plurality of important attributes are arranged in order of the degree of contribution. The analytical system according to claim 6.
8. The plurality of customers have transaction histories of at least one of products or services across a plurality of categories, The user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated genre among the plurality of genres. The analysis system according to any one of claims 1 to 5.
9. At least a portion of the plurality of customers has transaction histories with a plurality of brands that offer products or services across a plurality of genres; The user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated genre provided by one of a plurality of brands; The analytical system according to claim 8.
10. The attribute data includes a transaction amount of a product or service of each of the customers, the one or more graphs include a graph displaying a comparison of data on the transaction amounts of the customers constituting the first group and the second group, 6. The analysis system according to claim 2, 4 or 5.
11. Each of the learning models is a random forest classifier trained using the customer data of each of the first group and the second group as learning data. The analysis system according to any one of claims 1 to 5.
12. the one or more learning models include a plurality of learning models; The learning data used to generate the plurality of learning models includes attribute data relating to combinations of attributes different from each other. The analysis system according to any one of claims 1 to 5.
13. The contribution of each attribute in the prediction is calculated as a SHAP (SHapley Additive exPlanations) value. The analytical system of claim 11.
14. One or more computers A process of acquiring comparison data between a first group and a second group, the first group and the second group are each extracted from a plurality of customers included in the customer data based on an instruction from a user; the customer data includes attribute data relating to a plurality of attributes of each of the customers; the comparison data includes a plurality of important attributes effective for comparing the first group with the second group, and a contribution of each of the important attributes to the comparison; The comparison data includes results of one or more learning models; Each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction; The contribution degree quantitatively represents the degree of influence of each attribute used as a variable in the prediction on the prediction result, The plurality of important attributes are the top N attributes (N is a natural number equal to or greater than 2) having the highest contribution degree among the plurality of attributes; and inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; outputting an explanatory text describing a comparison result between the first group and the second group based on an output of the large-scale language model; The analytical method includes performing
15. One or more computers A process of extracting a first group and a second group from a plurality of customers included in customer data based on an instruction from a user, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data includes a plurality of important attributes effective for comparing the first group with the second group, and a contribution of each of the important attributes to the comparison; The comparison data includes results of one or more learning models; Each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction; The contribution degree quantitatively represents the degree of influence of each attribute used as a variable in the prediction on the prediction result, The plurality of important attributes are the top N attributes (N is a natural number equal to or greater than 2) having the highest contribution degree among the plurality of attributes; and generating one or more graphs for the contrast data; generating a legend for the one or more graphs; outputting said legend together with said one or more graphs; The analytical method includes performing
16. On one or more computers, A process of acquiring comparison data between a first group and a second group, the first group and the second group are each extracted from a plurality of customers included in the customer data based on an instruction from a user; the customer data includes attribute data relating to a plurality of attributes of each of the customers; the comparison data includes a plurality of important attributes effective for comparing the first group with the second group, and a contribution of each of the important attributes to the comparison; The comparison data includes results of one or more learning models; Each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction; The contribution degree quantitatively represents the degree of influence of each attribute used as a variable in the prediction on the prediction result, The plurality of important attributes are the top N attributes (N is a natural number equal to or greater than 2) having the highest contribution degree among the plurality of attributes; and inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; outputting an explanatory text describing a comparison result between the first group and the second group based on an output of the large-scale language model; A program to execute.
17. On one or more computers, A process of extracting a first group and a second group from a plurality of customers included in customer data based on an instruction from a user, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data includes a plurality of important attributes effective for comparing the first group with the second group, and a contribution of each of the important attributes to the comparison; The comparison data includes results of one or more learning models; Each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction; The contribution degree quantitatively represents the degree of influence of each attribute used as a variable in the prediction on the prediction result, The plurality of important attributes are the top N attributes (N is a natural number equal to or greater than 2) having the highest contribution degree among the plurality of attributes; and generating one or more graphs for the contrast data; generating a legend for the one or more graphs; outputting said legend together with said one or more graphs; A program to execute.
Citation Information
Patent Citations
Method for analyzing commodity power, system for analyzing commodity power, and device for analyzing commodity power
JP2006072607A
Clustering program, clustering method, and clustering device
JP2014106910A
Information processing method, information processing system, information processor and program thereof
JP2016167172A
Data analysis program
JP2016184329A
Cited By
Information processing apparatus, information processing method, information processing system, and program
JP2026105719A
system
JP2026155653A
Information Processing Apparatus, Information Processing Method, Information Processing System, and Program
JP7713083B1
Device and method
WO2026099924A1
Information processing device and information processing method
WO2026167740A1
