Analyzing system, analyzing method, and program

Abstract

To provide an analyzing system, an analyzing method, and a program.SOLUTION: The analyzing method causes one or more computers to execute: processing for acquiring comparison data between a first group and a second group, wherein the first group and the second group are extracted respectively from a plurality of customers included in customer data based on user instructions, the customer data includes attribute date relating to a plurality of attributes held by each customer, and the comparison data includes a plurality of important attributes effective for comparison between the first group and the second group and contribution to the comparison of respective important attributes; processing for inputting one or more instruction sentences for use in comparison between the first group and the second group to a large-scale language model together with the comparison data; and processing for outputting an explanatory text describing a result of the comparison between the first group and the second group based on the output of the large-scale language model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an analysis system, an analysis method, and a program. [Background technology]

[0002] When analyzing consumer purchasing behavior for marketing purposes, clustering of consumers may be performed based on information including their purchasing history. For example, Patent Document 1 discloses a clustering device that can group multiple consumers into multiple purchasing groups that show similar changes in purchasing behavior. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-106910 Summary of the Invention [Problem to be solved by the invention]

[0004] Marketing personnel analyze the purchasing behavior of each of multiple purchasing groups. Such analysis requires knowledge and experience, so it can be difficult for inexperienced personnel to perform an appropriate analysis.

[0005] The present disclosure aims to provide an analysis system, an analysis method, and a program that can assist in analyzing customer groups. [Means for solving the problem]

[0006] An analysis system according to one embodiment of the present disclosure includes at least one memory configured to store a program and at least one processor configured to execute one or more processes based on the content of the program, the one or more processes including: a process of acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; a process of inputting one or more instruction sentences for comparing the first group with the second group into a large-scale language model together with the comparison data; and a process of outputting an explanatory sentence describing the comparison result of the first group with the second group based on the output of the large-scale language model.

[0007] An analysis system according to another aspect of the present disclosure comprises at least one memory configured to store a program and at least one processor configured to execute one or more processes based on the contents of the program, the one or more processes including: a process of extracting a first group and a second group from a plurality of customers included in customer data based on a user's instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers; a process of generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; a process of generating one or more graphs for the comparison data; a process of generating explanatory text for the one or more graphs; and a process of outputting the explanatory text together with the one or more graphs.

[0008] An analysis method according to one embodiment of the present disclosure includes: a process of acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data including attribute data relating to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; a process of inputting one or more instruction sentences for comparing the first group with the second group into a large-scale language model together with the comparison data; and a process of outputting an explanatory sentence describing the comparison result of the first group with the second group based on the output of the large-scale language model.

[0009] An analysis method according to another aspect of the present disclosure includes causing one or more computers to execute the following processes: extracting a first group and a second group from a plurality of customers included in customer data based on user instructions, wherein the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers; generating comparison data between the first group and the second group, wherein the comparison data includes a plurality of important attributes that are effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; generating one or more graphs for the comparison data; generating explanatory text for the one or more graphs; and outputting the explanatory text together with the one or more graphs.

[0010] A program according to one embodiment of the present disclosure causes one or more computers to execute the following processes: a process of acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data including attribute data related to a plurality of attributes possessed by each of the customers, and the comparison data including a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes in the comparison; a process of inputting one or more instruction sentences for comparing the first group with the second group into a large-scale language model together with the comparison data; and a process of outputting an explanatory sentence describing the comparison result of the first group with the second group based on the output of the large-scale language model.

[0011] A program according to another aspect of the present disclosure causes one or more computers to execute the following processes: extracting a first group and a second group from multiple customers included in customer data based on user instructions, wherein the customer data includes attribute data related to multiple attributes possessed by each of the customers; generating comparison data between the first group and the second group, wherein the comparison data includes multiple important attributes that are effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; generating one or more graphs for the comparison data; generating explanatory text for the one or more graphs; and outputting the explanatory text together with the one or more graphs. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of an analysis system according to an embodiment. [Figure 2] Figure 2 is a graph showing the importance of attributes as SHAP values. [Figure 3] FIG. 3 is a graph showing the cumulative composition ratios of multiple attributes. [Figure 4] Figure 4 is a graph showing the SHAP value for one attribute. [Figure 5] FIG. 5 is a diagram illustrating an example of an input screen of the data generation application. [Figure 6] FIG. 6 is a diagram illustrating an example of the "Brand & Genre" selection field on the input screen of FIG. [Figure 7] FIG. 7 is a diagram showing an example of the "genre" selection field on the input screen of FIG. [Figure 8] FIG. 8 is a flowchart showing a method for generating data by a data generation application. [Figure 9] FIG. 9 is a diagram illustrating a result screen relating to "general statistics" of the data generation application. [Figure 10] FIG. 10 is a diagram illustrating a result screen relating to "life" of the data generation application. [Figure 11] FIG. 11 is a diagram illustrating detailed information about model accuracy of a data generation application. [Figure 12] FIG. 12 is a diagram illustrating detailed information about important feature quantities of a data generation application. [Figure 13] FIG. 15 is a graph showing the influence of each of several important attributes. [Figure 14] FIG. 14 is a diagram illustrating an example of a display screen of the explanation application. [Figure 15] FIG. 15 shows the summary results obtained by the explanation application. [Figure 16] FIG. 16 is a flowchart showing a method for generating answer insights. [Figure 17] FIG. 17 is a diagram illustrating an example of a chat screen of the analytical chatbot. [Figure 18] FIG. 18 is a flowchart showing a method for generating and acquiring comparison data via an analytical chatbot. DETAILED DESCRIPTION OF THE INVENTION

[0013] Examples of the analysis system 11, analysis method, and program of the present disclosure will be described with reference to Figures 1 to 18. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0014] [Overall configuration of the analysis system] 1, the analysis system 11 includes one or more analysis devices 20. Each analysis device 20 may be realized as a computer including, for example, one or more processors 21, one or more memories 22, and a communication interface (IF) 23. When the one or more analysis devices 20 include a plurality of analysis devices 20, some or all of their configurations may be different from each other.

[0015] A program 24 is stored in one or more memories 22. The program 24 includes program code for an application and an operating system. The one or more processors 21 execute one or more processes based on the program 24 to realize various functions. The memory 22 may store comparison data 25 generated by the execution of the program 24.

[0016] The communication interface 23 enables communication with other devices via the network 12. The network 12 includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, and the like.

[0017] The analysis system 11 may include one or more machine learning devices 30. Alternatively, the analysis device 20 may have the functionality of the machine learning device 30. The one or more analysis devices 20 may be able to communicate with one or more machine learning devices 30 via the network 12. Each machine learning device 30 may be realized as a computer including one or more processors 31, one or more memories 32, and a communication interface (IF) 33. The communication interface 33 enables communication with other devices via the network 12.

[0018] A program 34 is stored in one or more memories 32. The program 34 includes an application and an operating system. The one or more processors 31 execute processes based on the program 34 to realize various functions. The memory 32 may store one or more learning models 35 generated by executing the program 34. When generating comparison data, the analysis device 20 operates the machine learning device 30 to generate the learning model 35 and perform predictive calculations.

[0019] The processors 21 and 31 are, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), a field-programmable gate array (FPGA), or other arithmetic devices. The processors 21 and 31 are, for example, processing circuits configured to execute various software processes. The processing circuits may include dedicated hardware circuits (e.g., ASICs) that process at least part of the software processes. In other words, the software processes may be executed by processing circuitry that includes at least one or more software processing circuits and one or more dedicated hardware circuits.

[0020] The memories 22 and 32 may be non-transitory computer-readable media. The memories 22 and 32 may include, for example, random access memory (RAM) or other volatile memory. The memories 22 and 32 may be configured to temporarily store programs and data. The memories 22 and 32 may also include storage for permanently storing data including programs. The storage may be, for example, read-only memory (ROM), a hard disk drive, flash memory, or other non-volatile storage device. The storage may also be a removable storage device such as a memory card. The communication interfaces 23 and 33 may be realized, for example, as a LAN or other wired communication IF.

[0021] The analysis system 11 may include a database 13. The database 13 may be stored in a server device (not shown). Alternatively, the analysis device 20 may use data in a database 13 that is not included in the analysis system 11 via the network 12. Alternatively, the analysis device 20 or the machine learning device 30 may include part or all of the database 13.

[0022] The analysis system 11 may include a large-scale language model 14. Alternatively, the analysis device 20 may use a large-scale language model 14 that is not included in the analysis system 11 via the network 12. The large-scale language model 14 is a natural language processing model trained using a large amount of text data. The large-scale language model 14 is a general-purpose language model that can be adapted to various natural language processing tasks, such as information extraction, text summarization, text generation, or question and answering.

[0023] The large-scale language model 14 is configured to, when a prompt containing some instruction is input, create a sentence according to the instruction and output it as a completion. In this example, the analysis device 20 generates a prompt and inputs it to the large-scale language model 14, and then the analysis device 20 acquires the completion generated by the large-scale language model 14.

[0024] The user terminal 15 may be capable of communicating with the analytical device 20 via the network 12. The user terminal 15 may be, for example, a computer or a mobile terminal such as a tablet. The user terminal 15 may include a display 16. A user can use the functions of one or more applications provided by the analytical device 20 through the user terminal 15. Screens of the one or more applications are displayed on the display 16.

[0025] The one or more applications may be installed on a user terminal 15, or some or all of their functions may be provided as a web application from an analysis device 20. The one or more applications may include at least one of a data generation application (customer profiling), an insight summarization application (explanatory AI agent), or an analytical chatbot (AI analyst). The analysis system 11 may include three analysis devices 20 for providing the data generation application, the insight summarization application, and the analytical chatbot, respectively.

[0026] [Customer insight analysis] Customer insights (also known as consumer insights) generally refer to the motivations, true feelings, or psychology of customers that underlie their purchasing behavior. Analyzing customer insights makes it possible to review or improve strategies for providing existing products or services, or to carry out various marketing activities such as developing new products or services and cultivating new markets. Customer insights are generally obtained by analyzing various information, including customer attribute information and customer purchasing history.

[0027] One example of analysis is comparing the insights of your brand's customer groups with those of a competitor's brand. Comparing the insights of two customer groups can provide clues for differentiating your company from competitors. If you can provide products or services that customers potentially want based on this analysis, you can differentiate yourself from competitors or acquire new customers.

[0028] The analysis device 20, particularly the data generation application, is configured to provide contrast data 25 used in analyzing customer insights. In this disclosure, a consumer of a product or service is referred to as a "customer," a person in charge of marketing the product or service is referred to as an "analyst," and a person using the application to provide the contrast data 25 to the analyst is referred to as a "user." A "user" may also be an "analyst." A customer may include a potential customer, i.e., a consumer who has not purchased the product or service in question but may purchase it. An analyst may be a business providing the product or service, or a marketer contracted by the business to conduct marketing.

[0029] The data generation application is configured to generate comparison data 25 based on customer data. The comparison data 25 includes, for example, data comparing two customer groups (in this example, a first group and a second group) created based on a certain analysis theme. The two customer groups include a first group that is the subject of analysis and a second group that is the subject of comparison with the subject of analysis. The multiple customers included in the first group and the multiple customers included in the second group are each extracted from multiple (large number of) customers included in the customer data.

[0030] For example, if the first group is a group of purchasers of a certain product, the second group may be a group of potential customers who have not purchased that product. Alternatively, if the first group is a group of viewers who have viewed an advertisement for a certain product, the second group may be a group of viewers who have viewed an advertisement for a product other than the first product.

[0031] The user can set any analysis theme depending on the purpose of the analysis. The analysis theme may include the names of two objects to be compared, such as the names of two groups, two product names, or two service names. For example, when comparing a customer group that prefers "lager beer" with a customer group that prefers "ale beer," the analysis theme can be set as "lager beer vs. ale beer" or "How is lager beer different from ale beer?"

[0032] The customer data may be stored in database 13. The multiple (large number of) customers may include customers of multiple types of products or services, customers of multiple brands, and customers of multiple businesses. For example, the multiple customers may include a customer group that uses each of one or multiple e-commerce sites, and multiple businesses may offer products or services on one e-commerce site. Alternatively, or in addition, the multiple customers may include a customer group that participates in a common points program.

[0033] The customer data includes attribute data for each of a plurality of customers. In this disclosure, the term "attribute" may refer to information (e.g., "age") that includes a combination of an "attribute" as a data item (e.g., "age") and an "attribute value" that is a value for the attribute (e.g., "40 years old"). The attribute data may include static data items and dynamic data items related to customer attributes.

[0034] Static data items include, but are not limited to, date of birth, hometown, date of first purchase, and date of service use. Dynamic data items include, but are not limited to, name, age, address, family composition, annual income, occupation, income, employment status, living situation, hobbies, behavioral history, and interests. Behavioral history includes, but is not limited to, service usage history, product or service transaction history (purchase history), and website browsing or search history. Interests include, but are not limited to, information about products or services in which each customer is interested, and fields or genres of interest.

[0035] [Calculation of contribution using learning model] The one or more learning models 35 may be a predictive model that predicts whether each of a plurality of customers belongs to a first group or a second group based on customer data. This predictive model may be, for example, a random forest classifier. The one or more learning models 35 may include multiple learning models 35 generated using different learning data.

[0036] The learning model 35 is generated, for example, as follows. First, customer data from each of the first and second groups is extracted as learning data, and the learning data is then divided into training data and test data. A predictive model is then generated using the training data. The accuracy of the predictive model may then be confirmed by evaluating the predictive model created using the test data. The accuracy of the predictive model can be evaluated using, for example, a receiver operating characteristic curve (ROC curve) or a precision-recall curve (PR curve). A predictive model that has achieved a certain level of accuracy is stored in memory 32 as a practical learning model 35.

[0037] Each of the multiple attributes contained in the customer data can be a feature that changes the prediction result. When dealing with a learning model, the term "feature" is sometimes used as a term that includes an "attribute" and its "value." However, in this example, as mentioned above, "attribute" is used as a term that includes not only a simple data item but also an attribute and its value. Therefore, "attribute" is used as a term with the same meaning as "feature."

[0038] The random forest can display the contribution (importance) of each of the features used in the training. The data generation application may be configured to use the training model 35 to output the contribution of each attribute as part of the comparison data 25.

[0039] The contribution of each attribute may be calculated as the impact or importance of the feature, as a SHAP (SHapley Additive exPlanations) value, which is an application of the Shapley Value to machine learning. The SHAP value is used to quantitatively represent the degree of contribution of each attribute used as a variable in prediction. Outputting the quantitative contribution of such attributes (features) can be considered an additional function of the learning model 35.

[0040] In analyzing customer insights, the SHAP value, i.e., the contribution of each attribute to the prediction, is required, rather than the prediction result of "whether the customer belongs to the first or second group," among the calculation results of the learning model 35. Therefore, it is advisable to extract customer data from a desired analysis period (for example, the sales period of the product to be analyzed) as learning data.

[0041] In this case, prediction may be performed while adding attribute data for one attribute so that the feature quantity included in the training data increases by one, and the degree of influence of the added attribute data on the prediction result may be calculated sequentially. For example, if adding attribute data for a certain attribute results in a clearer classification result, it can be said that the attribute has a strong influence on the prediction result. In this case, the SHAP value of the attribute will be large.

[0042] In this way, the SHAP value indicates the degree of influence on the model output. More specifically, a SHAP value greater than 0 indicates a higher contribution, and a SHAP value less than 0 indicates a lower contribution. Therefore, an attribute with a larger SHAP value is a unique feature of the corresponding customer group, i.e., an important attribute. The data generation application or learning model 35 may be configured to output graphs 17 to 19 such as those shown in FIGS. 2 to 4 as part of the calculation results.

[0043] Figure 2 shows an example of a graph 17 in which the contributions of multiple attributes are output as SHAP values. Graph 17 displays the top 20 influential attributes (top features) among multiple attributes, with SHAP values ​​on the horizontal axis. The top 20 attributes are arranged from top to bottom in descending order of contribution. The top N (N is a natural number) attributes (top features) extracted in this way are called "multiple important attributes" that are useful for comparing the first group with the second group. The inverted triangular scale 17a on the left side of graph 17 tapers downward from the wide top edge, indicating that the influence of the attributes decreases as you go down.

[0044] Graph 17 is also known as a "SHAP Bea Swarm Plot," where each plot (dot) represents one customer's SHAP value for each attribute. The color of each plot may indicate the relative scale of the values, for example, values ​​closer to red indicate larger values ​​and values ​​closer to blue indicate smaller values. In Graphs 17, 86, and 88, darker colors represent blue and lighter colors represent red. The vertical axis 17b of the "Feature value" on the right side of Graph 17, which ranges from "High" to "Low," is displayed as a gradient that changes continuously from red (gray in the figure) at the highest end to blue (black in the figure) at the lowest end to indicate the color scale of the plot.

[0045] The horizontal position of each plot indicates the SHAP value for each customer feature (attribute). Here, a larger SHAP value (located to the right) indicates a greater positive contribution, and a smaller SHAP value (located to the left) indicates a greater negative contribution.

[0046] FIG. 3 shows an example of a graph 18 showing the cumulative composition ratio of SHAP values ​​for multiple feature quantities. Graph 18 includes a bar graph showing the composition ratio of each of the multiple feature quantities and a line graph showing the cumulative ratio. When displaying several feature quantities with high influence among multiple feature quantities, it is also possible to display feature quantities up to 50% in cumulative ratio. For example, in graph 18 of FIG. 3, the top four feature quantities are the top 50% of feature quantities, i.e., important feature quantities (important attributes).

[0047] FIG. 4 shows an example of a graph 19 indicating the SHAP value of one of multiple feature quantities. The feature quantity shown in FIG. 4 is a member rank in a points program or an e-commerce site. The member rank is indicated, for example, by a number from 0 to 7, with the higher the number, the higher the member rank. If the member rank is set according to the degree of use of the points program or the e-commerce site, the higher the degree of use (for example, the larger the purchase amount), the higher the member rank. Graph 19 shows that the attribute "high member rank (member rank 6, 7)" contributes in a positive direction, and the attribute "low or medium member rank (member rank 1 to 5)" contributes in a negative direction.

[0048] [Data generation application] The data generation application is configured to generate comparison data 25 in response to user input. The comparison data 25 mainly includes numerical data that is the result of calculations using the learning model 35. A series of processes executed in response to one user input or data generated for that series of processes is called a "job." Furthermore, the comparison data 25 output by one job is called "one dataset." A name (dataset name) may be automatically assigned to each dataset. The dataset name may be the same as the analysis theme.

[0049] The data generation application is configured to generate a new job when a new data set is output. The generated job and the data set generated by the job are stored in memory 22. A user can read one or more data sets generated in the past as existing jobs. When multiple users share one data generation application, dataset viewing authority may be set for each user so that a user can read jobs generated in the past by logging in to the data generation application.

[0050] 5 shows an input screen 40 of an exemplary data generation application. The input screen 40 is an example, and the design can be changed as desired. The input screen 40 may include a sidebar 41 and a main column 42.

[0051] The sidebar 41 may include, for example, at least one of a login account input field 43, an action selection field 44, and a settings button 45. The sidebar 41 may also include other input fields, selection fields, or buttons. In the data generation application, the action selection field 44 may be configured to display options of "Create a new job" and "View existing jobs."

[0052] The data generation application may operate the machine learning device 30 as a default setting to generate, for each job, multiple learning models 35. The multiple learning models 35 may be generated using attribute data including combinations of specified attributes related to, for example, "General Statistics," "Life," "Shopping," and "Attitude" as learning data.

[0053] Attributes related to "general statistics" include, for example, gender, age, and transaction amount per specified period on a specified EC site (for example, GMS: Gross Merchandise Sales, total distribution transaction amount), but are not limited to these.

[0054] Examples of attributes related to "lifestyle" include, but are not limited to, educational background, employment status, residential status, whether or not a person has children, whether or not they are married, whether or not they own a vehicle or license, and whether or not they participate in various sports. "Shopping" may include attributes related to the customer's transaction history (purchase history), such as attributes related to the genre, brand, and purchase amount of products purchased in the past.

[0055] Examples of the "attitude" attribute may include various customer attitudes in various areas including lifestyle and shopping. Examples of attitudes include, but are not limited to, "I prefer cashless shopping to cashless shopping," "I think online shopping saves time," "I like to earn points," "I think I have a sociable personality," "I don't like to use brands that other people have," and "I pay attention to my daily diet."

[0056] When the setting button 45 is selected, the data generation application may display one or more option setting fields 46 to 48. The setting field 46 may be a field for adding features. The setting field 46 may be a multiple-selection field, and it may be possible to add a combination of specified attributes such as "General Demography," "Interest," "Shopping Behavior," and "Service Usage."

[0057] "Interests" is a combination of attributes related to the interests of multiple customers. Attributes related to interests may be, for example, the level of interest in each category of products and services available on the e-commerce site, such as "interest in beauty and cosmetic products" or "interest in PCs and peripherals." Attribute data related to purchasing trends and service usage may be obtained from the transaction history of each category or brand of products and services available on the e-commerce site.

[0058] The setting field 46 may include an "Add New" option that allows any combination of attributes to be freely added. When a combination of attributes is added using the setting field 46, the data generation application causes the machine learning device 30 to operate so as to generate a learning model 35 using training data that includes the corresponding attribute data.

[0059] The setting field 47 may be a selection field for the number of important attributes included in the output data, and may be, for example, a choice between "up to the top 20" and "within a cumulative composition ratio of 50%." Additionally or alternatively, the setting field 47 may allow a user to select an option for adding detailed information about important attributes. Furthermore, the setting field 47 may allow the user to set the number of important attributes for which detailed information is to be displayed. The setting field 48 may allow the user to set the display language, and may be, for example, a choice between "Japanese" and "English."

[0060] The main column 42 may include a setting area 50 for an analysis group, which is a first group, and a setting area 60 for a comparison group, which is a second group. The setting area 50 may include an analysis group name input field 51, a data selection field 52, a start date input field 53 for inputting the start period of the data, and an end date input field 54 for inputting the end period of the data. The setting area 50 may further include one or more selection buttons 55.

[0061] Similar to the setting area 50, the setting area 60 may include a comparison group name input field 61, a data selection field 62, a start date input field 63, an end date input field 64, and one or more selection buttons 65. The user can select data for a desired number of customers from the entire customer data using the selection fields 52 and 62.

[0062] The selection field 62, input fields 63 and 64, and selection button 65 of the setting area 60 may be displayed or may become inputtable when input into the selection field 52 and input fields 53 and 54 is completed in the setting area 50. When input into the input fields 53 and 54 is completed, the same date as that in the input fields 53 and 54 may be automatically input as a default value into the input fields 63 and 64.

[0063] The one or more selection buttons 55 may be input fields for arbitrarily selectable options. For example, the one or more selection buttons 55 may be radio buttons for alternatively selecting one of "Brand & Genre," "Brand," or "Genre." The one or more selection buttons 65 may be radio buttons for alternatively selecting one of "Brand & Genre," "Brand," and "Genre," including an option "Random." When the "Random" option is selected, data for a specified number of people is randomly extracted. The selection field 66 indicated by a dashed line in FIG. 5 indicates that the selection field 66 is in an inoperable state due to the selection of "Random" with the selection button 65.

[0064] 5 shows all of the multiple input fields, selection fields, and buttons for ease of explanation, but the input fields, selection fields, or buttons may be displayed sequentially depending on the processing order of the data generation application. For example, the action selection field 44 may be displayed after logging in by entering an account in the input field 43. Furthermore, when "Create a new job" is selected in the action selection field 44, the name input fields 51 and 61 may be displayed. Furthermore, when the name input fields 51 and 61 are filled in, the selection field 52 and the input fields 53 and 54 may be displayed.

[0065] 6 and 7 are examples of displays when the additional selection fields 56, 66 are operated. When "Brand & Genre" is selected with the selection buttons 55, 65, options such as those shown in FIG. 6 may be displayed. When "Genre" is selected with the selection buttons 55, 65, options such as those shown in FIG. 7 may be displayed. When "Brand" is selected with the selection buttons 55, 65, a list of brand names may be displayed.

[0066] Genre options may include, but are not limited to, "TV, audio, cameras," "sports, outdoors," "smartphones, tablets," "diet, health," "computers, peripherals," "medicine, contact lenses, nursing care," and "home appliances." Genres may be the same as the categories used when handling products or services on an e-commerce site.

[0067] When one of multiple genres or brands is selected in selection field 56, customers included in the first group are extracted from multiple customers who have a transaction history of products or services of the selected designated genre or designated brand. Similarly, when one genre or brand is selected in selection field 66, customers included in the second group are extracted from multiple customers who have a transaction history of products or services of the selected designated genre or designated brand.

[0068] When "Brand & Genre" is selected with selection buttons 55, 65, a combination of brand and genre becomes selectable in selection fields 56, 66. For example, a brand name ("Brand Y" in FIG. 6) may first be selectable in selection fields 56, 66, and when one brand is selected, multiple genres belonging to the specified brand (multiple genres related to furniture in FIG. 6) may become selectable. Brand and genre may be selectable with separate selection buttons.

[0069] It may be possible to select multiple brand and genre combinations. For example, in FIG. 6, "Brand X & Bookshelf" has been selected as the first brand and genre, and "Brand Y" has been selected as the second brand and genre, and multiple genres belonging to Brand Y are displayed as selectable. By selecting "Brand & Genre," customers included in the first group or the second group are extracted from multiple customers who have a transaction history in the specified genre of the specified brand.

[0070] Genres and brands may be selectable from classifications across multiple levels (e.g., three levels). For example, in the case of brands, any level may be selectable from among the company name (corporate brand) at the first level, the product category name (family brand) at the second level, and the product name (product brand) at the third level. The genre hierarchy may be the same as that of an e-commerce site, for example.

[0071] 5, the main column 42 may include a confirmation button 59 and a decision button 69. When the confirmation button 59 is operated, a confirmation screen (not shown) may be displayed that displays the content entered on the input screen 40. The confirmation screen (not shown) may include a decision button 69. When the decision button 69 is operated, a process for generating data including the comparison data 25 is executed based on instructions entered by the user.

[0072] With reference to FIG. 8, a method for generating data via a data generation application will now be described. In step S1, processor 21 executes a process of extracting a plurality of customers belonging to a first group and a plurality of customers belonging to a second group from the plurality of customers included in the customer data. At this time, first customer data for the plurality of customers belonging to the first group and second customer data for the plurality of customers belonging to the second group are extracted from the entire customer data.

[0073] The customer data includes attribute data relating to a plurality of attributes possessed by each customer. In step S1, the processor 21 extracts, as training data, customer data corresponding to each of the plurality of learning models 35 to be generated. For example, the processor 21 extracts, from the customer data, training data relating to a combination of default attributes as well as training data relating to a combination of attributes additionally input by the user.

[0074] In step S2, processor 21 causes machine learning device 30 (processor 31) to generate multiple learning models 35 using the learning data. In step S3, processor 21 causes machine learning device 30 (processor 31) to execute calculation (prediction) processing using learning models 35. This calculation includes calculation for evaluating the accuracy of learning models 35 and calculation for calculating SHAP values ​​for each of multiple attributes included in the learning data. Steps S2 to S7 are executed for each of the multiple learning models 35.

[0075] In step S4, processor 21 acquires the calculation results of learning model 35. In the following step S5, processor 21 generates comparison data based on the calculation results. More specifically, from the multiple attributes used in the calculation, the top N attributes are extracted as important attributes based on the SHAP value of each attribute. The number N is specified by default or specified by user input. Therefore, the comparison data includes multiple important attributes that are effective for comparing the first group with the second group, and SHAP values ​​that indicate the contribution of each important attribute to the comparison.

[0076] In step S6, processor 21 generates explanatory text for the defined items and, if necessary, for items specified by the user, based on the comparison data. In addition, in step S7, processor 21 generates one or more graphs to pair with the explanatory text, based on the comparison data used when generating the explanatory text. That is, the explanatory text is generated so as to describe an explanation for the corresponding graph. Steps S6 and S7 may be reversed in order or may be performed simultaneously. In step S8, processor 21 outputs generated data including the generated explanatory text and graph.

[0077] FIG. 9 illustrates a results screen 70 of a data generation application for displaying generated data. The results screen 70 may display a title 70a of the corresponding job. The title 70a may be an analysis theme including the name of a first group and the name of a second group. The results screen 70 may also include a selection field 70b for selecting a job to display. Each job in the list displayed in the selection field 70b may display the name of the job and the date and time the job was created.

[0078] The results screen 70 may include a navigation column 71 at the top. The navigation column 71 may include multiple tabs, such as "General Statistics," "Lifestyle," "Shopping," "Attitude," "More," and "Take Action." The "Lifestyle," "Shopping," "Attitude," and "More" tabs may display the calculation results of multiple corresponding learning models 35.

[0079] "Lifestyle" may include comparison data related to a customer's life stages. "Shopping" may include comparison data related to, for example, a customer's search history, page browsing history, transaction history, and campaign participation history. "Attitude" is comparison data related to a customer's attitude. "Other" may include comparison data including all attributes, or comparison data related to attributes selected by the user. Since comparison data corresponding to multiple tabs is displayed using a similar interface, Figure 10 shows an example of comparison data related to "Lifestyle," and omits the display of comparison data corresponding to other tabs.

[0080] Operating each tab may jump to an area (shown in FIGS. 9 and 10) that displays the corresponding results. Alternatively, scrolling down the results screen 70 may sequentially display the results corresponding to each of the multiple tabs.

[0081] Operating the "General Statistics" tab may display a display area 72 comparing the first group and the second group. The display area 72 may include, but is not limited to, graph 72a comparing the number of customers in both groups, graph 72b comparing gender distribution, graph 72c comparing age distribution, and graph 72d comparing e-commerce site usage. For example, graph 72d may be displayed when the customer data includes data on customers using the e-commerce site.

[0082] Graph 72d shows the distribution of GMS (total merchandise transaction amount) during the data acquisition period as data on the transaction amounts of customers constituting each of Groups 1 and 2. The legend for graph 72d is, for example, "no purchase," "low GMS: less than 10,000 yen," "medium GMS: between 10,000 yen and 100,000 yen," and "high GMS: 100,000 yen or more."

[0083] As shown in FIG. 10, the results screen 70 may include buttons 75, 76, and 77 at the bottom for displaying detailed information about the learning model 35. Operating button 75 displays detailed information about the model accuracy. Operating button 76 displays detailed information about important features in the model. Operating button 77 displays detailed information about the influence of top features on the model results.

[0084] Operating a tab in the navigation column 71 may display a display field 70c that displays information related to the model accuracy of the corresponding learning model 35. "Model Confidence" shown in the display field 70c indicates the reliability of the model. The reliability may be determined, for example, by the value of the AUC (area under ROC curve), with AUC>75 being high (HIGH), AUC>0.6 being medium (MEDIUM), and any other value being low (LOW). Operating button 75 displays detailed information about the model accuracy (see FIG. 11).

[0085] As shown in FIG. 11 , the detailed information about model accuracy may include a graph 75a showing an ROC curve. The area under the ROC curve (AUC) is used as an evaluation index for the corresponding learning model 35. The detailed information about model accuracy may also include a confusion matrix 75b of the learning model 35. The confusion matrix provides information indicating the agreement and disagreement between the predicted values ​​and actual values ​​of the model. The confusion matrix includes four values: true positive (TP), true negative (TN), false positive (FP), and false negative (FN).

[0086] As shown in FIG. 10 , the screen showing the comparative data for "Lifestyle," "Shopping," "Attitude," and "Other" may include a graph display area 73 and an insight display area 74. The graph display area 73 may include a selection button 73a for selecting the graph display format, and a graph 73b displayed in the selected format. The selection button 73a may include a choice between a "SHAP Bar Plot," which shows SHAP values ​​as a bar graph, and a "SHAP Bea Swarm Plot," which is a graph in which SHAP values ​​are plotted as points.

[0087] In the "SHAP bar plot" graph 73b, the SHAP values ​​of each customer are converted into a single summary value for the top 20 influential features (important attributes) among multiple features, and are displayed as a bar graph. This clearly shows the extent to which each feature contributes, positively or negatively. The "SHAP bee swarm plot" graph 73b is a graph in the format shown in Figure 2, and important attributes (for example, the top 20 features) are displayed in order of their influence.

[0088] The insight display area 74 displays a string of text outlining the results shown in the graph 73b. As an example, the insight display area 74 may display explanatory text about the comparison data, particularly the top five important attributes among the top 20 important attributes shown in the graph 73b. That is, the explanatory text may describe a smaller number of important attributes among the multiple important attributes. This number may be specified by a user input in the setting field 47, or if not specified, a default number may be used.

[0089] The character string displayed in the insight display area 74 may be automatically generated from the data of the analysis result and a pre-prepared template. For example, the processor 21 may generate a sentence such as "highly likely to have children in high school" by combining the name of the attribute (e.g., "has children in high school") with a character string indicating the result "highly likely" for the top five feature quantities with the highest SHAP values.

[0090] FIG. 12 illustrates a graph 76a that shows the cumulative composition ratio of the SHAP value, which is displayed when button 76 is operated. Graph 76a has the same format as graph 18 shown in FIG. 3. In graph 76a, the top 20 feature amounts (important attributes) are arranged from top to bottom in descending order of composition ratio. Therefore, graph 76a shows how important each feature amount is, i.e., which attributes are important. For example, the top five feature amounts can be said to be particularly important attributes among the important attributes.

[0091] Operating button 77 may display one or more graphs 78 (see FIG. 13 ) showing how each of the most important attributes affects the model results. Here, the number of top features to display may be set in setting field 47.

[0092] 13 shows graphs 78a, 78b, 78b, 78d, and 78e that show the influence on the model results of the top five feature quantities mentioned in the insight display area 74. Each of the graphs 78a to 78e includes a plurality of plots (dots) corresponding to a plurality of customers, with the horizontal axis representing the first to fifth ranked feature quantities and the vertical axis representing the SHAP value for each feature quantity.

[0093] The "Action" tab can be used to perform additional analysis for marketing based on the above-mentioned comparison data. In the additional analysis, for example, an additional learning model 35 may be generated to extract customers who have attributes common to customers in the target group from among customers outside the target group. The additional learning model 35 is constructed, for example, to predict customers who have similar attributes similar to the important attributes of the first group. Such an additional learning model 35 can extract a group of potential customers to be targeted for marketing.

[0094] [Description application] Although the insight display area 74 of the data generation application displays a simple explanation of the comparison data, the rest is displayed as numerical data or graphs, so the analyst must interpret the numerical data and graphs to perform a detailed analysis.

[0095] Because such analyses require knowledge and experience, it is difficult for inexperienced analysts to perform them properly. Even for analysts with some experience, interpreting results from multiple graphs can be time-consuming. Therefore, explanatory applications are designed to summarize comparative data and answer questions about the comparative data.

[0096] 14 shows an exemplary display screen 80 of an explanatory application. The display screen 80 is an example, and the design can be changed as desired. The display screen 80 may include a sidebar 81 and a main column 82.

[0097] The sidebar 81 may include, for example, a login username input field 83 and a dataset selection field 84. The dataset selection field 84 lists the names of datasets previously created by each user. When a user logs in by entering their username in the input field 83, they can select a dataset for a job they previously created in the dataset selection field 84.

[0098] The main column 82 may display the name of the dataset selected by the user. The main column 82 may include a "Q&A" tab 82a and a "Summary" tab 82b. When the "Q&A" tab 82a is selected, a question input field 85 is displayed in the main column 82. When a question is input in the question input field 85, an answer insight 87, which is an answer to the question, is displayed below the question input field 85 together with a graph 86. The answer insight 87 and the graph 86 are generated based on comparison data related to the same important attribute. In other words, the explanation application is configured to output the answer to the question in the form of a sentence (character string) when a question related to the comparison data is input.

[0099] When the "Summary" tab 82b is selected, the main column 82 displays summary insights 89 (see FIG. 15) summarizing the results of the data set (e.g., "Lager vs. Ale"), i.e., the explanatory application is configured to generate explanatory text summarizing the results of the contrast data.

[0100] As shown in Figure 15, summary insights 89 may be displayed in the main column 82 along with a corresponding graph 88. The summary insights 89 and graph 88 are generated based on contrast data relating to the same important attribute. Graph 88 is an example of a "SHAP Beam Swarm Plot" graph, but may also be a "SHAP Bar Plot" graph, or multiple graphs may be displayed.

[0101] Summary insights 89 may be generated for each of the items "general statistics," "lifestyle," "shopping," and "attitude" included in the comparison data 25. While summary insights 89 for "lifestyle" are illustrated in FIG. 15, summary insights 89 for other items may also be included in the main column 82.

[0102] [How insights are generated] The insights 87 and 89 may be generated using a general-purpose large-scale language model 14. The insights 87 and 89 may be written separately for each of the first and second groups, or may be written together to describe the differences between the two groups.

[0103] The memory 22 of the analysis device 20 may store one or more templates of prompts for input to the large-scale language model 14. Here, a prompt for obtaining answer insights 87 is referred to as a question prompt, and a prompt for obtaining summary insights 89 is referred to as a summary prompt. The one or more templates may include a question template used to generate a question prompt and a summary template used to generate a summary prompt.

[0104] The prompt preferably includes at least one of a setting, an instruction, and source data to enable the large-scale language model 14 to generate an appropriate completion. Thus, a template may include at least one of one or more setting statements, one or more data specification statements, and one or more instruction statements.

[0105] The prompts and templates may be written in a language (e.g., English) that is frequently included in the training data in the large-scale language model 14. In this case, the prompts and templates may include instructions to translate the generated sentences into the target language so that insights 87, 89 are generated in the written language of the question entered in the question input field 85 or the language set in the setting field 48.

[0106] Insights 87 and 89 can be said to be documents that should be generated by an analyst by looking at the numerical data included in the comparison data or various graphs (e.g., graphs 86 and 88) generated from the numerical data. Therefore, the prompt may include a setting sentence for setting the character of an analyst to the large-scale language model 14. For example, one or more setting sentences may include a setting sentence for prompting the analyst to respond as an analyst, such as "You are a data scientist."

[0107] The one or more data specification statements may include a statement specifying the data to be analyzed, such as "Please analyze [data name]," and the prompt may include data corresponding to the data name. The data included in the prompt may be numeric data such as a SHAP value, or a descriptive text (character string) generated based on the numeric data.

[0108] [Data Name] may be, for example, the name of a data set or the name of an analysis item such as "General Statistics," "Lifestyle," "Shopping," or "Attitude." The data name may correspond to the data in graphs 86 and 88 to be displayed together with insights 87 and 89. A sentence specifying the data may be included in the setting sentence, such as "You are the analyst who will analyze [Data Name]."

[0109] The one or more instructional statements may include multiple action instructions to be performed by the large-scale language model 14. In this case, the one or more instructional statements may include a general instruction statement such as "Your task is to perform the following action" and multiple action instruction statements each corresponding to a respective one of the multiple actions.

[0110] The prompt or template may include, for example, a series of action instructions such as (1) to (4) below, but is not limited to these. (1) Please summarize the top three contributing features.

[0111] (2) Regarding the top three features with the highest contribution, please explain why these features have such a high contribution. (3) Based on the results of (1) and (2) above, please propose an effective marketing method.

[0112] (4) Please summarize the results of (1) to (3) above. In addition to a simple summary like (1), action instructions can provide more useful information to analysts by including an analysis of factors like (2) and even suggestions for marketing methods like (3). Furthermore, as the number of action instructions increases, the text becomes longer and more complex, so adding a sentence summarizing the output content like (4) can further help analysts understand the results.

[0113] Before generating a question prompt, the explanation application is configured to extract data or attributes to be used as information sources based on the question sentence entered in the question input field 85. Hereinafter, a method for generating an insight 87 that is an answer to the question sentence will be described with reference to FIG.

[0114] In step S11, the processor 21 acquires a question sentence relating to a comparison between the first group and the second group input by the user. In the following step S12, the processor 21 selects one or more related attributes related to the question sentence from among a plurality of attributes. In step S12, for example, a natural language processing model such as the large-scale language model 14 may be used to select attributes that have a high similarity to the question sentence or words contained in the question sentence as related attributes.

[0115] For example, processor 21 may input a prompt to large-scale language model 14 to instruct it to select the top few (e.g., 20) attributes related to the question. In this case, the prompt may include a description of each of the attributes as a source of information for determining relevance. The description may be generated from the analysis result data and a pre-prepared template, such as the character string displayed in insight display area 74.

[0116] In this case, the large-scale language model 14 selects and outputs multiple top attributes that are highly relevant to the question sentence, without considering the SHAP value. The processor 21 may then exclude attributes with a SHAP value of 0 from the selected multiple top attributes and set the remaining multiple top attributes as relevant attributes. This is because attributes with a SHAP value of 0 do not contribute to comparing the target group with the comparison group.

[0117] In step S13, the processor 21 obtains the SHAP value (contribution) of the selected related attribute. If there are multiple related attributes, in step S14, the processor 21 selects a specified number of related attributes with high contributions as important attributes. For example, the processor 21 may select the top N related attributes with high positive correlations and the top M related attributes with high negative correlations as important attributes. The specified numbers (the values ​​of N and M) can be set arbitrarily, and may be, for example, default values ​​specified in the explanation application or values ​​specified by the user. N and M may be the same value (for example, 1, 2, 3, or 5) or may be different values. For example, the value of M may be greater than or smaller than the value of L.

[0118] In step S15, the processor 21 creates a question prompt. At this time, the processor 21 may generate the prompt using a question template including the setting sentence as described above. Here, the data specification sentence may include, as information sources, one or more important attributes selected in step S14 and the contribution (which may be a SHAP value) of each important attribute. The information source may be an explanatory sentence about one or more important attributes instead of the contribution (SHAP value). This explanatory sentence may be a character string generated from the analysis result data and a fixed phrase, such as input to the large-scale language model 14 to select related attributes in step S12. Furthermore, the instruction sentence includes an action instruction sentence for answering the question, generated using, for example, a natural language processing model.

[0119] In step S16, processor 21 inputs the generated prompt to large scale language model (LLM) 14. Then, in step S17, processor 21 obtains a completion output by large scale language model 14. Furthermore, in step S18, processor 21 may edit the completion to generate a response sentence.

[0120] For example, if the completion is written in English, analysis device 20 may use a different machine translation model to translate the completion into the same language as the question (e.g., Japanese). Additionally, the answer may be generated by adding a note to the completion about the accuracy of the analysis results or adding the date and time when the answer was generated.

[0121] In step S19, processor 21 outputs an answer sentence generated based on the output of large-scale language model 14 as answer insight 87. This answer sentence is a sentence including an answer to the question and is also an explanatory sentence describing the results of comparing the first group with the second group. In step S19, answer sentences written in both the description language of the completion (e.g., English) and the description language of the question (e.g., Japanese) may be output. In this case, insight 87 may be written in both the description language of the completion and the description language of the question.

[0122] [Analysis Chatbot] 17 shows an example of a chat screen 90 of the analytical chatbot. The analytical chatbot is configured so that a virtual analytical assistant, an "AI analyst," answers questions from users in a chat format.

[0123] The analytical chatbot is configured to analyze the question and generate the answer using the large-scale language model 14. When generating the answer, the analytical chatbot may acquire contrast data related to the question and include the contrast data in the prompt as an information source for the answer, as in steps S11 to S19.

[0124] The chat screen 90 includes an input field 91 for the user to input a question, and a send button 92. The chat screen 90 also includes one or more bot speech bubbles 93, which are fields for comments from the AI ​​analyst, and one or more user speech bubbles 94, which reflect the text entered in the input field 91. When the user inputs text in the input field 91 and operates the send button 92, a user speech bubble 94 containing the input text is added to the chat screen 90. The analytical chatbot outputs a response to the question entered by the user to the bot speech bubble 93 as a comment from the AI ​​analyst.

[0125] As a first example, the analytical chatbot may generate an answer based on a dataset already generated by a data generation application. In the first example, the analytical chatbot may output a bot speech bubble 93 containing a list of datasets on the chat screen 90 along with an instruction such as "Please select a dataset to be analyzed." Alternatively, the analytical chatbot may obtain corresponding comparative data as an information source in response to a question.

[0126] As a second example, the analytical chatbot may generate a new data set and then generate an answer for the data set. In this case, the analytical chatbot may instruct the user to input two customer groups corresponding to the first group and the second group, and generate comparative data for the two input customer groups. In the second example, the analytical chatbot may output a bot speech bubble 93 including a "Create a new job" button along with a question such as "Do you want to create a new job?" to the chat screen 90.

[0127] Hereinafter, with reference to FIG. 18, a method for the analytical chatbot to acquire already created comparison data as in the first example, or to generate new comparison data as in the second example, will be described. Note that the function (program code) for realizing the method described here may be the same as the function (program code) when the data generation application performs the process of acquiring a data set and creating a new job. Therefore, the analytical chatbot may share the program for executing these processes with the data generation application. Furthermore, the analytical chatbot may have the user sequentially input items to be entered on the input screen 40 in a chat format.

[0128] As shown in FIG. 18, the processor 21 may determine whether to generate a new job in step S21 based on the user's response to a question such as "Do you want to generate a new job?".

[0129] If the determination in step S21 is negative (NO), in step S22, processor 21 may display a selection of created datasets together with an instruction such as "Please select a dataset to be analyzed." Subsequently, in step S23, processor 21 acquires the dataset (comparison data) selected by the user.

[0130] Instead of step S22, processor 21 may display a question such as "What kind of data do you want to analyze?" and obtain a free-form answer from the user. In this case, processor 21 may extract relevant attributes, important attributes, or data sets including them related to the question, as in steps S11 to S14.

[0131] If the determination in step S21 is affirmative (YES), the processor 21 outputs an instruction such as "Please input the names of the analysis target and comparison target," thereby acquiring the first group name and the second group name in step S24.

[0132] In step S25, processor 21 generates a new job. Furthermore, processor 21 prompts the user to input conditions for data generation as necessary, and then in step S26 generates comparison data 25. The generation of comparison data 25 in step S26 can be performed in the same manner as steps S1 to S8 in FIG. 8.

[0133] In step S27, processor 21 stores the generated comparison data in memory 22. Thereafter, when a question about the generated comparison data is input from the user, processor 21 acquires the corresponding comparison data in step S28 and sets it as an information source of the answer.

[0134] Thereafter, the analytical chatbot may output an answer in response to a request input by the user based on the acquired comparison data. For example, when a question regarding certain comparison data is input, the analytical chatbot may generate an answer using the method shown in Fig. 16. In this case, the analytical chatbot may share a program for executing the process shown in Fig. 16 with the explanation application.

[0135] More specifically, the analytical chatbot may select a feature with a high contribution from the features related to the question, and input a prompt to the large-scale language model 14 to instruct the generation of an explanation for the feature. Then, the analytical chatbot may output an answer sentence generated based on the completion via the bot speech bubble 93. The analytical chatbot may also output a graph showing the comparative data that was the source of the answer together with the answer sentence.

[0136] [Effects of the present disclosure] The analysis device 20 can generate explanatory text based on the calculated numerical data. When the explanatory text is a concise description of important attributes as displayed in the insight display area 74, it helps to understand the overview of the comparison data. Furthermore, when the explanatory text includes an analysis of factors or a proposal for a marketing method, as in the summary insight 89, the analysis device 20 can assist the analyst in his or her work from multiple angles. Furthermore, when the explanatory text is output as a response to an open-ended question, as in the answer insight 87 or an analytical chatbot, the analysis device 20 can provide information that meets the analyst's needs.

[0137] The analysis device 20 can display explanatory text for the comparison data together with a graph generated using the same data as the source. This allows analysts to visually recognize the graph while reading the explanatory text and understanding it linguistically. While analysts differ in their cognitive characteristics, such as whether they are more likely to recognize visual information or linguistic information, many analysts have complex cognitive characteristics. Therefore, displaying a graph, which is visual information, and explanatory text, which is linguistic information, together can help analysts understand the data.

[0138] These explanations can also be generated using large-scale language models 14, such as answer insights 87 or analytical chatbots. However, a major issue is that the output of generative AI using typical large-scale language models often lacks accuracy. Therefore, the analysis device 20 can output more accurate answers by including contrast data, which serves as a source of information for the answer, in the prompt.

[0139] However, because there is a limit to the token length that can be entered as a prompt, it is not possible to include all of the customer data in the prompt. Furthermore, it is difficult to base an answer on simple customer data alone. In this regard, contrast data can be a concise yet useful source of information, as it contains one or more important attributes characteristic of the classification between the first and second groups and the contribution of each important attribute.

[0140] [Effects of this disclosure] According to the present disclosure, the following effects can be achieved. (1) By inputting the comparison data together with instructions into the large-scale language model 14, the analysis device 20 can output explanatory text describing the results of the comparison between the first group and the second group, in addition to numerical data such as the contributions of multiple attributes. This can help an analyst who analyzes the two customer groups, the first group and the second group, by looking at the comparison data.

[0141] (2) The summary insight 89 and the graph 88 are generated based on the same comparison data. Furthermore, since the summary insight 89 is displayed together with the corresponding graph 88, an analyst can more easily understand the data shown in the graph 88 by reading the summary insight 89.

[0142] (3) The explanation application's answer insight 87 or analytical chatbot outputs customer insights as answers to user questions. Therefore, users or analysts can understand the content of the comparison data by reading the explanation text without having to interpret the numerical data and graphs included in the comparison data.

[0143] (4) The answer insight 87 and the graph 86 are generated based on the same comparison data. Furthermore, the summary insight 89 is displayed together with the corresponding graph 88, so that the analyst can more easily understand the data shown in the graph 88 by reading the summary insight 89.

[0144] (5) The analysis device 20 generates comparative data of the first group with the second group and outputs explanatory text along with one or more graphs of the comparative data. The explanatory text helps the analyst to better understand the data shown in the one or more graphs, thereby assisting the analyst in analyzing the two customer groups, the first group and the second group, by viewing the comparative data.

[0145] (6) Graphs 73b, 86, and 88 show multiple important attributes and the contribution of each of the multiple important attributes, allowing analysts to understand the important attributes of customers that lead to the difference between the first and second groups and the extent to which these important attributes have an influence.

[0146] (7) In the graphs 73b and 88, the important attributes are arranged in descending order of their contribution. Therefore, by looking at the order of the important attributes, the user can intuitively understand the relationship between them and the degree of their influence.

[0147] (8) Multiple customers have transaction histories for at least one of multiple categories of products or services. The user can specify, via the input screen 40, that at least one of the first and second groups be extracted from multiple customers who have transaction histories for products or services in a specified category among the multiple categories. In this case, comparison data for customers in the specified category can be output.

[0148] (9) At least some of the customers have transaction histories with one or more brands that offer products or services across multiple categories. The user can then specify, via the input screen 40, to extract at least one of the first and second groups from among the customers who have transaction histories with products or services in a single designated category offered by a single designated brand. In this case, comparative data on customers in the designated category of the designated brand can be output.

[0149] (10) The attribute data includes the transaction amount of each customer's goods or services. Graph 72d displays data on the transaction amounts (e.g., GMS) of customers in the first and second groups in comparison. By outputting graph 72d, analysts can intuitively understand the consumption trends of purchasers in each of the first and second groups.

[0150] (11) Based on the contribution of each attribute obtained as a calculation result of the learning model 35, it is possible to estimate, from among multiple attributes, an important attribute that separates the first group from the second group.

[0151] (12) The comparison data includes the calculation results of multiple learning models 35, and the multiple learning models 35 are generated using attribute data relating to different combinations of attributes. The combinations of attributes can be, for example, predefined combinations such as general statistics, lifestyle, shopping, and attitude, or combinations arbitrarily selected by the user. By selecting characteristic combinations of attributes or related combinations of attributes from a large number of customer attributes and using them as learning data, the first group and the second group can be compared from multiple perspectives.

[0152] (13) By using the SHAP value, the contribution can be quantitatively calculated. [Example of change] This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0153] [Change Example 1] The various processes for realizing the data generation application, the insight summarization application, and the analytical chatbot may be distributed and executed by multiple analysis devices 20. In this case, not only may three analysis devices 20 individually realize the data generation application, the insight summarization application, and the analytical chatbot, but multiple processes required for realizing the three applications may be distributed and executed by any number of analysis devices 20.

[0154] [Change Example 2] When all the processes for realizing the data generation application, the insight summarization application, and the analytical chatbot are implemented by a single analysis device 20, the analysis system 11 is implemented by a single device (e.g., a computer). In this case, the single analysis device 20 may include the large-scale language model 14 and the machine learning device 30.

[0155] [Change Example 3] The first and second groups may each include the same customer group. In this case, by setting different data periods for the first and second groups, comparative data can be obtained on trends for the same customer group over two periods. For example, if a designated brand releases different new products each season or year, such data can be used to analyze the purchasing trends of the same customer group for those products.

[0156] [Change Example 4] The data generation application may have the functionality of an insight summarization application. For example, in the insight display area 74 of the data generation application, explanatory text (summary insights) generated using the large-scale language model 14 may be displayed, similar to the insight summarization application. Alternatively, the main column 82 may be displayed by operating the "Action" tab on the result screen 70. With this configuration, after generating contrast data, answer insights 87 and summary insights 89 about the contrast data can be obtained using the same data generation application. In addition, the data generation application, the insight summarization application, and the analytical chatbot may share some or all of their functions with each other.

[0157] [Change Example 5] The analysis device 20 may not have one or more of the functions of the data generation application, the insight summarization application, and the analytical chatbot. If the analysis device 20 does not have a data generation application, the insight summarization application and the analytical chatbot may acquire the comparison data already generated. In this case, the analysis system 11 may not have a machine learning device 30 for generating the learning model 35.

[0158] [Change Example 6] The insight summarization application may provide only answer insights 87 or summary insights 89 .

[0159] The following are some aspects that can be understood based on the above-described embodiment and modifications. [1] at least one memory configured to store a program; and at least one processor configured to perform one or more processes based on the content of the program, The one or more processes are a process for acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers, and the comparison data includes a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; a process of outputting an explanatory sentence describing a result of comparing the first group with the second group based on an output of the large-scale language model; an analysis system,

[0160] [2] The one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text. The analysis system according to [1] above.

[0161] [3] The one or more processes are: A process of acquiring a question input by the user regarding a comparison between the first group and the second group; selecting one or more relevant attributes from the plurality of attributes that are relevant to the question; a process of selecting a predetermined number of related attributes with high contribution rates from the one or more related attributes; Including, In the process of acquiring the comparison data, the predetermined number of related attributes are the plurality of important attributes, In the process of inputting into the large-scale language model, the one or more instruction sentences include an instruction sentence for answering the question, In the process of outputting the explanatory text, the explanatory text includes an answer to the question. The analysis system according to [1] above.

[0162] [4] The one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text. The analysis system according to [3] above.

[0163] [5] at least one memory configured to store a program; and at least one processor configured to perform one or more processes based on the content of the program, The one or more processes are A process of extracting a first group and a second group from a plurality of customers included in customer data based on a user instruction, wherein the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution of each of the important attributes to the comparison; generating one or more graphs of the contrast data; generating a description for the one or more graphs; outputting the legend together with the one or more graphs; an analysis system,

[0164] [6] The one or more graphs include a graph showing the plurality of important attributes and the contribution of each of the plurality of important attributes. An analysis system according to any one of [2], [4] and [5] above.

[0165] [7] The plurality of important attributes are arranged in descending order of the degree of contribution. The analysis system according to [6] above. [8] The plurality of customers have a transaction history of at least one of products or services across multiple genres, The user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated genre among the plurality of genres; The analysis system according to any one of the above items [1] to [7].

[0166] [9] At least some of the customers have transaction histories with multiple brands that offer products or services across multiple genres; The user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated genre that are offered by one brand out of a plurality of brands; The analysis system according to any one of [1] to [8] above.

[0167]

[10] The attribute data includes the transaction amount of each customer's goods or services; the one or more graphs include a graph that displays a comparison of data related to the transaction amounts of customers who constitute the first group and the second group, The analysis system according to any one of [1] to [9] above.

[0168]

[11] The comparison data includes calculation results of one or more learning models; each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation results include a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction. The analysis system according to any one of the above [1] to

[10] .

[0169]

[12] the one or more learning models include a plurality of learning models; the learning data used to generate the plurality of learning models includes attribute data relating to combinations of attributes that are different from each other; The analysis system according to

[11] above.

[0170]

[13] The contribution of each attribute in the prediction is calculated as a SHAP (SHapley Additive exPlanations) value. The analysis system according to

[11] or

[12] above.

[0171]

[14] One or more computers, a process for acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers, and the comparison data includes a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; a process of outputting an explanatory sentence describing a result of comparing the first group with the second group based on an output of the large-scale language model; 3. A method of analysis comprising:

[0172]

[15] One or more computers, A process of extracting a first group and a second group from a plurality of customers included in customer data based on a user instruction, wherein the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution of each of the important attributes to the comparison; generating one or more graphs of the contrast data; generating a description for the one or more graphs; outputting the legend together with the one or more graphs; 3. A method of analysis comprising:

[0173]

[16] One or more computers, a process for acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers, and the comparison data includes a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; a process of outputting an explanatory sentence describing a result of comparing the first group with the second group based on an output of the large-scale language model; A program that executes.

[0174]

[17] One or more computers, A process of extracting a first group and a second group from a plurality of customers included in customer data based on a user instruction, wherein the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution of each of the important attributes to the comparison; generating one or more graphs of the contrast data; generating a description for the one or more graphs; outputting the legend together with the one or more graphs; A program that executes. [Explanation of symbols]

[0175] 11...Analysis system, 12...Network, 13...Database, 14...Large-scale language model, 15...User terminal, 16...Display, 17-19, 72a-72d, 73b, 75a, 76a, 78, 78a-78e, 86, 88...Graph, 20...Analysis device, 21, 31...Processor, 22, 32...Memory, 23, 33...Communication interface, 24, 34...Program, 25...Contrast data, 30...Machine learning device, 35...Learning model, 40...Input screen, 41, 81...Sidebar, 42, 82...Main column, 43...Account input field, 44...Action selection field, 45...Settings button, 46-48...Option setting field, 50...Settings area, 51, 61...Name input field, 5 2, 56, 62, 66, 70b...selection field, 53, 63, 54, 64, 83, 91...input field, 55, 65, 73a...selection button, 59...confirmation button, 60...setting area, 69...decision button, 70...result screen, 70a...title, 70c...display field, 71...navigation column, 72...display area, 73...graph display area, 74...insight display area, 75-77...buttons, 75b...confusion matrix, 80...display screen, 82a, 82b...tabs, 83...user name input field, 84...dataset selection field, 85...question input field, 87...insights, 87...answer insights, 89...summary insights, 90...chat screen, 92...send button, 93...bot speech bubble, 94...user speech bubble.

Claims

1. at least one memory configured to store a program; and at least one processor configured to execute one or more processes based on the content of the program, The one or more processes a process for acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers, and the comparison data includes a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; a process of outputting an explanation describing a result of comparing the first group with the second group based on an output of the large-scale language model; an analysis system,

2. the one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text; The analytical system of claim 1 .

3. The one or more processes acquiring a question input by the user regarding a comparison of the first group with the second group; selecting one or more relevant attributes from the plurality of attributes that are relevant to the question; a process of selecting a predetermined number of related attributes having high contribution rates from the one or more related attributes; Including, In the process of acquiring the comparison data, the predetermined number of related attributes are the plurality of important attributes, In the process of inputting into the large-scale language model, the one or more instruction sentences include an instruction sentence for answering the question, In the process of outputting the explanatory text, the explanatory text includes an answer to the question. The analytical system of claim 1 .

4. the one or more processes include a process of outputting one or more graphs generated based on the comparison data together with the explanatory text; The analysis system according to claim 3 .

5. at least one memory configured to store a program; and at least one processor configured to execute one or more processes based on the content of the program, The one or more processes a process of extracting a first group and a second group from a plurality of customers included in customer data based on a user instruction, the customer data including attribute data relating to a plurality of attributes of each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution of each of the important attributes to the comparison; generating one or more graphs of the contrast data; generating a description for the one or more graphs; outputting the legend along with the one or more graphs; an analysis system,

6. the one or more graphs include a graph showing the plurality of important attributes and the contribution of each of the plurality of important attributes; 6. The analysis system according to claim 2, 4 or 5.

7. The plurality of important attributes are arranged in descending order of the degree of contribution. The analysis system according to claim 6 .

8. The plurality of customers have transaction histories of at least one of products or services across a plurality of categories, the user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated genre among the plurality of genres; The analysis system according to any one of claims 1 to 5.

9. At least some of the customers have transaction histories with multiple brands that offer products or services across multiple genres, The user's instruction includes extracting at least one of the first group and the second group from a plurality of customers who have transaction histories of products or services in one designated genre that are offered by one brand out of a plurality of brands; The analysis system according to claim 8 .

10. The attribute data includes a transaction amount of each customer's goods or services, the one or more graphs include a graph displaying a comparison of data relating to the transaction amounts of customers constituting the first group and the second group, 6. The analysis system according to claim 2, 4 or 5.

11. the comparison data includes calculation results of one or more learning models; each of the learning models is a classifier that predicts whether each of the plurality of customers belongs to the first group or the second group; the calculation result includes a prediction result of whether each of the plurality of customers belongs to the first group or the second group, and a contribution degree of each of the attributes in the prediction. The analysis system according to any one of claims 1 to 5.

12. the one or more learning models include a plurality of learning models; the learning data used to generate the plurality of learning models includes attribute data relating to combinations of attributes that are different from each other; The analytical system of claim 11.

13. The contribution of each attribute in the prediction is calculated as a SHAP (Shapely Additive exPlanations) value. The analytical system of claim 11.

14. On one or more computers, a process for acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers, and the comparison data includes a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; a process of outputting an explanation describing a result of comparing the first group with the second group based on an output of the large-scale language model; 3. A method of analysis comprising:

15. On one or more computers, a process of extracting a first group and a second group from a plurality of customers included in customer data based on a user instruction, the customer data including attribute data relating to a plurality of attributes of each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution of each of the important attributes to the comparison; generating one or more graphs of the contrast data; generating a description for the one or more graphs; outputting the legend along with the one or more graphs; 3. A method of analysis comprising:

16. On one or more computers, a process for acquiring comparison data between a first group and a second group, wherein the first group and the second group are each extracted from a plurality of customers included in customer data based on a user instruction, the customer data includes attribute data relating to a plurality of attributes possessed by each of the customers, and the comparison data includes a plurality of important attributes effective for comparing the first group with the second group and the contribution of each of the important attributes to the comparison; inputting one or more directives for contrasting the first group with the second group together with the contrast data into a large-scale language model; a process of outputting an explanation describing a result of comparing the first group with the second group based on an output of the large-scale language model; A program that executes.

17. On one or more computers, a process of extracting a first group and a second group from a plurality of customers included in customer data based on a user instruction, the customer data including attribute data relating to a plurality of attributes of each of the customers; A process for generating comparison data between the first group and the second group, the comparison data including a plurality of important attributes effective for comparing the first group with the second group and a contribution of each of the important attributes to the comparison; generating one or more graphs of the contrast data; generating a description for the one or more graphs; outputting the legend along with the one or more graphs; A program that executes.

Images