Information processing systems, information processing methods, and programs

Abstract

To enable efficient structuring and analysis of VoC data. [Solution] An information processing system comprising: a VoC data acquisition unit that acquires VoC data from customers; and a labeling unit that inputs the VoC data and prompts containing instruction information specifying judgment criteria for each of the 5W1H items into a large-scale language model and assigns labels to the VoC data based on the components of the 5W1H.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses a technique for calculating the degree of potential customers according to a predetermined calculation logic using customer attribute information, contract information, transaction history, and the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Information such as customer demographic data, contract information, and transaction history is relatively easy for the business operator to obtain. However, it is difficult to grasp customer insights (true feelings, requests, motivations, etc.) based only on such superficial information. On the other hand, the voice of the customer (VoC: Voice of Customer), such as posts, reviews, word-of-mouth, and testimonials sent by the customers themselves, contains insights based on the customers' own feelings and experiences and is an important information source for extracting customer insights. However, the content of such VoC data is not stereotyped and has a large variation in expression, making it difficult to organize the meaning using conventional numerical analysis methods.

[0005] An object of an exemplary embodiment of the present disclosure is to provide a technique capable of efficiently structuring and analyzing VoC data.

Means for Solving the Problems

[0006] An information processing system according to an aspect of the present invention is A VoC data acquisition unit that acquires VoC data from customers, The system includes a labeling unit that inputs the VoC data and prompts containing instruction information specifying judgment criteria for each of the 5W1H items into a large-scale language model, and assigns labels to the VoC data according to the components of the 5W1H.

[0007] According to the present invention, VoC data can be efficiently structured and analyzed. Other problems disclosed in this application and their solutions are clarified in the section on embodiments of the invention and in the drawings. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram illustrating the configuration of an information processing system according to one embodiment of the present disclosure. [Figure 2] Figure 2 is a block diagram illustrating the hardware configuration of the management server shown in Figure 1. [Figure 3] Figure 3 is a block diagram illustrating the functions of the management server shown in Figure 1. [Figure 4] Figure 4 is a flowchart showing an example of information processing performed by the information processing system shown in Figure 1. [Modes for carrying out the invention]

[0009] An information processing system according to one embodiment of this disclosure will be described with reference to the drawings. In the attached drawings, identical or similar elements are denoted by identical or similar reference numerals and names, and redundant descriptions of identical or similar elements will be omitted as appropriate in the description of the embodiment. The contents shown in each drawing are illustrative examples for explaining this embodiment and are only shown in a schematic manner to facilitate understanding. They may be modified or changed to the extent that it does not impede technical limitations.

[0010] <System Overview> The information processing system according to this embodiment (hereinafter referred to as "this system") is a system that performs processing related to the analysis of customer voice data (VoC data) for products (goods and / or services) provided by a user company.

[0011] In this embodiment, "VoC data (Voice of Customer)" refers to the collective information relating to opinions, evaluations, experiences, questions, requests, and other expressive acts voluntarily or at the request of a customer, regardless of the medium, format, or acquisition route. The data format of VoC data may include data such as text, audio, images, and videos. This system structures VoC data from the perspectives of Who, What, When, Where, Why, and How by providing a predetermined prompt and VoC data to a large-scale language model and assigning 5W1H labels. In this embodiment, "label" refers to metadata assigned to describe the context and meaning of VoC data.

[0012] As shown in Figure 1, the information processing system of this embodiment may include, for example, a management server 1, an LLM server 2, one or more corporate terminals 3, and one or more customer terminals 4. Each of these devices is connected to each other so as to be able to communicate via a network NW. In this embodiment, the network NW is mainly assumed to be the internet, but it may also be constructed using a public telephone network, a mobile phone network, a wireless communication network, Ethernet (registered trademark), etc. Furthermore, if necessary, external systems such as social networking services, billing systems, and membership management systems may be connected to the management server 1 and other components so as to be able to communicate via the network NW.

[0013] Management Server 1 is an information processing device managed by the service provider that provides services related to this system. Management Server 1 may be built on-premises using a general-purpose computer such as a workstation or personal computer, or it may be logically configured using cloud computing. Some of the functions of Management Server 1, described later, may be implemented as functions realized by the processor of other devices such as enterprise terminals 3, or they may be implemented in a distributed manner on external servers.

[0014] LLM Server 2 is a server that operates a Large Language Model (LLM) to provide services such as text generation and question answering. In this system, the LLM receives prompts and VoC data sent from Management Server 1 as input and performs labeling processing for each VoC data based on this. The generated labels are sent back to Management Server 1, which stores these labels in association with the corresponding VoC data. LLM Server 2 may be configured as a physical server installed in an on-premises environment or as a virtual server built on a cloud computing service.

[0015] Enterprise terminal 3 is an information processing terminal of a user organization that uses the services of this system. Enterprise terminal 3 is used to set prompts and output labeling results and analysis results. Enterprise terminal 3 is intended to be used by the system administrator or marketing personnel of the user organization, and may be a general-purpose computer such as a workstation or personal computer, or a mobile terminal such as a smartphone or tablet. In the description of this embodiment, when "user" is used, the "user" refers to the person in charge of the user organization who operates enterprise terminal 3.

[0016] The customer terminal 4 is an information processing terminal used by customers (general consumers) who purchase and / or use the products provided by the user organization. The customer terminal 4 is used to input VoC data in a predetermined data format such as free description or selection format via media such as a questionnaire form, a membership app, a community tool, or SNS. The customer terminal 4 is realized by any terminal such as a personal computer, a smartphone, or a tablet, and is connected to the management server 1 via the network NW.

[0017] <Hardware Configuration> As shown in FIG. 2, the management server 1 includes a processor 10, a memory 11, a storage 12, a transceiver 13, an input / output unit 14, etc., which are electrically connected to each other through a bus 17. Note that the illustrated configuration of the management server 1 is an example, and the management server 1 may have other configurations.

[0018] The processor 10 is an arithmetic device that controls the operation of the entire management server 1, controls the transmission and reception of data between each element, and performs information processing necessary for the execution and authentication processing of applications. For example, the processor 10 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and executes a program for this system stored in the storage 12 and expanded in the memory 11 to perform each information processing described later.

[0019] The memory 11 includes a main memory composed of a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory composed of a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive). The memory 11 is used as a work area of the processor 10, and stores a BIOS (Basic Input / Output System) executed at the startup of the management server 1 and various setting information.

[0020] The storage 12 stores various programs such as application programs for executing the information processing shown in this embodiment. A database storing data used for each information processing may be constructed in the storage 12. For example, the storage unit 120 described later may be provided in a part of the storage areas of the memory 11 and / or the storage 12.

[0021] The transmission / reception unit 13 is a communication interface for connecting the management server 1 to the network. Note that the transmission / reception unit 13 may include a short-range communication interface for Bluetooth (registered trademark) and BLE (Bluetooth Low Energy).

[0022] The input / output unit 14 includes input devices such as keyboards and mice, and output devices such as speakers and displays. In the management server 1, the input device and the output device may be mounted on the management server 1 as individual devices, or a touch panel type display having both the functions of the input device and the output device may be mounted on the management server 1.

[0023] The bus 17 is commonly connected to the above elements, and transmits, for example, address signals, data signals, and various control signals.

[0024] Note that the hardware configurations of other devices such as the LLM server 2, the enterprise terminal 3, and the customer terminal 4 are the same as those of the management server 1. The LLM server 2, the enterprise terminal 3, and the customer terminal 4 all include a processor, a memory, a storage, a transmission / reception unit, an input / output unit, etc., and these are electrically connected to each other through a bus. These components are the same as the processor 10, the memory 11, the storage 12, the transmission / reception unit 13, the input / output unit 14, and the bus 17 of the management server 1, and the description thereof is omitted.

[0025] <Functions of the information processing system (software configuration)> Figure 3 is a block diagram illustrating the functions implemented in this system. The management server 1 may include a VoC acquisition unit 101, a preprocessing unit 102, a prompt generation unit 103, a label assignment unit 104, a completion unit 105, and a clustering unit 106. The various functional units of the management server 1 are realized by the processor 10 reading data and programs stored in the storage 12 and executing various programs in the working area of ​​the memory 11. The storage unit 120 of the management server 1 may also include a VoC data storage unit 121, a customer information storage unit 122, a prompt storage unit 123, and a label information storage unit 124. Each of these storage units stores data necessary for various information processing in this system and is configured to be accessible from each functional unit executed by the processor 10.

[0026] The VoC data storage unit 121 stores acquired VoC data and manages each data entry by assigning unique identification information to it. The VoC data storage unit 121 can also store VoC data imported in CSV format.

[0027] The VoC data storage unit 121 may store the original VoC data at the time of collection and the normalized data after preprocessing in association with each other. Each VoC data may be associated with information to identify the originating customer, and such information may include the customer account identifier in the acquisition system, the type of the identifier, and the type of media from which the data was acquired. Furthermore, metadata such as the collection time, acquisition name, language, character code, and flags related to data quality (presence or absence of missing data, results of character encoding detection, etc.) may be added and managed. From the perspective of privacy protection, the system may be configured to store highly personally identifiable attributes in accordance with prescribed regulations, by masking or pseudo-naming them.

[0028] The customer information storage unit 122 stores information about the customer. The customer information storage unit 122 stores customer ID, customer contact information, demographic attribute information, etc. Demographic attribute information includes information such as age, gender, residential area, occupation, and income.

[0029] The prompt storage unit 123 stores prompts used in this system, such as prompt templates used when labeling VoC data. In addition to prompts for labeling, the prompt storage unit 123 may also store prompt templates used for clustering.

[0030] The label assignment prompt may include, for example, instructional information specifying the criteria for each of the 5W1H items, information showing output examples, or a field for attaching VoC data. The instructional information may include definitions of the 5W1H classification labels (definitions of the item names what, when, where, who, how, and why), and information indicating criteria such as how to represent the values, selection policy when there are multiple candidates, how to handle output values ​​when they cannot be predicted, degree of concretization or summarization, and conditions for uniform notation. The clustering template may include instructional information defining clustering rules such as the number of clusters, criteria for determining homogeneity, and how to represent the output.

[0031] The label information storage unit 124 stores the 5W1H label information assigned to the VoC data. The label information storage unit 124 stores information for each of the following items corresponding to each VoC data: Who, What, When, Where, Why, and How.

[0032] In addition to the database described above, the storage unit 120 may also have a corporate information storage unit that stores information about user organizations. The corporate information storage unit may store, for example, master information about products provided by the user organization (product name, service name, price, specifications, etc.), brand information, and sales channel information (information about physical stores, information about e-commerce sites, etc.) associated with identification information (corporate ID) to uniquely identify the user organization. The storage unit 120 may also have a database that stores the results of analysis using VoC data, such as clustering results. Furthermore, when VoC data is acquired from community tools (member sites, member apps, etc.), e-commerce sites, SNS, etc. via API integration, the storage unit 120 may also store configuration information used for such integration (connection destination identifier, authentication token or key, acquisition target scope, acquisition cycle or presence or absence of event integration, threshold for rate control, callback URL, etc.).

[0033] The VoC acquisition unit 101 acquires VoC data from customers. The acquisition route for VoC data is not particularly limited. For example, the VoC acquisition unit 101 may acquire VoC data from external systems such as community tools (member apps, member sites, etc.), e-commerce sites, survey systems, and social networking services by using technologies such as API integration.

[0034] For example, when acquiring VoC data from community tools such as membership sites, VoC data such as product reviews and testimonials posted on the tool during a specified period may be acquired via API integration. Alternatively, when collecting VoC data from social media, posts containing the name of the product provided by the user organization may be extracted and acquired using that keyword, or posts made to the user organization's official account may be acquired.

[0035] In addition to acquiring VoC data from external systems as described above, the VoC acquisition unit 101 may also acquire VoC data collected by the user organization from the corporate terminal 3. In this case, the VoC acquisition unit 101 may acquire VoC data by presenting a graphical user interface (hereinafter referred to as GUI) for data import to the corporate terminal 3 and accepting input of the VoC data collected by the user organization via the GUI.

[0036] The VoC acquisition unit 101 may acquire VoC data from multiple sources. Furthermore, when acquiring VoC data, it may acquire supplementary information such as information indicating the source medium, the date and time of posting the VoC data, and information identifying the customer who posted the VoC data (e.g., customer identification information or account information on community tools, e-commerce sites, social media, etc.) and link this information to the VoC data. The acquired VoC data is recorded in the VoC data storage unit 121 described above.

[0037] The preprocessing unit 102 performs rule-based preprocessing on the collected VoC data. For example, as a preliminary step before performing labeling, the preprocessing unit 102 may perform denoising, normalization, or a combination thereof, on the VoC data. Denoising may involve removing or neutralizing non-verbal elements that hinder the interpretation of LLM text, such as emojis, URLs, hashtags, line break tags, special symbols, and machine-dependent characters included in SNS and community posts. It may also include removing unnecessary whitespace, redundant sentences, and formulaic greetings.

[0038] The normalization process may include, for example, a process of unifying different notations with the same meaning. For example, it is possible to perform normalization of notation fluctuations such as unifying "できる", "出来る", "できます", and "出来ます" to "できる", or normalization of converting synonyms such as "美味しい", "美味", and "うまい" into representative terms. In addition, it may include the unification of alphabetic notations, the normalization of full-width and half-width characters, and the process of converting symbols such as emojis representing emotional expressions into words or phrases representing their meanings (for example, emotional words such as "嬉しい", "悲しい", and "驚いた"). As a result, the vocabulary system of the input sentence is standardized, and the accuracy of semantic analysis by the LLM is improved.

[0039] Furthermore, the normalization process executed by the preprocessing unit 102 may include a process of converting the text of the VoC data into a predetermined data structure. For example, the preprocessing unit 102 performs preprocessing of converting the VoC data into a format of a predetermined data structure without summarizing the VoC data. The predetermined data structure may have a description format including a set of keys and values for dividing text data. In this case, the key may be specified by the user or defined by default. In this normalization process, the VoC data is divided into sentence units or paragraph units, or divided into word units, and converted into a predetermined data structure defined in advance. For example, it may be converted into structured data such as JSON format having a pair of keys and values. In this case, "customer ID", "utterance content", "utterance date and time", "media type", "language", "sentence number", etc. may be used as keys, and the corresponding values may be stored respectively.

[0040] In addition, the preprocessing unit 102 may be provided with a validation process for checking the data quality. For example, when the input data includes a sentence with an extremely small number of characters or a sentence whose language identification is difficult, the data may be excluded or output as a separate category. The preprocessing result is stored in the VoC data storage unit 121 in the storage unit 120 of the management server 1 and held in a state that can be referred to by each functional unit such as the subsequent labeling unit 104.

[0041] The prompt generation unit 103 generates prompts that are input to the Large-Scale Language Model (LLM) during the labeling process. A "prompt" is instruction information given to the LLM. Prompts for labeling include, for example, basic instruction information that instructs the labeling (post classification) task, definitions of 5W1H, and instruction information that specifies the criteria for each item of 5W1H. In addition, output rule information such as how to write labels may be included, and reference information that is helpful when performing the task, such as output examples, may also be included. The criteria for judgment are not necessarily limited. For example, the criteria may include: "What should be extracted prioritizing the product / service name, and if none is available, extract the subject"; "When should be extracted prioritizing the scene over the time of day"; "Where should emphasize the specificity of the location"; "Who should extract the single most important person"; "How should it be used"; and "Why should indicate the reason for choosing that product / service."

[0042] The instruction information described above may be defined in a pre-prepared template. The prompt generation unit 103 generates prompts to be given to the LLM by applying the acquired VoC data, such as by attaching it to a prompt template. At this time, the prompt generation unit 103 may present a list of VoC data stored in the VoC data storage unit 121 to the enterprise terminal 3 and allow the user to select the VoC data to be analyzed. Alternatively, the prompt generation unit 103 may present standard phrases for instruction information, such as the definition of 5W1H and the judgment criteria for each item of 5W1H, to the enterprise terminal 3 and accept editing of the instruction information by the user.

[0043] Furthermore, the prompt generation unit 103 may perform a process to generate prompts used by the clustering unit 106, which will be described later. The prompt for clustering may include basic instruction information that instructs the clustering task and clustering rules, or it may include instruction information that specifies the output format. The prompt generation unit 103 also generates the clustering prompt by applying the labeled VoC data to a pre-prepared prompt template, such as by attaching it. In this case, the prompt generation unit 103 may accept user input specifying the clustering rules from the enterprise terminal 3 and set that input information as instruction information in the prompt, or it may present a standard phrase for instruction information to the enterprise terminal 3 and accept editing by the user.

[0044] The labeling unit 104 performs the process of assigning labels to each VoC data using the 5W1H components. Specifically, the labeling unit 104 inputs the VoC data to be analyzed and prompts containing instruction information specifying the judgment criteria for each of the 5W1H items (what, when, where, who, how, why) into a large-scale language model (LLM), and causes it to perform the process of assigning labels to the VoC data using the 5W1H components. The large-scale language model is a natural language processing model such as GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), or T5 (Text-to-Text Transfer Transformer).

[0045] The prompts used by the labeling unit 104 specify concrete criteria for each of the 5W1H items. For example, the prompts may include instructions such as "Extract customer attributes or position" for the Who item, "Extract the target product or service" for the What item, and "Extract the date or period in YYYY / MM / DD format" for the When item. Multiple VoC data may be attached together as the target of analysis in the prompts, and even in this case, each VoC data will be assigned a label for each of the 5W1H items. The labels for each of the 5W1H items may be assigned in the form of a single value.

[0046] As an example of labeling, consider a case where the following is given as VoC data: Review text "I don't have much time on weekdays and mornings are hectic, so I tried meal kit A. I was able to prepare it in my kitchen at home in 10 minutes before going to work, which was a great help." When this text and a prompt are given to the LLM, the LLM will output single values ​​as responses, such as what is "meal kit A", when is the scene "preparing a meal before going to work", where is "my kitchen at home", who is "the reviewer", how is "quick cooking", and why is "I chose it because I don't have much time on weekdays and it can be prepared quickly". The labeling unit 104 registers the response values ​​for each of the 5W1H items as 5W1H label information linked to the VoC data in the label information storage unit 124. Alternatively, the VoC data and the 5W1H label information attached to that data may be output to the enterprise terminal 3 as processing results.

[0047] After receiving a response to a prompt from the LLM, the labeling unit 104 may check whether the data conforms to the prompt's instructions, such as whether there are any missing or extra items, whether the data types are correct, whether the data is represented by a single value, and whether it conforms to the notation conditions. If the check is successful, the labeling unit 104 may determine the items as what, when, where, who, how, and why for the target VoC data. If the data does not conform to the instructions, minor format inconsistencies may be corrected according to predetermined correction rules, and if correction is not possible, the data may be re-executed or put on hold. In addition, the LLM may be instructed to re-label VoC data that has been labeled as non-conforming.

[0048] The labeling unit 104 may output "Not Applicable" for items among the 5W1H items for which inference is impossible. For example, the labeling unit 104 may include instruction information in the prompt such as "For items where information is insufficient in the VoC data and cannot be inferred, always output 'Not Applicable'," and instruct the LLM to execute the task at that prompt. As a result, the LLM will output "Not Applicable" for information that cannot be clearly read from the VoC data, without attempting to fill in the gaps through inference.

[0049] The completion unit 105 performs a process to complete the items that are impossible to guess and are marked as "not applicable" among the 5W1H labeling results assigned by the labeling unit 104. The completion unit 105 identifies the customer ID corresponding to the VoC data that includes the "not applicable" items. It may then perform a process to send a questionnaire to the customer that includes questions corresponding to the "not applicable" items. The completion unit 105 creates a questionnaire to collect additional information regarding the "not applicable" items. For example, if the "When" item is not applicable, it creates a questionnaire that includes questions such as "Around what time did you have your experience?", or if the "Where" item is not applicable, it creates a questionnaire that includes questions such as "Which store did you have your experience?".

[0050] The method for identifying customers is not particularly limited. The supplementary unit 105 may identify the customer ID by comparing the VoC data with the customer information stored in the customer information storage unit 122. For example, if customer account information is linked when the VoC data is acquired, the customer ID may be identified by searching the customer information stored in the customer information storage unit 122 based on that account information. After identification, the supplementary unit 105 may generate a questionnaire that includes a question corresponding to the "Not Applicable" item, using the question text set on the corporate terminal 3 or a predefined standard question text.

[0051] The supplementary unit 105 sends the above-described questionnaire to the customer terminal 4 by any method. The transmission route is not particularly limited and may include, for example, notifications / messages on community tools such as member sites and member apps, messages via official SNS accounts, or My Page notifications on e-commerce sites. The questionnaire may also include the answer format (free response or multiple choice) and the deadline for response. Furthermore, the correspondence between the identifier of the transmitted questionnaire and the target VoC data may be recorded, and when a response is received to the questionnaire, the original VoC data corresponding to the response may be identified based on this correspondence.

[0052] When a customer returns a survey response, the completion unit 105 stores the customer's response in the label information storage unit 124 and completes the items that were marked as "not applicable". At this time, the completion unit 105 may also update the labels for the items that were marked as "not applicable" by providing the original VoC data and the survey response to the LLM and having it perform labeling again.

[0053] The supplementary unit 105 may perform a process to grant a reward to the customer who has responded to the survey. The reward to be granted is not particularly limited, and may include, for example, the awarding of points, the issuance of coupons, the granting of access to exclusive content, or the application of discounts.

[0054] The clustering unit 106 performs a process of clustering VoC data based on the 5W1H labels assigned by the labeling unit 104. In clustering, VoC data is classified into a predetermined number of groups based on the similarity of at least one item of the 5W1H. For example, VoC data for products with the same "What" item and VoC data for reasons with the same "Why" item may be grouped together to form clusters. Clusters may be formed based on the similarity of any one of the 5W1H items, or on the similarity of two or more of the 5W1H items. In clustering, the user may be allowed to select which aspects of the 5W1H to use for cluster formation and which aspects to emphasize. When multiple items are used to form clusters, weights may be set for each of the 5W1H items according to their importance for cluster formation, and the similarity between VoC data may be calculated accordingly.

[0055] The clustering conditions (rules) described above may be predetermined based on conditions specified by the user (person in charge of the user organization), or default clustering conditions based on rules of thumb may be used. The clustering unit 106 may perform rule-based clustering based on these conditions, or it may use a large-scale language model (LLM) and have the LLM perform the clustering. In the latter case, the clustering unit 106 inputs VoC data with 5W1H labels and prompts defining the clustering rules into the large-scale language model and has the large-scale language model perform the clustering.

[0056] The prompts used for clustering may include instructional information such as specifying the number of clusters, criteria for determining similarity, the method for outputting a summary representation of each cluster, and the output format. For example, the prompts for clustering may include instructions such as "Group VoCs with similar content" and "Assign a representative name to each group." The prompts are generated based on a clustering template stored in the prompt storage unit 123, and the VoC data after labeling is attached. The number of clusters may be fixed, or it may be variable depending on the number of data points or the data distribution in the population.

[0057] The clustering unit 106 may accept the range of VoC data to be used as input for the process (period, medium, product, etc.) from the enterprise terminal 3 before performing clustering, and may extract the 5W1H labels of the data selected by the user and apply them to the prompt. Similarity is determined based on the clustering rules defined in the prompt. For example, the prompt may instruct a policy to prioritize the similarity of the When and / or why labels, while considering the similarity of how as a secondary consideration when forming clusters.

[0058] The response of the large-scale language model may include the cluster name (representative topic), representative example (example), and cluster definition (summary), along with the assignment result of each VoC data to a cluster. The clustering unit 106 registers the cluster assignment result in association with the VoC data identifier in the storage unit 120, such as the VoC data storage unit 121. The clustering unit 106 may also output the clustering result to the enterprise terminal 3. The cluster name and its definition may be editable on the enterprise terminal 3, and the edited name may be retained as the final display name of the cluster.

[0059] The clustering unit 106 may accept instructions from the enterprise terminal 3 for re-execution or reorganization. For example, it may be configured to accept instructions to increase or decrease the number of clusters, change the weight of specific 5W1H items, edit the name of a specific cluster, or merge or split it, and then generate a prompt again to re-input into the large-scale language model. The re-execution results may be displayed together with the previous results for history reference.

[0060] The clustering results are used for various analytical processes, such as analysis for understanding customers and analysis of action hints for formulating measures to be implemented in business activities. For example, statistics on the number of posts per cluster may be calculated, or the trend in the number of posts for each cluster may be compared over a period and output in a dashboard format. Alternatively, analysis may be performed to infer customer insights (requirements, motivations, etc.) for each classified cluster based on the VoC data belonging to the cluster and its labels. Furthermore, VoC data may be extracted from minority opinions that can be grouped into other clusters, or VoC data that is unique in how the product is used or utilized, as indicated by the "how" label, and action hints may be generated by the LLM based on this data. Additionally, action hints may be generated for each cluster based on information such as typical usage scenarios (when), typical usage motivations or behavioral motivations (why), and typical usage locations (where) for each cluster, which are estimated from the statistics of the VoC data for each cluster. The management server 1 may have a function unit (analysis unit) that executes the above analyses on a rule basis, or that executes them on the LLM.

[0061] <An example of an information processing method> Figure 4 is a flowchart showing an example of information processing performed by this system.

[0062] First, the VoC acquisition unit 101 collects VoC data using a predetermined acquisition method (step S1). The acquisition route and acquisition method are not particularly limited and may include acquisition from an external system via API linkage, input from the corporate terminal 3, or collection by other appropriate means. Furthermore, the acquisition of VoC data may be performed in response to user instructions from the corporate terminal 3, or it may be performed periodically. Subsequently, the preprocessing unit 102 performs noise reduction, normalization, and structuring on the acquired VoC data (step S2). The preprocessing in step S2 is optional and may be omitted.

[0063] The prompt generation unit 103 generates a prompt for labeling by applying the VoC data preprocessed in the previous step, based on the template held in the prompt storage unit 123 (step S3). The prompt generation unit 103 may also present the generated prompt, or a prompt template with default instruction information, to the enterprise terminal 3 and accept editing from the user.

[0064] The label assignment unit 104 inputs the prompt and VoC data into a large-scale language model and assigns a 5W1H label to each VoC data (step S4). The assignment results are registered in the label information storage unit 124 and output to the enterprise terminal 3 (step S5). Next, it is determined whether or not the result includes an item labeled "Not applicable" (step S6).

[0065] If there is an item marked "Not Applicable," the supplementation unit 105 identifies the customer ID associated with the VoC data and sends a questionnaire containing the corresponding question to the customer terminal 4 (step S7). When a response is received, the label for the item is updated and registered to reflect the response (step S8). Steps S6-S8 are optional and may be omitted. Regardless of whether the supplementation processing in steps S6-S8 is performed, the clustering unit 106 performs clustering based on the assigned 5W1H labels according to predetermined rules or clustering prompts (step S9). The results of cluster assignment and summarization are registered in the storage unit 120, such as the label information storage unit 124, and output to the corporate terminal 3 to complete the process (step S10). The flowchart shown in Figure 4 is merely an example, and steps may be added, changed, or rearranged as appropriate.

[0066] According to the information processing system of this embodiment, customer Voice of the Customer (VoC) data can be structured using 5W1H labels assigned with a large-scale language model. As a result, compared to conventional manual analysis, large amounts of VoC data can be processed efficiently and converted into an analyzable format. Furthermore, items that are difficult to infer can be output as "not applicable," and by sending questionnaires to customers to supplement the missing information, more accurate and complete structured data can be obtained. In addition, by clustering the structured VoC data, similar customer voices can be grouped, enabling effective analysis and decision support.

[0067] The embodiments described above are merely illustrative to facilitate understanding of this disclosure and are not intended to limit it. This disclosure may be modified and improved without departing from its intent, and its equivalents are included.

[0068] For example, the processing performed by each functional unit of the management server 1 described above may be executed by any of the functional units. Furthermore, different functional units may be added to perform some of the processing performed by each of the functional units described above. Also, the functional units of the management server 1 may be distributed across multiple computers.

[0069] Furthermore, the information stored in each memory unit of the management server 1 may be stored in any of the memory units. That is, the information stored in the multiple memory units mentioned above may be stored in a single memory unit, or a portion of the information stored in one memory unit may be stored in another memory unit.

[0070] <Example 1> In the embodiment described above, an example was shown in which labels are assigned to all 5W1H items, but it is also possible to assign labels only to specific items specified by the user. In this case, the labeling unit 104 generates prompts corresponding to the items specified by the user via the user interface (for example, only What and Why) and inputs them into the large-scale language model. This makes it possible to efficiently extract only the necessary items according to the purpose of the analysis.

[0071] <Modification 2> In the embodiment described above, an example was shown in which labeling is performed using a single large-scale language model. However, different large-scale language models specialized for each of the 5W1H items may also be used. In this case, the labeling unit 104 executes multiple models optimized for each item in parallel, such as a model specialized for extracting Who items and a model specialized for extracting What items. Each model is trained with specialized training data related to its respective item, enabling more accurate labeling.

[0072] <Variation 3> In the embodiment described above, an example was shown of sending a questionnaire to the customer for items that did not apply. However, items that did not apply may be estimated from similar past VoC data. In this case, the interpolation unit 105 searches for data with similar content to the VoC data currently being processed from past VoC data stored in the label information storage unit 124, and calculates an estimated value by referring to the value of the corresponding item in the similar data. If the confidence level of the estimated value exceeds a threshold, the estimated value is automatically interpolated for items that did not apply. If the confidence level is low, the questionnaire is sent.

[0073] <Modification 4> In labeling, the 5W1H items may be expanded or subdivided. For example, "when" may be subdivided into "time of day," "scene," "season," etc., and "why" may be divided into "preference motive," "constraining factors," and "reasons for not adopting alternatives," so that the labeling unit can also assign single values ​​to the expanded items. The prompt instruction information includes definitions and judgment criteria for the expanded items, and the clustering unit calculates similarity by weighting the expanded items. This enables multifaceted analysis, including the background of preferences and usage constraints.

[0074] <Modification 5> Furthermore, processing may be performed to convert VoC data obtained in data formats other than text. For example, the VoC acquisition unit 101 receives audio, images, videos, etc., and the preprocessing unit converts it into text or metadata using speech recognition, in-image character recognition, or scene description extraction, etc., before supplying it to the labeling unit. The labeling unit uses the text conversion result and metadata (candidate shooting location, time information, object tags, etc.) to determine the 5W1H, and marks items that cannot be inferred as "not applicable". This makes it possible to extract insights along the same axis even from non-text-centric VoC such as review videos and photo posts.

[0075] <Variation 6> Furthermore, the system may perform processes to dynamically adjust the prompts. For example, the prompt generation unit 103 automatically switches the judgment criteria for each item and the degree of emphasis on output examples from a template according to the analysis objective specified from the corporate terminal (e.g., new product evaluation, understanding purchasing barriers, detection of signs of churn). In addition, it may refer to operational indicators such as the "not applicable" occurrence rate and the re-execution rate for each item in the most recent labeling results, and make adjustments such as tightening the definitions of items that are often ambiguous. This ensures stable consistency that can keep pace with changes in the operating environment and objectives.

[0076] <Example 7> In the embodiment described above, an example of performing clustering using a large-scale language model was shown, but clustering may also be performed by vectorizing the 5W1H label information and using a machine learning algorithm. In this case, the clustering unit 106 converts the 5W1H label information of each VoC data into a numerical vector and applies algorithms such as k-means or hierarchical clustering. In vectorization, customer attributes in the Who field, product categories in the What field, and time periods in the When field are each quantified, and clusters are formed based on the distance in a multidimensional vector space. This method enables faster and more stable clustering for large amounts of VoC data.

[0077] The information processing system described herein may be implemented as a single device, or it may be implemented by multiple devices, some or all of which are connected by a network. For example, the functions of the processor 10 and storage 12 of the management server 1 may be implemented by different servers connected to each other by a network.

[0078] Furthermore, the series of processes performed by the information processing system described herein may be implemented using software, hardware, or a combination of software and hardware. It is also possible to create a computer program to implement each function of the management server 1 according to this embodiment and implement it on a PC or the like. A computer-readable recording medium on which such a computer program is stored can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, flash memory, etc. In addition, the above-mentioned computer program may be distributed, for example, via a network, without using a recording medium.

[0079] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0080] The information processing system, information processing method, and program disclosed herein have, for example, the following configuration. [Item 1] A VoC data acquisition unit that acquires VoC data from customers, A labeling unit inputs the aforementioned VoC data and prompts containing instruction information specifying judgment criteria for each of the 5W1H items into a large-scale language model, and assigns labels to the VoC data according to the components of the 5W1H. An information processing system equipped with the following features. [Item 2] The labeling unit outputs "Not applicable" for any item among the 5W1H items for which it is impossible to make a guess, as described in item 1. [Item 3] The system further comprises a supplementary unit that supplements the items output as "not applicable," and the supplementary unit identifies a customer ID corresponding to the VoC that includes the "not applicable" item, as described in item 2. [Item 4] The supplementary unit is the information processing system described in item 3, which sends a questionnaire to the customer corresponding to the customer ID, including questions corresponding to the items that were marked as "not applicable". [Item 5] The aforementioned supplementary unit is the information processing system described in item 4, which grants a benefit to the customer who has responded to the aforementioned questionnaire. [Item 6] The information processing system described in item 1, further comprising a clustering unit that clusters VoC data based on 5W1H labels assigned by the labeling unit. [Item 7] The information processing system described in item 6, wherein the clustering unit inputs the VoC data to which the 5W1H labels have been assigned and a prompt for clustering into a large-scale language model, and causes the large-scale language model to perform clustering. [Item 8] Steps to obtain VoC data from customers, The steps include inputting the VoC data and prompts containing instruction information specifying judgment criteria for each of the 5W1H items into a large-scale language model, and assigning labels to the VoC data according to the components of the 5W1H; Information processing methods including [Item 9] The computer is used to acquire Voice of the Customer (VoC) data from customers. The VoC data and prompts containing instruction information specifying judgment criteria for each of the 5W1H items are input into a large-scale language model, and labels based on the 5W1H components are assigned to the VoC data. program. [Explanation of Symbols]

[0081] 1. Management Server 2 LLM Servers 3. Enterprise terminals 4 Customer terminals 101 VoC Data Acquisition Unit 104 Labeling Unit

Claims

1. A VoC data acquisition unit that acquires VoC data from customers regarding products provided by a user company, A labeling unit inputs the aforementioned VoC data and prompts that instruct a labeling task for the VoC data, including instruction information specifying judgment criteria for each of the 5W1H items, into a large-scale language model, and assigns labels for each of the 5W1H items to the VoC data. A clustering unit that clusters the VoC data based on the labels of each of the 5W1H items assigned to each of the VoC data, An information processing system equipped with the following features.

2. The information processing system according to claim 1, wherein the labeling unit outputs "Not applicable" for any item among the 5W1H items for which it is impossible to make a guess.

3. The system further includes a supplementary unit that completes the items that were output as "Not applicable". The information processing system according to claim 2, wherein the complement unit identifies a customer ID corresponding to the VoC that includes the "not applicable" item.

4. The information processing system according to claim 3, wherein the supplementary unit sends a questionnaire to the customer corresponding to the customer ID, including questions corresponding to the items that were marked as "not applicable".

5. The information processing system according to claim 4, wherein the supplementary unit grants a benefit to the customer who has responded to the questionnaire.

6. The information processing system according to claim 1, wherein the labeling unit assigns the descriptions in the VoC data corresponding to each of the 5W1H items to the VoC data as labels for each of the 5W1H items.

7. The information processing system according to claim 1, wherein the clustering unit inputs the VoC data to which the 5W1H labels have been assigned and a prompt for clustering to a large-scale language model, and causes the large-scale language model to perform clustering.

8. A step of obtaining VoC data from customers regarding a product provided by a user company, The steps include inputting the VoC data and prompts instructing a labeling task for the VoC data, which include instruction information specifying the judgment criteria for each of the 5W1H items, into a large-scale language model, and assigning labels for each of the 5W1H items to the VoC data, The steps include: clustering the VoC data based on the labels for each of the 5W1H items assigned to each of the VoC data; A method of information processing performed by a computer.

9. On the computer, We will obtain customer Voice of the Customer (Voice of the Customer) data for the products provided by the user company. The VoC data and prompts instructing a labeling task for the VoC data, including instruction information specifying the judgment criteria for each of the 5W1H items, are input to a large-scale language model, and the VoC data is then labeled with each of the 5W1H items. The VoC data is clustered based on the labels for each of the 5W1H items assigned to each of the VoC data. program.

Images