Information processing systems, information processing methods, and programs

Abstract

To enable appropriate labeling of customer feedback data. [Solution] The present invention relates to an information processing system comprising a VoC acquisition unit that acquires VoC data from customers, a prompt that includes label definitions and output rules, and a label assignment unit that inputs the VoC data into a large-scale language model and assigns labels to the VoC data.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In Patent Document 1, a technique for automatically assigning tags to natural sentences such as product descriptions has been proposed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, since a dictionary is referred to extract phrases such as proper expressions and semantic role words included in a sentence, an attribute tag may be erroneously assigned to a phrase that does not indicate the characteristics of the presentation target depending on the content of the presentation target description, and as a result, the quality of the data may be degraded.

[0005] An object of an exemplary embodiment of the present invention is to provide a technique capable of appropriately labeling customer voice data.

Means for Solving the Problems

[0006] An information processing system according to one aspect of the present invention includes a VoC acquisition unit that acquires VoC data from a customer, a label assignment unit that inputs a prompt including label definitions and output rules and the VoC data into a large language model to assign labels to the VoC data.

[0007] According to the present invention, customer voice data (VoC: Voice of Customer) can be appropriately labeled. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram showing an example configuration of an information processing system according to one embodiment of the present invention. [Figure 2] Figure 2 is a block diagram illustrating the hardware configuration of the management server shown in Figure 1. [Figure 3] Figure 3 is a block diagram illustrating the functions of the management server shown in Figure 1. [Figure 4] Figure 4 is a flowchart showing an example of information processing performed by an information processing system. [Modes for carrying out the invention]

[0009] An information processing system according to one embodiment of this disclosure will be described with reference to the drawings. In the attached drawings, identical or similar elements are denoted by identical or similar reference numerals and names, and redundant descriptions of identical or similar elements will be omitted as appropriate in the description of the embodiment. The contents shown in each drawing are illustrative examples for explaining this embodiment and are only shown in a schematic manner to facilitate understanding. They may be modified or changed to the extent that it does not impede technical limitations.

[0010] <System Overview> The information processing system according to this embodiment (hereinafter referred to as "this system") is a system that performs processing related to the analysis of customer voice data (hereinafter referred to as "VoC data") for products (goods and / or services) provided by a user company.

[0011] In this embodiment, "VoC (Voice of Customer) data" refers to all information relating to opinions, evaluations, experiences, questions, requests, and other expressive acts made by customers voluntarily or at their request, regardless of the medium, format, or acquisition route. The data format of VoC data may include data such as text, audio, images, and video.

[0012] This system assigns predetermined labels, such as semantic labels, service labels, and sentiment labels, to VoC data, preparing the data for insight analysis using VoC data. In this embodiment, "label" refers to metadata assigned to describe the context and meaning of VoC data.

[0013] As shown in Figure 1, the information processing system of this embodiment may include, for example, a management server 1, an LLM server 2, one or more corporate terminals 3, and one or more customer terminals 4. Each of these devices is connected to each other so as to be able to communicate via a network NW. In this embodiment, the network NW is mainly assumed to be the internet, but it may also be constructed using a public telephone network, a mobile phone network, a wireless communication network, Ethernet (registered trademark), etc. Furthermore, if necessary, external systems such as social networking services, billing systems, and membership management systems may be connected to the management server 1 and other components so as to be able to communicate via the network NW.

[0014] Management Server 1 is an information processing device managed by the service provider that provides services related to this system. Management Server 1 may be built on-premises using a general-purpose computer such as a workstation or personal computer, or it may be logically configured using cloud computing. Some of the functions of Management Server 1, described later, may be implemented as functions realized by the processor of other devices such as enterprise terminals 3, or they may be implemented in a distributed manner on external servers.

[0015] LLM Server 2 is a server that operates a Large Language Model (LLM) to provide services such as text generation and question answering. In this system, the LLM receives prompts and VoC data sent from Management Server 1 as input and performs labeling processing for each VoC data based on this. The generated labels are sent back to Management Server 1, which stores these labels in association with the corresponding VoC data. LLM Server 2 may be configured as a physical server installed in an on-premises environment or as a virtual server built on a cloud computing service.

[0016] Enterprise terminal 3 is an information processing terminal of a user organization that uses the services of this system. Enterprise terminal 3 is used to set prompts and output labeling results and analysis results. Enterprise terminal 3 is intended to be used by the system administrator or marketing personnel of the user organization, and may be a general-purpose computer such as a workstation or personal computer, or a mobile terminal such as a smartphone or tablet. In the description of this embodiment, when "user" is used, the "user" refers to the person in charge of the user organization who operates enterprise terminal 3.

[0017] Customer terminal 4 is an information processing terminal used by customers (general consumers) who purchase and / or use products provided by the user organization. Customer terminal 4 is used to input VoC data in predetermined data formats, such as free-response and multiple-choice formats, via media such as survey forms, membership apps, community tools, and social networking services (SNS). Customer terminal 4 can be any device, such as a personal computer, smartphone, or tablet, and is connected to management server 1 via a network (NW).

[0018] <Hardware Configuration> As shown in FIG. 2, the management server 1 includes a processor 10, a memory 11, a storage 12, a transceiver 13, an input / output unit 14, etc., which are electrically connected to each other through a bus 17. Note that the configuration of the illustrated management server 1 is an example, and the management server 1 may have other configurations.

[0019] The processor 10 is an arithmetic unit that controls the overall operation of the management server 1, controls the transmission and reception of data between each element, and performs information processing necessary for the execution and authentication processing of applications. For example, the processor 10 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and executes a program for this system stored in the storage 12 and expanded in the memory 11 to perform each information processing described later.

[0020] The memory 11 includes a main memory composed of a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory composed of a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive). The memory 11 is used as a work area of the processor 10, and stores a BIOS (Basic Input / Output System) executed when the management server 1 is started, and various setting information.

[0021] The storage 12 stores various programs such as application programs for executing the information processing shown in this embodiment. A database storing data used for each information processing may be constructed in the storage 12. For example, a storage unit 120 described later may be provided in a part of the storage area of the memory 11 and / or the storage 12.

[0022] The transceiver 13 is a communication interface for connecting the management server 1 to a network. Note that the transceiver 13 may include a short-range communication interface for Bluetooth (registered trademark) and BLE (Bluetooth Low Energy).

[0023] The input / output unit 14 includes input devices such as keyboards and mice, and output devices such as speakers and displays. In the management server 1, the input devices and output devices may be mounted on the management server 1 as separate devices, or a touch panel display that combines the functions of both input and output devices may be mounted on the management server 1.

[0024] Bus 17 is connected in common to all of the above elements and transmits, for example, address signals, data signals, and various control signals.

[0025] Furthermore, the hardware configuration of other devices such as LLM Server 2, Enterprise Terminal 3, and Customer Terminal 4 is the same as that of Management Server 1. LLM Server 2, Enterprise Terminal 3, and Customer Terminal 4 are all equipped with processors, memory, storage, transceivers, input / output units, etc., and these are electrically connected to each other via a bus. These components are the same as those of Management Server 1's processor 10, memory 11, storage 12, transceiver 13, input / output unit 14, and bus 17, and therefore their explanation is omitted.

[0026] <Functions (Software Configuration) of Information Processing Systems> Figure 3 is a block diagram illustrating the functions implemented in this system. The management server 1 may include a VoC acquisition unit 101, a preprocessing unit 102, a prompt generation unit 103, a label assignment unit 104, and an output unit 105. Each functional unit of the management server 1 is functionally realized by the processor 10 reading programs and data stored in the storage 12 and executing various programs in the working area of ​​the memory 11. The storage unit 120 of the management server 1 may also include a VoC data storage unit 121, a prompt storage unit 122, and a label result storage unit 123. Each of these storage units stores data necessary for various information processing in this system and is configured to be accessible from each functional unit executed by the processor 10.

[0027] The VoC data storage unit 121 stores the acquired VoC data and manages each data item by assigning unique identification information to it. The VoC data storage unit 121 can also store VoC data imported in CSV format. The VoC data storage unit 121 may also store the original data at the time of VoC data collection and the normalized data after preprocessing in association with each other. Each VoC data item may be associated with information to identify the originating customer, and such information may include the customer account identifier in the acquisition destination system, the type of the identifier, and the type of media from which the data was acquired. Furthermore, metadata such as the collection time, acquisition destination name, language, character code, and flags related to data quality (presence or absence of missing data, results of character encoding detection, etc.) may be assigned and managed. From the perspective of privacy protection, the system may be configured to mask or pseudonymize attributes that have a high degree of personal identification according to prescribed regulations.

[0028] The prompt memory unit 122 stores prompts to be input to the large-scale language model. The prompt memory unit 122 may also store prompts for labeling (assigning labels), which include label definitions and output rules. The prompt memory unit 122 may also store prompts for generating a label classification system, which include instructional information that instructs the task of generating label classification items, and a description of the business type under analysis (a description of the user organization's business content, and / or a product description, etc.).

[0029] The label result storage unit 123 stores the label results assigned by the label assignment unit 104. The label information, which is the label result, may have related metadata (confidence level, assignment date and time, creator, etc.) attached to it, and the label result storage unit 123 stores the label information in association with the VoC data. In addition to the above database, the storage unit 120 may have an enterprise information storage unit that stores information about user organizations.

[0030] The VoC acquisition unit 101 acquires VoC data from customers. The acquisition route for VoC data is not particularly limited. For example, the VoC acquisition unit 101 may acquire VoC data from external systems such as community tools (member apps, member sites, etc.), e-commerce sites, survey systems, and social networking services by using technologies such as API integration.

[0031] For example, when acquiring VoC data from community tools such as membership sites, VoC data such as product reviews and testimonials posted on the tool during a specified period may be acquired via API integration. Alternatively, when collecting VoC data from social media, the user organization may use the names of the products it offers as keywords and extract posts containing those keywords, or it may acquire posts submitted to the user organization's official account. Furthermore, the results of surveys conducted on external systems may be acquired from those systems. The method of collecting VoC data through surveys is not limited to using communication tools, survey systems, or social media; other methods may also be employed.

[0032] In addition to acquiring VoC data from external systems as described above, the VoC acquisition unit 101 may also acquire VoC data collected by the user organization from the corporate terminal 3. In this case, the VoC acquisition unit 101 may acquire VoC data by presenting a graphical user interface (hereinafter referred to as GUI) for data import to the corporate terminal 3 and accepting input of the VoC data collected by the user organization via the GUI.

[0033] The VoC acquisition unit 101 may acquire VoC data from multiple sources. Furthermore, when acquiring VoC data, it may acquire supplementary information such as information indicating the source medium, the date and time of posting the VoC data, and information identifying the customer who posted the VoC data (e.g., customer identification information or account information on community tools, e-commerce sites, social media, etc.) and link this information to the VoC data. The acquired VoC data is recorded in the VoC data storage unit 121 described above.

[0034] The preprocessing unit 102 performs rule-based preprocessing on the collected VoC data. For example, as a preliminary step before performing the labeling process, the preprocessing unit 102 may perform cleansing processing on the VoC data by noise removal, normalization, or a combination thereof. Noise removal may be a process of deleting or neutralizing non-linguistic elements that interfere with the sentence understanding of the LLM, such as emojis, URLs, hashtags, line break tags, special symbols, and device-dependent characters included in SNS and community posts. It may also include a process of removing unnecessary blanks, duplicate sentences, stereotypical greeting sentences, etc.

[0035] The normalization process may include, for example, a process of unifying different notations having the same meaning. For example, normalization of notation fluctuations such as unifying "出来る", "できる", "出来ます", "できます" to "できる", or normalization of converting synonyms such as "美味しい", "美味", "うまい" to representative terms may be performed. It may also include a process of unifying alphabetic notations, normalizing full-width and half-width forms, and converting symbols such as emojis representing emotional expressions into words and phrases representing their meanings (for example, emotional words such as "嬉しい", "悲しい", "驚いた", etc.). As a result, the vocabulary system of the input sentence is standardized, and the accuracy of semantic analysis by the LLM is improved.

[0036] Furthermore, the normalization process performed by the preprocessing unit 102 may include a process to convert the text of the VoC data into a predetermined data structure. For example, the preprocessing unit 102 performs a preprocessing step to convert the VoC data into a predetermined data structure format without summarizing it. The predetermined data structure may have a description format that includes a set of keys and values ​​for dividing the text data. In this case, the keys may be specified by the user or defined by default. In this normalization process, the VoC data is divided into sentence units or paragraph units, or into word units, and converted into a predetermined data structure. For example, it may be converted into structured data such as JSON format with key-value pairs. In this case, "customer ID," "speech content," "speech date and time," "media type," "language," "speech number," etc., may be used as keys, and the corresponding values ​​may be stored.

[0037] Furthermore, the preprocessing unit 102 may include validation processing to check data quality. For example, if the input data contains sentences with an extremely small number of characters or sentences that are difficult to identify as languages, such data may be excluded or output as a separate category. The preprocessing results are stored in the VoC data storage unit 121 in the storage unit 120 of the management server 1 and are kept in a state that can be referenced by subsequent functional units such as the labeling unit 104.

[0038] The prompt generation unit 103 is a function that generates prompts to be input to the Large-Scale Language Model (LLM) in the label assignment process. In this embodiment, "prompt" refers to instruction information given to the LLM. Prompts for label assignment are generated with a configuration that includes, for example, label definition information and output rule information. In addition, a description of the business type to be analyzed may be included. The prompt generation unit 103 may also acquire setting information input via the corporate terminal 3 and construct prompts suitable for the analysis case by applying said setting information to a template.

[0039] Specifically, the prompt may include basic instruction information to guide the task, definitions of classification items to be used as labels, criteria for assigning labels to each VOC data, and detailed instruction information that specifies output rules such as output format. It may also include other information to be analyzed, such as a description of the user organization's business or product (goods and / or services), and input fields for attaching VoC data. The output rules may specify how labels are written and their structured format (e.g., key-value pair format or JSON format).

[0040] The prompt generation unit 103 may acquire a description of the user organization's business and / or product (prerequisite information) and analysis setting information (target, label type, output format, etc.) from the corporate terminal 3, and dynamically construct case-specific prompts by applying these settings to a templated form. For example, the prompt generation unit 103 may present a graphical user interface for prompt generation to the corporate terminal 3. This GUI may include input fields that allow the user to specify, for example, the analysis target (product name, service name, category, etc.), the type of label (semantic label, service label, sentiment label), and the output format (JSON, table format, etc.). The prompt generation unit 103 may dynamically generate case-specific prompts by applying this input information to a templated prompt form and replacing variable parts in the template (e.g., "<target service name>", "<output format>", etc.). The generated prompts are stored in the prompt storage unit 122.

[0041] Labels are not necessarily limited, and examples of labels that can be used to interpret each VoC data include semantic labels, service labels, and sentiment labels. Each VoC data may be assigned one of the following label types: semantic label, service label, or sentiment label, or two or more types of labels may be assigned.

[0042] Semantic labels are used to classify the content of statements and descriptions included in VoC data based on the purpose and intent of the statement. Semantic labels consist of seven types, for example: usage, motivation for use, requests, complaints, recommendations, positive feedback, and others. Here, "usage" describes how customers actually use the service, "motivation for use" shows the reasons and background for deciding to use the service, "requests" are statements that request service improvements or the addition of new features, and "complaints" are statements that point out dissatisfaction or inconvenience. "Recommendations" are statements that recommend the service to a third party or give a positive evaluation, and "positive feedback" is content in which customers express emotional satisfaction. "Other" includes statements that do not belong to any of the above categories, including general impressions.

[0043] The prompt generation unit 103 may include each label name and its description within the prompt it generates in order to help the LLM understand the definition of the semantic labels. For example, the definitions may be explicitly stated in natural language, such as "Usage: Content that specifically describes how the service was used" or "Complaint: Content that expresses dissatisfaction or inconvenience."

[0044] A service label is a label used to identify the product name, service name, or function mentioned by a customer in VoC data. For example, if a user organization offers multiple services, a service label may be assigned to identify which service the comment pertains to. Alternatively, service labels may be defined for multiple functional elements of a product provided by the user organization (e.g., reservation system, customer service, service speed, price, product range, store cleanliness, payment, etc.), and the VoC data may be assigned to identify which function of the user organization's product the comment pertains to. The prompt generation unit 103 may refer to the product master or service master obtained from the corporate terminal 3 and insert a "service name list" in a format that the LLM can refer to in the label definition unit to set the classification items for the service label. Alternatively, the prompt generation unit 103 may accept input from the user, such as the user organization's business content and detailed product description, and output a draft of service label classification items corresponding to the product to the corporate terminal 3, or it may accept input of service label classification items created by the user.

[0045] Emotional labels are labels that indicate the emotional expressions of customers included in VoC data. The classification items for emotional labels may be, for example, a three-level classification of positive, negative, and neutral, or a configuration that outputs multiple values ​​by pre-defining specific emotional categories such as joy, anticipation, surprise, anger, disgust, fear, disappointment, relief, and confusion.

[0046] The management server 1 may have a function to generate a label classification system (label classification items and their definitions) using an LLM. For example, the label assignment unit 104, described later, may perform a process to generate a label classification system before performing the labeling process. In this case, the prompt generation unit 103 may accept input of a description of the business type to be analyzed, such as a description of the user organization's business content and a description of the products provided by the user organization, and generate a prompt that includes the input content and default instruction information indicating an instruction to generate a label classification system. The label assignment unit 104 generates label classification items and their definitions by inputting the prompt containing the description of the business type to be analyzed and the VoC data attached to the prompt into a Large-Scale Language Model (LLM). The prompt generation unit 103 may generate a prompt for labeling (label assignment) based on the generated label classification items and their definitions.

[0047] In constructing a label classification system, groups of concepts or themes included in the VoC data may be extracted, and the classification items and definitions of the labels themselves may be generated based on the extracted groups of concepts or themes. For example, if features such as "easy to use," "simple to operate," and "good UI" are detected from the VoC data, these can be integrated and defined as the "usability" category. Frequently occurring expressions and contextual features in the VoC data can be automatically classified, and candidate lists of semantic labels, service labels, and sentiment labels can be generated from them.

[0048] This label construction process may be performed based on instructions from the user organization's corporate terminal 3, and the management server 1 may be configured to accept the selection of target data sets and business type information, and the specification of the generation granularity (for example, "higher-level categories only" or "generate up to detailed categories"). The generated classification items and definitions are stored in the prompt storage unit 122 and may be reflected in the prompt as final label definition information after being confirmed and modified by the user.

[0049] The labeling unit 104 inputs prompts and VoC data, including label definitions and output rules, into a large-scale language model (LLM), and has the LLM perform labeling processing on the VoC data. The labeling unit 104 may also control communication with the LLM and have functions such as formatting the input data structure, analyzing output results, and controlling re-execution. In the labeling process, VoC data may be individually input into the LLM and labeling of each VoC data may be performed individually, or multiple VoC data may be attached to a prompt and labels may be applied to these multiple VoC data in a batch process.

[0050] The labeling unit 104 sends a request to the LLM that includes a prompt and at least one VoC data item. Based on this request, the LLM generates a label corresponding to each VoC data item according to the definition statement and output rules described in the prompt. The labeling result may be output as structured data that includes the label name and its value in key-value format. The labeling unit 104 may receive this output data and perform processing to organize it by label type (semantic label, service label, sentiment label).

[0051] The labeling unit 104 assigns semantic labels to the content of the VoC data, classifying it into at least one of the following categories: actual usage, motivation for use, requests, complaints, recommendations, impressions, and others. For example, the semantic label "request" is assigned to the VoC data "I don't know how to use the product," and the semantic label "complaint" is assigned to the VoC data "The product broke."

[0052] The labeling unit 104 assigns a service label corresponding to the product or service covered by the VoC data. For example, for VoC data such as "smartphone battery lasts longer," the service label "smartphone" is assigned. The labeling unit 104 can automatically determine and assign service labels to identify multiple products and services offered by a company.

[0053] The labeling unit 104 assigns emotion labels to the VoC data to indicate the emotions contained within it. Emotion labels can be classified into three levels, for example, positive, negative, and neutral. The labeling unit 104 can also assign more detailed emotion categories, such as joy, anger, sadness, and surprise.

[0054] The labeling unit 104 may also assign multiple types of labels simultaneously within a single prompt. For example, the LLM may be provided with definitions for multiple types of labels in a single prompt, and controlled to simultaneously output "Request" as a semantic label, "Reservation Function" as a service label, and "Negative" as an emotional label for a single VoC data. By generating multiple labels in an integrated manner in this way, the subsequent analysis unit can analyze customer intent while maintaining the relationships between the labels.

[0055] Furthermore, the labeling unit 104 may also have a function to analyze the LLM output results and calculate the confidence level. For example, it can perform a process to evaluate the confidence level lower for outputs that have a low agreement rate with the definition sentence in the prompt, or outputs that contain many ambiguous expressions ("unknown," "not applicable," etc.). If the confidence level falls below a predetermined threshold, the data may be automatically resent and the labeling process executed again. This can improve the accuracy and reproducibility of labeling.

[0056] Furthermore, the labeling unit 104 may be configured to perform syntax verification and generate re-parsing or error feedback if it detects output that does not conform to the output format specified in the prompt (e.g., JSON structure).

[0057] The label assignment unit 104 may associate the LLM output results (label information) with each VoC data and register them in the VoC data storage unit 121. The label information may include metadata such as "customer ID," "input text," "assigned label," "confidence level," and "label assignment date and time." The label assignment unit 104 may also perform a process to dynamically update the label definition information. For example, if LLM outputs a new concept that does not correspond to an existing label during the label assignment process, the output can be analyzed and frequently occurring expressions can be registered as new label candidates.

[0058] The output unit 105 outputs the label assignment results. The output unit 105 retrieves the label assignment results from the label result storage unit 123 and sends them to the enterprise terminal 3. The output unit 105 can visualize and output the label assignment results in tabular or graphical format. For example, it may display the frequency of occurrence of each label in a graph, or show changes in customer sentiment in a time-series chart based on the aggregated results of sentiment labels. The output results are displayed on the dashboard screen of the enterprise terminal 3, and can also be exported as a data file in CSV or JSON format.

[0059] The management server 1 may have an analysis unit as a functional unit executed by the processor 10. The analysis unit performs predetermined analyses using the label information assigned by the label assignment unit 104 and the VoC data. For example, it may perform analyses to extract customer behavioral trends, emotional trends, or areas of interest. The analysis unit may calculate statistical indicators such as the frequency of occurrence, co-occurrence relationships, and time-series changes for each label, and perform processes to derive customer insights. It may also perform analyses to model customer behavioral patterns based on combinations of semantic labels, service labels, and emotional labels.

[0060] The analysis department may perform subset analysis limited to specific periods or product categories. For example, by extracting only VoC data corresponding to a specific service label and aggregating the trends in sentiment labels, customer satisfaction on a service-by-service basis can be calculated. Furthermore, common keywords can be extracted from VoC data sets containing usage motivation labels and used to inform marketing strategies and product development.

[0061] Furthermore, the analysis unit may have functions to analyze the similarity and cluster structure between labels. For example, it may be configured to integrate similar semantic labels to form higher-level categories, or to generate a customer emotional response map using the co-occurrence frequency of emotional labels and service labels.

[0062] Figure 4 is a flowchart showing an example of the information processing flow in this system. The VoC acquisition unit 101 of the management server 1 acquires customer VoC data from community tools, surveys, or SNS (step S1), and stores the acquired VoC data in the VoC data storage unit 121 (step S2). The preprocessing unit 102 acquires VoC data from the VoC data storage unit 121 and performs cleansing processing such as noise reduction and normalization (step S3). In this cleansing processing, for example, the preprocessing unit 102 may perform preprocessing to convert the VoC data into a predetermined data structure format without summarizing it. The labeling unit 104 acquires prompts including label definitions and output rules from the prompt storage unit 122, and inputs the preprocessed VoC data and prompts into a large-scale language model to assign labels (step S4). The label assignment unit 104 stores the assigned label information in the label result storage unit 123, linking it to the VoC data (step S5), and the output unit 105 transmits the label assignment result to the corporate terminal 3 (step S6).

[0063] Note that the information processing method flow shown in Figure 4 is merely an example, and steps may be added, deleted, modified, or rearranged as appropriate.

[0064] According to the information processing system of this embodiment, high-precision labeling can be performed on VoC data by utilizing a large-scale language model. This enables appropriate labeling according to context, which was difficult with conventional dictionary-based methods, and improves the accuracy of VoC data analysis. Furthermore, multifaceted labeling, such as semantic labels, service labels, and sentiment labels, allows for multi-faceted analysis of customer voices. In addition, by using prompts that include descriptions of business types, it becomes possible to assign labels that take into account industry-specific contexts, resulting in more practical analysis results. Structuring by the preprocessing unit improves data quality without losing information from the original text, and contributes to improving the accuracy of subsequent analysis processes.

[0065] Although these embodiments have been described above, they are intended to facilitate understanding of the present invention and are not intended to limit its interpretation. The present invention can be modified and improved without departing from its spirit, and equivalents thereof are also included.

[0066] For example, the processing performed by each functional unit of the management server 1 described above may be executed by any of the functional units. Furthermore, different functional units may be added to perform some of the processing performed by each of the functional units described above. Also, the functional units of the management server 1 may be distributed across multiple computers.

[0067] Furthermore, the information stored in each memory unit of the management server 1 may be stored in any of the memory units. That is, the information stored in the multiple memory units mentioned above may be stored in a single memory unit, or a portion of the information stored in one memory unit may be stored in another memory unit.

[0068] <Example 1> In the embodiment described above, an example was shown in which prompts and VoC data are input to a large-scale language model at once to perform labeling. However, labeling may also be performed in stages. Specifically, the labeling unit 104 may perform semantic labeling in the first stage, service labeling in the second stage by referring to the results of the first stage, and sentiment labeling in the third stage by referring to the results of the previous stage. In this case, the labeling unit 104 can achieve more accurate labeling by inputting different prompts to the large-scale language model at each stage and including the results of the previous stage in the prompt of the next stage.

[0069] <Modification 2> In the embodiments described above, an example was shown in which labeling is performed using a single large-scale language model, but multiple large-scale language models may be used in combination. Specifically, the labeling unit 104 may use a general-purpose large-scale language model for assigning semantic labels, a large-scale language model specialized for sentiment analysis for assigning sentiment labels, and a large-scale language model fine-tuned for the relevant business domain for assigning service labels. In this case, the labeling unit 104 can achieve the highest accuracy for each label type by selecting the optimal large-scale language model according to each label type and performing labeling using the corresponding prompt.

[0070] <Variation 3> In the embodiment described above, an example was shown in which VoC data is processed individually and labeled. However, multiple VoC data can also be labeled together in a batch process. Specifically, the labeling unit 104 may group a predetermined number of VoC data and input them into the large-scale language model along with prompts in a group-by-group manner to perform labeling. In this case, the labeling unit 104 can improve processing efficiency by preparing a prompt template for batch processing, arranging multiple VoC data in a structured format and incorporating it into the prompt, and obtaining multiple label results from the large-scale language model all at once.

[0071] <Modification 4> In the embodiments described above, an example of label assignment using a fixed label classification was shown, but label assignment may also be performed by dynamically generating label classifications. Specifically, the label assignment unit 104 may input the VoC data set to be analyzed into a large-scale language model in advance to generate an optimal label classification system for the data set, and then use the generated label classification system to assign labels to individual VoC data. In this case, the label assignment unit 104 can achieve label assignment optimized for data characteristics by inputting a prompt for label classification generation and a sample of VoC data into the large-scale language model in the first stage to obtain a custom label classification, and then using a prompt including the generated label classification in the second stage to assign labels to each VoC data.

[0072] <Modification 5> The above-described embodiment shows an example of processing text-formatted VoC data, but it is also possible to process multimodal VoC data including audio data and image data. Specifically, the VoC acquisition unit 101 may acquire audio reviews and reviews with images, the preprocessing unit 102 may perform speech recognition processing to convert audio data into text and image recognition processing to generate descriptive text from image data, and the labeling unit 104 may input a prompt combining the converted text data and the original multimodal information into a large-scale language model to perform labeling. In this case, the preprocessing unit 102 can also transcribe emotional features of the audio and visual information of the images into text and include them in the prompt, thereby achieving labeling based on richer information.

[0073] <Variation 6> In this embodiment, the labeling unit 104 is configured to perform labeling using LLM, but a feedback mechanism for verifying label quality may be added. For example, a user on the enterprise terminal 3 may check the label content, send the result of correcting misjudgments to the management server 1, and have the corrected result reflected back into the LLM as training data. By configuring it in this way, autonomous learning operation that continuously improves labeling accuracy becomes possible.

[0074] <Example 7> In this embodiment, the labeling process is configured to be executed at any time in response to user instructions. However, a real-time processing configuration that sequentially assigns labels as soon as VoC data is generated is also possible. For example, SNS posts and chat logs may be monitored as streams, and the labeling unit 104 may start processing whenever new data is detected, and the results may be immediately reflected on the dashboard. This makes it possible to instantly visualize rapid changes and trends in customer sentiment, supporting quick business decisions.

[0075] The information processing system described herein may be implemented as a single device, or it may be implemented by multiple devices, some or all of which are connected by a network. For example, the functions of the processor 10 and storage 12 of the management server 1 may be implemented by different servers connected to each other by a network.

[0076] Furthermore, the series of processes performed by the information processing system described herein may be implemented using software, hardware, or a combination of software and hardware. It is also possible to create a computer program to implement each function of the management server 1 according to this embodiment and implement it on a PC or the like. A computer-readable recording medium on which such a computer program is stored can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, flash memory, etc. In addition, the above-mentioned computer program may be distributed, for example, via a network, without using a recording medium.

[0077] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0078] The information processing system, information processing method, and program disclosed herein have, for example, the following configuration. [Item 1] A VoC acquisition unit that acquires VoC data from customers, A labeling unit that inputs a prompt including label definitions and output rules, and the VoC data into a large-scale language model and assigns labels to the VoC data, An information processing system equipped with the following features. [Item 2] The labeling unit assigns semantic labels to the contents of the VoC data, classifying them into at least one of the following categories: actual usage, motivation for use, requests, complaints, recommendations, impressions, and others, as described in item 1. [Item 3] The labeling unit is an information processing system according to item 1 that assigns a service label corresponding to the goods or services covered by the VoC data. [Item 4] The labeling unit is an information processing system according to item 1 that assigns emotion labels indicating emotions included in the VoC data. [Item 5] The information processing system described in item 1, wherein the labeling unit generates labels for the VoC data by inputting the prompt, which includes a description of the business type to be analyzed, and the VoC data attached to the prompt, into the large-scale language model. [Item 6] The VoC acquisition unit is an information processing system as described in item 1, which collects the VoC data from community tools, questionnaires, or social networking services. [Item 7] The system further includes a preprocessing unit that performs rule-based preprocessing on the collected VoC data. The preprocessing unit performs noise reduction, normalization, or a combination thereof, as described in item 1, which is the information processing system. [Item 8] The information processing system described in item 7, wherein the preprocessing unit performs preprocessing to convert the VoC data into a predetermined data structure format without summarizing it. [Item 9] The information processing system described in item 8, wherein the predetermined data structure has a descriptive format that includes a set of keys and values ​​for dividing text data, the keys being specified by the user or defined by default. [Item 10] Steps to obtain VoC data from customers, A step of inputting a prompt including label definitions and output rules, and the VoC data into a large-scale language model and assigning labels to the VoC data, Information processing methods including [Item 11] The computer is used to acquire VoC data from customers. The system inputs a prompt containing label definitions and output rules, along with the VoC data, into a large-scale language model, and assigns labels to the VoC data. program. [Explanation of symbols]

[0079] 1. Management Server 2 LLM Servers 3. Enterprise terminals 4 Customer terminals 101 VoC acquisition department 104 Labeling Unit

Claims

1. A VoC acquisition unit that acquires VoC data from customers, A prompt that instructs the task of labeling the VoC data, including the definition of the label, output rules, and a description of the business type of the user organization to be analyzed; a labeling unit that inputs the VoC data into a large-scale language model and assigns labels to the VoC data; An information processing system equipped with the following features.

2. The information processing system according to claim 1, wherein the labeling unit assigns semantic labels to the contents of the VoC data, classifying them into at least one of the following categories: actual usage, motivation for use, requests, complaints, recommendations, impressions, and others.

3. The information processing system according to claim 1, wherein the labeling unit assigns a service label corresponding to the goods or services targeted by the VoC data.

4. The information processing system according to claim 1, wherein the labeling unit assigns emotion labels indicating emotions included in the VoC data.

5. The information processing system according to claim 1, wherein the labeling unit generates the label classification items and the label definitions by inputting a prompt that instructs the generation of a label classification system including a description of the business type to be analyzed, and the VoC data attached to the prompt, into the large-scale language model before labeling the VoC data.

6. The information processing system according to claim 1, wherein the VoC acquisition unit collects the VoC data from a community tool, a survey, or a social networking service (SNS).

7. The system further includes a preprocessing unit that performs rule-based preprocessing on the collected VoC data. The information processing system according to claim 1, wherein the preprocessing unit performs noise reduction, normalization, or a cleansing process by a combination thereof.

8. The information processing system according to claim 7, wherein the preprocessing unit performs preprocessing to convert the VoC data into a predetermined data structure format without summarizing it.

9. The information processing system according to claim 8, wherein the predetermined data structure has a description format including a set of keys and values ​​for dividing text data, the keys being specified by the user or defined by default.

10. The labeling unit inputs the prompt and the VoC data into the large-scale language model and assigns two or more labels from among semantic labels, service labels, and sentiment labels to the VoC data. The aforementioned semantic labels are labels that classify the content of the VoC data into at least one of the following categories: actual usage, motivation for use, requests, complaints, recommendations, impressions, and others. The aforementioned service label is a label corresponding to the product or service covered by the VoC data, The information processing system according to claim 1, wherein the emotion label is a label indicating an emotion included in the VoC data.

11. Steps to obtain VoC data from customers, A prompt instructing the task of labeling the VoC data, including a label definition, output rules, and a description of the business type of the user organization to be analyzed; a step of inputting the VoC data into a large-scale language model and having the VoC data labeled; A method of information processing performed by a computer.

12. The computer is used to acquire VoC data from customers. A prompt instructing the task of labeling the VoC data, including the definition of the label, output rules, and a description of the business type of the user organization to be analyzed, and inputting the VoC data into a large language model and labeling the VoC data, program.

Images