System

Abstract

A system is provided.SOLUTION: A system, comprising: means for automatically collecting customer vocalizations from an external source; means for pre-processing the collected customer vocalizations; means for generating a AI model that summarizes the pre-processed customer vocalizations; means for automatically generating a report based on the summarized vocalizations; and means for delivering the generated report to a service representative.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Effective collection and analysis of customer feedback (VOC) is essential for service growth and improvement. However, conventional methods require enormous costs and time for VOC collection and analysis, and manual inspection and analysis entails a significant workload. Furthermore, using external tools available on the market often incurs high costs. This invention aims to resolve these issues by providing a system that efficiently collects and analyzes customer feedback at low cost and provides appropriate reports. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by the following means. A system is provided that includes a means for automatically collecting customer feedback data from external data sources, a means for preprocessing the collected customer feedback data, a generative AI model means for summarizing the preprocessed customer feedback data, a means for automatically generating reports based on the summarized data, and a means for distributing the generated reports to service personnel. This system efficiently collects customer feedback and quickly provides analysis results as reports, enabling improved service quality and faster response.

[0006] "External Data Sources" refers to sources that provide data, including Voice of the Customer (VOC), accessible via the Internet or other networks.

[0007] "Voice of Customer Data" refers to text data containing customer opinions and impressions, such as customer feedback, reviews, comments, and ratings.

[0008] "Automatic collection" refers to the process of acquiring data based on pre-set schedules or conditions without human intervention.

[0009] "Preprocessing" refers to the process of removing unnecessary information from collected data and preparing it in an appropriate format for analysis and summarization.

[0010] "Generative AI model" refers to a generative artificial intelligence used to analyze customer voice data and extract and summarize meaningful information.

[0011] Summarization refers to the process of extracting the main points and important information from a wide range of data and summarizing them concisely.

[0012] "Automatically generating a report" refers to automatically creating a report based on summarized information without human intervention.

[0013] "Distribute" refers to delivering the generated report to relevant parties via email or internal messaging systems.

[0014] "Service Personnel" refers to the individual or group responsible for managing or improving the service provided. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, performs preprocessing, summarizes it using a generative AI model, automatically generates a report based on the summarized data, and finally delivers the report to service personnel. This allows for the collection, analysis, and reporting of customer feedback at low cost and efficiently.

[0037] System Overview

[0038] This system is centered around a server and includes the following main functions:

[0039] 1. Collecting data from external sources

[0040] The server collects customer feedback data from external data sources such as app stores, social media, customer support systems, etc. This collection can be done using APIs or scraping techniques, for example, using the Twitter API to automatically retrieve posts about the product.

[0041] 2. Data Preprocessing

[0042] The server preprocesses the collected customer feedback data. Preprocessing includes text cleaning, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[0043] 3. Data Summarization

[0044] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[0045] 4. Automatic report generation

[0046] The server automatically generates reports based on the summarized data, using report templates customized to the needs of each service. For example, it could categorize summarized customer testimonials into product advantages and disadvantages and generate a PDF report with graphs and statistics.

[0047] 5. Report Distribution

[0048] The server automatically delivers the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[0049] Specific examples

[0050] A specific example of this system is shown below.

[0051] Voice of the Customer Data Collection Example

[0052] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[0053] Data Preprocessing Example

[0054] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[0055] Data Summarization Example

[0056] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[0057] Report Generation Example

[0058] The server automatically generates a detailed product report based on the summarized information, in PDF format, including charts and graphs.

[0059] Report Delivery Example

[0060] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[0061] In this way, this system efficiently collects and analyzes customer feedback and promptly provides reports, thereby supporting service improvements and rapid response.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[0065] Step 2:

[0066] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[0067] Step 3:

[0068] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[0069] Step 4:

[0070] The server cleans the text data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and inappropriate language) from the raw data.

[0071] Step 5:

[0072] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, which improves the accuracy of the analysis.

[0073] Step 6:

[0074] The server initializes and configures the generative AI model, which (e.g., GPT-4) uses specific configuration values ​​and prompts to summarize the data.

[0075] Step 7:

[0076] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates a concise summary.

[0077] Step 8:

[0078] The server automatically generates reports based on the summarized data, inserting the summarized information into pre-configured report templates to create customized reports tailored to the needs of each service.

[0079] Step 9:

[0080] The server saves the generated report and formats the document in the specified format, such as PDF or HTML.

[0081] Step 10:

[0082] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report.

[0083] Step 11:

[0084] The server adds a report summary to the body of the email and attaches the generated report document, allowing you to understand the contents of the email concisely.

[0085] Step 12:

[0086] The server sends emails to personnel and sends notifications to an internal messaging system, ensuring that all relevant personnel receive the latest report information.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] Traditional methods for collecting, analyzing, and reporting customer feedback data required a great deal of time and effort, making it difficult to efficiently collect and analyze information. Furthermore, because much of the processing was manual, it was difficult to maintain data accuracy and consistency. Traditional methods were particularly difficult to use when large amounts of data needed to be collected and analyzed in a short amount of time.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for automatically collecting customer voice data from external data sources, means for preprocessing the collected customer voice data, a generative AI model means for summarizing the preprocessed customer voice data, means for instructing the generative AI model to perform summarization using a prompt sentence, means for automatically generating a report in PDF format based on the summarized data, and means for delivering the generated report to a service representative via email or an internal message system. This makes it possible to efficiently and accurately collect and analyze customer voice data and quickly provide information to a service representative.

[0092] "External Data Sources" refers to sources of data obtained from various platforms and services on the Internet.

[0093] "Customer voice data" refers to text data such as opinions, impressions, and evaluations expressed by users about products and services.

[0094] "Preprocessing" refers to the process of removing unnecessary information from collected data and preparing it in a format suitable for analysis.

[0095] A "generative AI model" refers to an artificial intelligence program that learns from large amounts of data and performs natural language processing and text generation.

[0096] A "prompt sentence" refers to input text that instructs a generative AI model on specific actions and processing details.

[0097] "PDF format" is an abbreviation for Portable Document Format, and refers to a file format for electronically storing and displaying documents and image information.

[0098] "Mail" is an abbreviation for electronic mail and refers to a means of communication for exchanging text and files over the Internet.

[0099] An "internal messaging system" refers to a dedicated communication system for exchanging information and messages within an organization.

[0100] "Service representative" refers to a person who handles customer service and support tasks.

[0101] "API" stands for Application Programming Interface and refers to an interface for exchanging data and functions between different software applications.

[0102] This section describes an embodiment of the present invention. This system is primarily comprised of a server and has the functions of collecting customer feedback data from external data sources, preprocessing and summarizing it, and automatically generating and distributing reports. This system makes it possible to efficiently collect, analyze, and report customer feedback.

[0103] System configuration

[0104] The system consists of the following major hardware and software components:

[0105] Server: The central unit that processes and analyzes data. A general-purpose server equipped with a high-performance CPU and large memory capacity is used.

[0106] Network Interface: A network connection for communicating with external data sources.

[0107] Generative AI models: Generative AI models such as GPT-4 are used as models for natural language processing.

[0108] API: Used to retrieve data from external data sources. Examples include the Facebook API and Twitter API.

[0109] Mail Server: Email sending facility for delivering reports to service personnel.

[0110] Internal messaging system: A dedicated communication system for sharing information within an organization.

[0111] Data collection

[0112] The server runs a scheduled task every morning at 2:00 AM to collect customer feedback data from external data sources using APIs. For example, it uses the Facebook API to collect posts related to "New Product X." By sending an API request, data that matches the specified criteria is retrieved.

[0113] Data Preprocessing

[0114] The collected data is pre-processed on the server, which includes removing unnecessary characters and tags, filtering duplicate data, and eliminating inappropriate content. This process provides a clean dataset suitable for analysis.

[0115] Data Summarization

[0116] The preprocessed data is then input into a generative AI model, which uses prompts to provide specific summarization instructions to the model. For example, a prompt might be, "Summarize the main customer complaints about new product X." The generative AI model then extracts key information from the input data and provides a summary result.

[0117] Automatic report generation

[0118] The server automatically generates a PDF report based on the summarized data, including graphs and statistics based on the summary content, and formats the report according to an automatically generated template.

[0119] Report distribution

[0120] The generated reports are delivered from the server to service personnel via email from the mail server and via the internal messaging system, allowing personnel to receive timely feedback from customers.

[0121] Specific examples

[0122] For example, the server collects posts about "New Product X" using the Facebook API at 2:00 a.m. every morning and temporarily stores them in a database. It then performs data preprocessing and summarizes them by inputting a prompt to the generative AI model: "Summarize the main customer complaints about New Product X." Finally, it generates a PDF report based on the summary results and sends it to the service representative via email.

[0123] In this way, the system automates the entire process from data collection to report distribution, enabling efficient analysis and reporting of customer feedback.

[0124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0125] Step 1:

[0126] Data collection

[0127] The server runs a scheduled task every morning at 2am, using APIs to collect voice of customer data from external data sources.

[0128] Input: Facebook API key, search query (e.g. "New Product X").

[0129] Specific operation: The server sends an API request to retrieve data based on the specified conditions, and the retrieved data is temporarily stored in a database.

[0130] Output: The raw data collected (e.g. Facebook posts).

[0131] Step 2:

[0132] Data Preprocessing

[0133] The server pre-processes the collected raw data.

[0134] Input: Raw data collected.

[0135] What it does: The server uses regular expressions to remove unwanted characters and tags, and also filters out duplicate data and spam posts.

[0136] Output: A clean dataset.

[0137] Step 3:

[0138] Data Summarization

[0139] The server inputs the preprocessed data into a generative AI model to generate summaries.

[0140] Input: A clean dataset, a prompt statement (e.g., "Summarize the main customer complaints about new product X").

[0141] Specific operation: The server sends prompt sentences to the generative AI model (GPT-4) to generate a summary of the data.

[0142] Output: Summarized text data.

[0143] Step 4:

[0144] Automatic report generation

[0145] The server automatically generates a PDF report based on the summarized data.

[0146] Input: Abstracted text data.

[0147] What it does: The server embeds summary information into a report template, adds graphs and statistics, and generates a PDF report.

[0148] Output: Generated PDF report.

[0149] Step 5:

[0150] Report distribution

[0151] The server delivers the generated report to the service representative.

[0152] Input: PDF report, service representative's email address or internal messaging system user ID.

[0153] What it does: The server uses the SMTP library to send emails and notify through its internal messaging system.

[0154] Output: The delivered report.

[0155] (Application example 1)

[0156] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0157] In today's highly competitive market, brick-and-mortar stores need to quickly and efficiently collect customer feedback and use it to improve their services. Traditional feedback collection methods are manual, time-consuming, and labor-intensive, and analyzing data and generating reports is difficult. In particular, there is a need for an integrated system that utilizes smart devices such as smartphones, tablets, and smart glasses to efficiently collect feedback and effectively analyze it.

[0158] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0159] In this invention, the server includes means for automatically collecting customer feedback data from external data sources, means for preprocessing the collected customer feedback data, means for generating an AI model that summarizes the preprocessed customer feedback data, means for automatically generating a report based on the summarized data, means for distributing the generated report to a service representative, means for collecting customer feedback via a smart device, and means for automatically sending the generated report to a store manager or staff member, thereby enabling efficient collection and analysis of customer feedback and rapid service improvement.

[0160] Definitions of important words

[0161] "External data sources" are external data providers from which customer feedback data can be obtained, such as app stores, social media, and customer support systems.

[0162] "Voice of customer data" refers to text data that includes feedback, opinions, and impressions provided by customers regarding products and services.

[0163] "Preprocessing" refers to a series of processes that remove unnecessary characters and tags from collected text data, delete duplicates, and filter inappropriate content.

[0164] A "generative AI model" is an artificial intelligence model used to extract and summarize important information from collected and pre-processed data. A specific example is GPT-4.

[0165] A "report" is a document generated from summarized data that visualizes and presents product or service performance, trends, and areas for improvement.

[0166] "Smart devices" are portable electronic devices with advanced functions, such as smartphones, tablets, and smart glasses.

[0167] "Store Manager" means the person responsible for the operation and management of the physical store.

[0168] "Staff" refers to employees who deal with customers and perform other duties at the store.

[0169] "API" stands for Application Programming Interface, a standardized means of exchanging data and functions between different software programs.

[0170] "Automatic generation" is a process in which a system automatically processes and generates results without human intervention.

[0171] "Auto Send" is a feature that automatically sends data such as reports generated by the system to designated recipients.

[0172] MODE FOR CARRYING OUT THE INVENTION

[0173] An embodiment of the present invention will now be described. This system is centered around a server, and efficiently collects, analyzes, and generates reports on customer feedback in physical stores via smart devices. This allows store managers and staff to understand customer feedback in real time and respond quickly.

[0174] System Overview

[0175] Hardware and software used

[0176] Hardware:

[0177] Smartphone

[0178] tablet

[0179] Smart Glasses

[0180] software:

[0181] iOS / Android app

[0182] Smart Glasses App

[0183] Server (Amazon Web Services, etc.)

[0184] Generative AI model (GPT-4)

[0185] Program processing description

[0186] 1. Data Collection:

[0187] Terminal: Smart devices (smartphones, tablets, smart glasses) installed in the store provide a feedback input form for customers. The feedback data entered by customers is sent to the server. In addition, external posting data about the store is collected using SNS APIs (e.g., Instagram API and Twitter API).

[0188] Example: Customers can enter feedback via in-store tablets, such as:

[0189] "The staff were amazing! I wish they'd hire more staff so the cashiers don't get too crowded."

[0190] 2. Data Preprocessing:

[0191] Server: Text cleans the collected data, removing unnecessary characters, tags, and duplicate data, as well as filtering out inappropriate content.

[0192] Example: Remove emojis and HTML tags to generate clean text data.

[0193] 3. Data Summary:

[0194] Server: Feeds pre-processed data into a generative AI model (GPT-4) to summarize key topics, trends, customer requests and complaints.

[0195] Example prompt sentence:

[0196] Summarize the following customer feedback:

[0197] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[0198] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[0199] 4. Report Generation:

[0200] Server: Based on the summarized information, the server visualizes store performance and customer opinions, automatically generating reports with charts and statistics, which are saved in PDF and HTML formats.

[0201] Example: The report includes the advantages and disadvantages of the product or service, the items most mentioned by customers, and suggestions for improvement.

[0202] 5. Report Delivery:

[0203] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback with staff using the notification system.

[0204] Example: Every day, an updated report is emailed to the store manager's smartphone and also notified via an internal notification system.

[0205] This enables efficient collection and analysis of customer feedback and rapid service improvement.

[0206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0207] Program processing steps

[0208] Step 1: Data collection

[0209] input:

[0210] Customer feedback input from in-store smart devices

[0211] External posting data from SNS API

[0212] Specific behavior:

[0213] Terminals: Smart devices (smartphones, tablets, smart glasses) installed in the store collect customer feedback and send it to the server.

[0214] Server: Uses APIs to collect store-related post data from social media and external data sources (e.g., Instagram API and Twitter API) and stores it in temporary storage.

[0215] output:

[0216] Feedback data and SNS posting data stored in temporary storage

[0217] Step 2: Data Preprocessing

[0218] input:

[0219] Feedback data and SNS posting data stored in temporary storage

[0220] Specific behavior:

[0221] Server: Cleans the collected data, removing unnecessary characters, tags, and duplicate data, and filters out inappropriate content, using regular expressions (RegEx) and text analysis tools.

[0222] output:

[0223] Clean text data

[0224] Step 3: Summarize the data

[0225] input:

[0226] Clean text data

[0227] Specific behavior:

[0228] Server: Inputs the preprocessed data into a generative AI model (GPT-4) to summarize important information and trends, and provides appropriate instructions to the model using prompts.

[0229] Example prompt sentence:

[0230] Summarize the following customer feedback:

[0231] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[0232] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[0233] output:

[0234] Summarized feedback information

[0235] Step 4: Generate a report

[0236] input:

[0237] Summarized feedback information

[0238] Specific behavior:

[0239] Server: Based on the summarized information, the server automatically generates reports containing charts and statistics that visualize store performance and customer opinions. Reports are saved in PDF and HTML formats.

[0240] output:

[0241] PDF and HTML reports

[0242] Step 5: Report Delivery

[0243] input:

[0244] PDF and HTML reports

[0245] Specific behavior:

[0246] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback using an internal notification system.

[0247] output:

[0248] Reports and notifications sent to store managers and staff's smart devices

[0249] By implementing each processing step in this manner, it becomes possible to efficiently collect and analyze customer feedback and quickly improve services.

[0250] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0251] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, preprocesses it, summarizes it using a generative AI model, and then recognizes user emotions using an emotion engine. It then automatically generates a report and delivers it to service personnel. This allows for a deeper understanding of customer feedback, and enables efficient collection, analysis, and reporting at low cost.

[0252] System Overview

[0253] This system is centered around a server and includes the following main functions:

[0254] 1. Collecting data from external sources

[0255] The server collects customer feedback data from external sources such as app stores, social media, and customer support systems using APIs or scraping techniques. For example, Twitter APIs can be used to automatically retrieve product posts.

[0256] 2. Data Preprocessing

[0257] The server preprocesses the collected customer feedback data. Preprocessing includes text cleaning, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[0258] 3. Data Summarization

[0259] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[0260] 4. Emotion Recognition by Emotion Engine

[0261] The server inputs the summarized data into an emotion engine to analyze the user's emotions. The emotion engine uses natural language processing technology to recognize the customer's emotions (e.g., joy, anger, sadness, etc.) from the text data.

[0262] 5. Automatic report generation

[0263] The server automatically generates reports based on the summarized data and the results of sentiment analysis by the sentiment engine. The reports include a summary of customer feedback along with the results of the sentiment analysis. A customized report template is used depending on the needs of each service. For example, a report in PDF format is generated that categorizes the summarized customer feedback into product advantages and disadvantages and includes graphs and statistics that reflect the sentiment analysis results.

[0264] 6. Report Distribution

[0265] The server automatically delivers the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[0266] Specific examples

[0267] A specific example of this system is shown below.

[0268] Voice of the Customer Data Collection Example

[0269] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[0270] Data Preprocessing Example

[0271] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[0272] Data Summarization Example

[0273] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[0274] Emotion Recognition Example

[0275] The server inputs the data summarized by the generative AI model into an emotion engine to recognize customer emotions, for example, extracting "joy" from the text "This product is very good."

[0276] Report Generation Example

[0277] The server automatically generates a detailed product report based on the summarized information and emotion recognition results, in PDF format, including charts and graphs.

[0278] Report Delivery Example

[0279] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[0280] In this way, this system efficiently collects and analyzes customer feedback, understands customer emotions through sentiment analysis, and promptly provides reports, thereby supporting service improvements and rapid response.

[0281] The processing flow will be explained below.

[0282] Step 1:

[0283] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[0284] Step 2:

[0285] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[0286] Step 3:

[0287] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[0288] Step 4:

[0289] The server cleans the raw data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and profanity) from the text data, for example by using regular expressions to remove emojis and tags.

[0290] Step 5:

[0291] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, improving the accuracy of the analysis. For example, it generates hash values ​​to remove duplicate posts and combines data with the same hash value.

[0292] Step 6:

[0293] The server initializes and configures the generative AI model. The generative AI model (e.g., GPT-4) uses specific settings and prompts for summarizing the data, such as the length of the summary or the priority of certain keywords.

[0294] Step 7:

[0295] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates concise summaries, such as identifying the characteristics and issues frequently mentioned by customers.

[0296] Step 8:

[0297] The server inputs the data summarized by the generative AI model into the emotion engine to recognize the user's emotions. The emotion engine uses natural language processing technology to analyze customer emotions (e.g., joy, anger, sadness, etc.) from the text data. For example, it extracts "joy" from the text "This product is very good."

[0298] Step 9:

[0299] The server automatically generates reports based on the summarized data and the sentiment analysis results from the sentiment engine. The server then inserts the summarized information and sentiment analysis results into pre-configured report templates to create customized reports tailored to the needs of each service. For example, the server visually displays the sentiment analysis results as charts and graphs.

[0300] Step 10:

[0301] The server saves the generated report and renders the document in a specified format, such as PDF or HTML. For example, the report can be saved in PDF format and made available as an email attachment.

[0302] Step 11:

[0303] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report to, for example, obtains the latest email address from the list of representatives.

[0304] Step 12:

[0305] The server adds a report summary to the body of the email and attaches the generated report document, giving a concise understanding of the email's contents. For example, it might insert a simple message such as "This week's customer feedback report is attached."

[0306] Step 13:

[0307] The server sends emails to the relevant personnel and sends notifications to an internal messaging system, so that all relevant personnel receive the latest report information, for example, by sending notifications to an internal chat tool.

[0308] Example 2

[0309] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] There is a need to collect, analyze, and report customer opinions effectively and efficiently, but traditional methods require a great deal of time and effort to process large amounts of data. It is also difficult to accurately grasp customer sentiment, which can delay service improvements and prompt responses. This can lead to lower customer satisfaction and a loss of trust in companies.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0312] In this invention, the server includes means for automatically collecting customer opinion data from an external data source, means for preprocessing the collected customer opinion data, means for generating an AI model that summarizes the preprocessed customer opinion data, means for analyzing the summarized data and recognizing customer emotions, means for automatically generating a report based on the summarized data and the emotion recognition results, and means for delivering the generated report to a person in charge. This makes it possible to efficiently collect and analyze customer opinions, accurately grasp customer emotions, and quickly provide them as a report.

[0313] "External Data Sources" refers to data available from publicly available databases on the internet, social media platforms, online stores, and other digital environments.

[0314] "Customer Opinion Data" means customer ratings, feedback, reviews, comments, and other textual opinions about products and services.

[0315] "Automated collection means" refers to technology designed to collect data continuously, using programmable means, without human intervention.

[0316] "Preprocessing means" refers to technologies used to process collected data, such as by cleaning text, removing duplicate data, and filtering inappropriate content, before analyzing the data.

[0317] "Generative AI model means" refers to a system or program that uses artificial intelligence technology to extract important information from input data and generate a summary statement.

[0318] "Means for recognizing emotions" refers to technology that uses natural language processing technology to analyze customer emotions from text data and assign specific emotional labels (e.g., joy, anger, sadness, etc.).

[0319] "Means for automatically generating reports" refers to a system or program that automatically generates reports based on summarized data and emotion recognition results using predefined templates.

[0320] "Delivery means" refers to a system or program for sending generated reports to appropriate personnel via email or internal messaging systems.

[0321] A specific embodiment for implementing this invention will be described below. This system automatically collects customer opinion data from external data sources, preprocesses it, summarizes it using a generative AI model, recognizes customer emotions through an emotion engine, and automatically generates a report based on the results and delivers it to the person in charge.

[0322] System Overview

[0323] This system is centered around the server and includes the following main functions:

[0324] Data collection from external data sources

[0325] The server uses databases and social media platforms publicly available on the Internet as external data sources. Specific examples include collecting data using the Twitter API and Facebook API. For example, every morning at 2:00, the server automatically retrieves tweets related to "product name" via the Twitter API and stores them in temporary storage.

[0326] Data Preprocessing

[0327] The server pre-processes the collected data, which includes text cleaning (removing unnecessary characters and tags), removing duplicate data, and filtering inappropriate content. Regular expressions and filtering algorithms are used to remove emojis and HTML tags, and then the clean data is generated.

[0328] Data Summarization

[0329] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to extract and summarize important information. For example, the server passes text data to a GPT-4 model and receives text summarizing key topics and customer opinions as output.

[0330] emotion recognition

[0331] The server inputs the summarized data into an emotion engine to analyze customer emotions. The emotion engine uses natural language processing technology to assign emotion labels such as "happiness," "anger," and "sadness" to the text. For example, it extracts "happiness" from the text "This product is very good."

[0332] Report Generation

[0333] The server automatically generates a report based on the summarized data and emotion recognition results. The report is created using a template and includes product advantages and disadvantages, as well as customer sentiment analysis in the form of charts and graphs. The generated report is saved in PDF or HTML format.

[0334] Report Distribution

[0335] The server automatically delivers the generated report to the person in charge by attaching the report to an email using the SMTP protocol and sending it to a specified email address. For example, the server could send the latest report to the person in charge's mailbox every Monday.

[0336] Specific examples

[0337] A specific example of this system is shown below.

[0338] Customer Opinion Data Collection Example

[0339] Every morning at 2am, the server uses the Facebook API to collect posts about a specific product and stores them in temporary storage.

[0340] Specific examples of data preprocessing

[0341] The server then removes unnecessary characters and tags from the collected posts, filters out duplicates and inappropriate content, and uses regular expressions and filtering algorithms to generate a clean dataset.

[0342] Examples of data summarization

[0343] The server feeds the preprocessed data into a GPT-4 model, which generates text summarizing key topics and trends.

[0344] Specific examples of emotion recognition

[0345] The server inputs the summaries generated by the generative AI model into an emotion engine to analyze customer emotions, for example, extracting the emotion "joy" from the text "This product is very good."

[0346] Example of report generation

[0347] The server automatically generates a detailed report based on the summarized data and emotion recognition results, which is saved in PDF format and includes charts and graphs.

[0348] Example of report distribution

[0349] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[0350] This format makes it possible to efficiently collect customer opinions, conduct detailed analysis, and quickly provide the results in report format.

[0351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0352] Step 1: Collect data from external data sources

[0353] The server collects customer opinion data from external data sources. Specifically, every morning at 2:00 AM, the server retrieves posts related to specific keywords or hashtags using the Twitter API or Facebook API. The collected data is stored in temporary storage in JSON format.

[0354] Input: External data source (Twitter API, Facebook API, etc.)

[0355] Data processing: API requests, data acquisition, conversion to JSON format

[0356] Output: JSON format data stored in temporary storage

[0357] Step 2: Preprocessing the data

[0358] The server preprocesses the collected data. First, it reads the JSON data and removes unnecessary characters and tags (e.g., emojis, HTML tags). Second, it detects and removes duplicate data. Third, it filters out inappropriate content.

[0359] Input: JSON format data stored in temporary storage

[0360] Data operations: text cleaning, de-duplicating data, filtering

[0361] Output: A preprocessed, clean dataset

[0362] Step 3: Summarize the data

[0363] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate a summary. Specifically, the server passes the clean text data to GPT-4 and receives a summary that extracts important topics and claims.

[0364] Input: Preprocessed clean dataset

[0365] Data calculation: Extraction of important information and generation of summaries using generative AI models

[0366] Output: Generated summary

[0367] Step 4: Emotion Recognition

[0368] The server inputs the summary sentence into the emotion engine for emotion analysis. The emotion engine uses natural language processing technology to identify the customer's emotion (e.g., joy, anger, sadness) from the text.

[0369] Input: Generated summary

[0370] Data Computing: Sentiment Analysis with Natural Language Processing

[0371] Output: Sentiment analysis results with emotion labels

[0372] Step 5: Generate a report

[0373] The server automatically generates a report based on the summary and the results of the sentiment analysis. The report is created based on a pre-prepared template and includes the summary and the results of the sentiment analysis as charts and graphs. The generated report is saved in PDF or HTML format.

[0374] Input: Summary and sentiment analysis results

[0375] Data calculation: Insert data into templates, automatically generate reports

[0376] Output: Report in PDF or HTML format

[0377] Step 6: Report Distribution

[0378] The server automatically delivers the generated report to the person in charge by attaching the report to an email and sending it to the person's email address. It also notifies the person in charge via the internal messaging system.

[0379] Input: Report in PDF or HTML format

[0380] Data calculation: Email composition and sending using the SMTP protocol

[0381] Output: Reports sent to the assignee's mailbox and notifications in the internal messaging system

[0382] Through these steps, this system is able to efficiently collect and analyze customer opinion data and quickly provide it as a report.

[0383] (Application example 2)

[0384] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0385] Autonomous vehicles require efficient collection and analysis of passenger feedback and the ability to improve vehicle performance and user experience based on that feedback. However, traditional methods often require manual collection of feedback, and data analysis and emotion recognition are ineffective, resulting in limited real-time performance and accuracy. Additionally, there is a lack of a way to quickly share reports generated based on feedback with relevant parties. This can delay the implementation of prompt improvements based on feedback.

[0386] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automatically collecting customer voice data from an external data source, means for preprocessing the collected customer voice data, generative AI model means for summarizing the preprocessed customer voice data, emotion engine means for analyzing emotions based on the summarized data, means for automatically generating a report based on the analyzed data, and means for distributing the generated report to a vehicle manager or developer of the autonomous vehicle. This makes it possible to collect and analyze passenger feedback in real time and quickly distribute the results to relevant parties.

[0387] "External data sources" are sources of data obtained from outside sources such as social media, customer support systems, app stores, etc.

[0388] "Voice of Customer Data" is text data that includes feedback, opinions, and ratings expressed by customers about products and services.

[0389] "Preprocessing" is a series of data cleansing steps that remove unnecessary characters and tags from raw data and filter out duplicate data and inappropriate content.

[0390] A "generative AI model" is an artificial intelligence model used to extract and summarize key topics and trends from large amounts of data, such as GPT-4.

[0391] An "emotion engine" is an engine that uses natural language processing technology to analyze customer emotions (joy, anger, sadness, etc.) from text data.

[0392] A "report" is a document generated based on summarized voice of customer data and sentiment analysis results, and may include graphs and statistics.

[0393] "Vehicle managers and developers" are people or positions involved in the operation, maintenance, and development of autonomous vehicles who implement improvement measures based on feedback data.

[0394] System Overview

[0395] A system for implementing this invention automatically collects customer voice-of-mouth data from external data sources, summarizes it using a generative AI model and an emotion engine, and performs sentiment analysis. It then automatically generates reports based on the analysis results and distributes them to fleet managers and developers. This allows feedback on autonomous vehicles to be efficiently collected and analyzed and quickly shared with relevant parties.

[0396] Hardware and software used

[0397] 1. Hardware:

[0398] Server (for data collection and processing)

[0399] Smartphone (for collecting feedback)

[0400] 2. Software:

[0401] Requests: An HTTP request library for gathering data from external data sources.

[0402] OpenAI API: Generate summaries of data using generative AI models (e.g., GPT-4)

[0403] EmotionEngine: A natural language processing engine for emotion analysis

[0404] ReportLab: A library for saving automatically generated reports in PDF format.

[0405] Data processing and calculation

[0406] 1. Data Collection

[0407] The server automatically collects customer feedback data from external data sources (e.g., social media, customer support systems) via APIs, for example, by sending an HTTP request to an API endpoint and temporarily storing the retrieved data.

[0408] 2. Data Preprocessing

[0409] The collected customer feedback data is preprocessed by the server, which removes unnecessary characters and emojis, deletes HTML tags, and filters out duplicate data to generate clean text data.

[0410] 3. Data Summary

[0411] The preprocessed text data is then fed into a generative AI model (e.g., GPT-4) to summarize the key information. Generative AI models extract the most important topics and trends from the vast amount of data.

[0412] 4. Emotion analysis

[0413] The summarized text data is input to an emotion engine, which analyzes the customer's emotions (e.g., joy, anger, sadness, etc.). The emotion analysis results show the distribution of emotions for the customer's feedback.

[0414] 5. Report Generation and Delivery

[0415] Based on the analyzed data, the server automatically generates a report in PDF format. The report includes summary information, sentiment analysis results, graphs, and statistical information. The generated report is automatically distributed to vehicle managers and developers. For example, the report can be sent by email every week.

[0416] Specific examples

[0417] For example, suppose a passenger posts feedback such as, "This car is very comfortable, but the air conditioning doesn't work well." This feedback can be summarized as, "The vehicle is comfortable, but I'm dissatisfied with the air conditioning performance," and sentiment analysis can extract "happiness" and "dissatisfaction."

[0418] Prompt Sentence Examples

[0419] "Please summarize the following text:

[0420] This car is very comfortable, but the air conditioning doesn't work well.

[0421] By using the above system, customer feedback can be efficiently collected and analyzed, and used to improve the quality of autonomous vehicles.

[0422] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0423] Step 1:

[0424] Data collection

[0425] The server automatically collects customer feedback data from external data sources (e.g., social media or customer support systems). Specifically, the server uses an API to send an HTTP request and temporarily stores the data obtained as an API response. For example, a request is made to an API endpoint of a social media service to obtain post data containing keywords such as "vehicle," "comfort," and "air conditioning." This data is often stored in JSON format. The input is raw data obtained from the external data source, and the output is unprocessed data stored in temporary storage.

[0426] Step 2:

[0427] Data Preprocessing

[0428] The collected customer voice data is pre-processed by the server. Specifically, it removes inappropriate characters (e.g., emojis, HTML tags) and duplicate data, and normalizes the text. For example, "This car is very comfortable." The feedback "This car is very comfortable, but the air conditioning doesn't work well 🎉" is transformed into "This car is very comfortable, but the air conditioning doesn't work well." The input is raw data, and the output is clean text data.

[0429] Step 3:

[0430] Data Summary

[0431] The preprocessed customer voice data is input into a generative AI model by the server. This AI model summarizes the text using, for example, GPT-4. Specifically, it extracts key topics and trends from large amounts of feedback data and outputs them as summarized text. The prompt includes the instruction "Please summarize the following text:\n[text data]". The input is clean text data, and the output is summarized text.

[0432] Step 4:

[0433] Emotion analysis

[0434] The server inputs the summarized text data into an emotion engine, which analyzes the customer's emotions. Specifically, the data is classified into emotion categories such as "happiness," "anger," and "sadness," and a determination is made as to which emotion the text best expresses. For example, "happiness" and "dissatisfaction" are extracted from the text "This car is very comfortable, but the air conditioning doesn't work well." The input is the summarized text, and the output is the emotion analysis results.

[0435] Step 5:

[0436] Report Generation

[0437] Based on the analysis results, the server automatically generates a report. Specifically, it uses a library such as ReportLab to create a PDF report. The report includes summarized text, sentiment analysis results, graphs, and statistical information. For example, the report includes the original feedback in the "Product Benefits" section. The input is the summarized data and sentiment analysis results, and the output is a completed report file.

[0438] Step 6:

[0439] Report Distribution

[0440] The generated reports are automatically distributed by the server to vehicle managers and developers. Specifically, reports are sent to relevant parties using email or an internal messaging system. For example, the latest report is automatically sent at a fixed date and time each week. The input is a completed report file, and the output is a report delivered to the relevant parties' mailboxes.

[0441] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0442] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0443] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0444] [Second embodiment]

[0445] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0446] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0447] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0448] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0449] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0450] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0451] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0452] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0453] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0454] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0455] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0456] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0457] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, performs preprocessing, summarizes it using a generative AI model, automatically generates a report based on the summarized data, and finally delivers the report to service personnel. This allows for the collection, analysis, and reporting of customer feedback at low cost and efficiently.

[0458] System Overview

[0459] This system is centered around a server and includes the following main functions:

[0460] 1. Collecting data from external sources

[0461] The server collects customer feedback data from external data sources such as app stores, social media, customer support systems, etc. This collection can be done using APIs or scraping techniques, for example, using the Twitter API to automatically retrieve posts about the product.

[0462] 2. Data Preprocessing

[0463] The server preprocesses the collected customer feedback data. Preprocessing includes cleaning the text, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[0464] 3. Data Summarization

[0465] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[0466] 4. Automatic report generation

[0467] The server automatically generates reports based on the summarized data, using report templates customized to the needs of each service. For example, it could categorize summarized customer testimonials into product advantages and disadvantages and generate a PDF report with graphs and statistics.

[0468] 5. Report Distribution

[0469] The server automatically distributes the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[0470] Specific examples

[0471] A specific example of this system is shown below.

[0472] Voice of the Customer Data Collection Example

[0473] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[0474] Data Preprocessing Example

[0475] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[0476] Data Summarization Example

[0477] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[0478] Report Generation Example

[0479] The server automatically generates a detailed product report based on the summarized information, in PDF format, including charts and graphs.

[0480] Report Delivery Example

[0481] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[0482] In this way, this system efficiently collects and analyzes customer feedback and promptly provides reports, thereby supporting service improvements and rapid response.

[0483] The processing flow will be explained below.

[0484] Step 1:

[0485] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[0486] Step 2:

[0487] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[0488] Step 3:

[0489] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[0490] Step 4:

[0491] The server cleans the text data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and inappropriate language) from the raw data.

[0492] Step 5:

[0493] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, which improves the accuracy of the analysis.

[0494] Step 6:

[0495] The server initializes and configures the generative AI model, which (e.g., GPT-4) uses specific configuration values ​​and prompts to summarize the data.

[0496] Step 7:

[0497] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates a concise summary.

[0498] Step 8:

[0499] The server automatically generates reports based on the summarized data, inserting the summarized information into pre-configured report templates to create customized reports tailored to the needs of each service.

[0500] Step 9:

[0501] The server saves the generated report and formats the document in the specified format, such as PDF or HTML.

[0502] Step 10:

[0503] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report.

[0504] Step 11:

[0505] The server adds a report summary to the body of the email and attaches the generated report document, allowing you to understand the contents of the email concisely.

[0506] Step 12:

[0507] The server sends emails to personnel and sends notifications to an internal messaging system, ensuring that all relevant personnel receive the latest report information.

[0508] Example 1

[0509] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0510] Traditional methods for collecting, analyzing, and reporting customer feedback data required a great deal of time and effort, making it difficult to efficiently collect and analyze information. Furthermore, because much of the processing was manual, it was difficult to maintain data accuracy and consistency. Traditional methods were particularly difficult to use when large amounts of data needed to be collected and analyzed in a short amount of time.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0512] In this invention, the server includes means for automatically collecting customer voice data from external data sources, means for preprocessing the collected customer voice data, a generative AI model means for summarizing the preprocessed customer voice data, means for instructing the generative AI model to perform summarization using a prompt sentence, means for automatically generating a report in PDF format based on the summarized data, and means for delivering the generated report to a service representative via email or an internal message system. This makes it possible to efficiently and accurately collect and analyze customer voice data and quickly provide information to a service representative.

[0513] "External Data Sources" refers to sources of data obtained from various platforms and services on the Internet.

[0514] "Customer voice data" refers to text data such as opinions, impressions, and evaluations expressed by users about products and services.

[0515] "Preprocessing" refers to the process of removing unnecessary information from collected data and preparing it in a format suitable for analysis.

[0516] A "generative AI model" refers to an artificial intelligence program that learns from large amounts of data and performs natural language processing and text generation.

[0517] A "prompt sentence" refers to input text that instructs a generative AI model on specific actions and processing details.

[0518] "PDF format" is an abbreviation for Portable Document Format, and refers to a file format for electronically storing and displaying documents and image information.

[0519] "Mail" is an abbreviation for electronic mail and refers to a means of communication for exchanging text and files over the Internet.

[0520] An "internal messaging system" refers to a dedicated communication system for exchanging information and messages within an organization.

[0521] "Service representative" refers to a person who handles customer service and support tasks.

[0522] "API" stands for Application Programming Interface and refers to an interface for exchanging data and functions between different software applications.

[0523] This section describes an embodiment of the present invention. This system is primarily comprised of a server and has the functions of collecting customer feedback data from external data sources, preprocessing and summarizing it, and automatically generating and distributing reports. This system makes it possible to efficiently collect, analyze, and report customer feedback.

[0524] System configuration

[0525] The system consists of the following major hardware and software components:

[0526] Server: The central unit that processes and analyzes data. A general-purpose server equipped with a high-performance CPU and large memory capacity is used.

[0527] Network Interface: A network connection for communicating with external data sources.

[0528] Generative AI models: Generative AI models such as GPT-4 are used as models for natural language processing.

[0529] API: Used to retrieve data from external data sources. Examples include the Facebook API and Twitter API.

[0530] Mail Server: Email sending facility for delivering reports to service personnel.

[0531] Internal messaging system: A dedicated communication system for sharing information within an organization.

[0532] Data collection

[0533] The server runs a scheduled task every morning at 2:00 AM to collect customer feedback data from external data sources using APIs. For example, it uses the Facebook API to collect posts related to "New Product X." By sending an API request, data that matches the specified criteria is retrieved.

[0534] Data Preprocessing

[0535] The collected data is pre-processed on the server, which includes removing unnecessary characters and tags, filtering duplicate data, and eliminating inappropriate content. This process provides a clean dataset suitable for analysis.

[0536] Data Summarization

[0537] The preprocessed data is then input into a generative AI model, which uses prompts to provide specific summarization instructions to the model. For example, a prompt might be, "Summarize the main customer complaints about new product X." The generative AI model then extracts key information from the input data and provides a summary result.

[0538] Automatic report generation

[0539] The server automatically generates a PDF report based on the summarized data, including graphs and statistics based on the summary content, and formats the report according to an automatically generated template.

[0540] Report distribution

[0541] The generated reports are delivered from the server to service personnel via email from the mail server and via the internal messaging system, allowing personnel to receive timely feedback from customers.

[0542] Specific examples

[0543] For example, the server collects posts about "New Product X" using the Facebook API at 2:00 a.m. every morning and temporarily stores them in a database. It then performs data preprocessing and summarizes them by inputting a prompt to the generative AI model: "Summarize the main customer complaints about New Product X." Finally, it generates a PDF report based on the summary results and sends it to the service representative via email.

[0544] In this way, the system automates the entire process from data collection to report distribution, enabling efficient analysis and reporting of customer feedback.

[0545] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0546] Step 1:

[0547] Data collection

[0548] The server runs a scheduled task every morning at 2am, using APIs to collect voice of customer data from external data sources.

[0549] Input: Facebook API key, search query (e.g. "New Product X").

[0550] Specific operation: The server sends an API request to retrieve data based on the specified conditions, and the retrieved data is temporarily stored in a database.

[0551] Output: The raw data collected (e.g. Facebook posts).

[0552] Step 2:

[0553] Data Preprocessing

[0554] The server pre-processes the collected raw data.

[0555] Input: Raw data collected.

[0556] What it does: The server uses regular expressions to remove unwanted characters and tags, and also filters out duplicate data and spam posts.

[0557] Output: A clean dataset.

[0558] Step 3:

[0559] Data Summarization

[0560] The server inputs the preprocessed data into a generative AI model to generate summaries.

[0561] Input: A clean dataset, a prompt statement (e.g., "Summarize the main customer complaints about new product X").

[0562] Specific operation: The server sends prompt sentences to the generative AI model (GPT-4) to generate a summary of the data.

[0563] Output: Summarized text data.

[0564] Step 4:

[0565] Automatic report generation

[0566] The server automatically generates a PDF report based on the summarized data.

[0567] Input: Abstracted text data.

[0568] What it does: The server embeds summary information into a report template, adds graphs and statistics, and generates a PDF report.

[0569] Output: Generated PDF report.

[0570] Step 5:

[0571] Report distribution

[0572] The server delivers the generated report to the service representative.

[0573] Input: PDF report, service representative's email address or internal messaging system user ID.

[0574] What it does: The server uses the SMTP library to send emails and notify through its internal messaging system.

[0575] Output: The delivered report.

[0576] (Application example 1)

[0577] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] In today's highly competitive market, brick-and-mortar stores need to quickly and efficiently collect customer feedback and use it to improve their services. Traditional feedback collection methods are manual, time-consuming, and labor-intensive, and analyzing data and generating reports is difficult. In particular, there is a need for an integrated system that utilizes smart devices such as smartphones, tablets, and smart glasses to efficiently collect feedback and effectively analyze it.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0580] In this invention, the server includes means for automatically collecting customer feedback data from external data sources, means for preprocessing the collected customer feedback data, means for generating an AI model that summarizes the preprocessed customer feedback data, means for automatically generating a report based on the summarized data, means for distributing the generated report to a service representative, means for collecting customer feedback via a smart device, and means for automatically sending the generated report to a store manager or staff member, thereby enabling efficient collection and analysis of customer feedback and rapid service improvement.

[0581] Definitions of important words

[0582] "External data sources" are external data providers from which customer feedback data can be obtained, such as app stores, social media, and customer support systems.

[0583] "Voice of customer data" refers to text data that includes feedback, opinions, and impressions provided by customers regarding products and services.

[0584] "Preprocessing" refers to a series of processes that remove unnecessary characters and tags from collected text data, delete duplicates, and filter inappropriate content.

[0585] A "generative AI model" is an artificial intelligence model used to extract and summarize important information from collected and pre-processed data. A specific example is GPT-4.

[0586] A "report" is a document generated from summarized data that visualizes and presents product or service performance, trends, and areas for improvement.

[0587] "Smart devices" are portable electronic devices with advanced functions, such as smartphones, tablets, and smart glasses.

[0588] "Store Manager" means the person responsible for the operation and management of the physical store.

[0589] "Staff" refers to employees who deal with customers and perform other duties at the store.

[0590] "API" stands for Application Programming Interface, a standardized means of exchanging data and functions between different software programs.

[0591] "Automatic generation" is a process in which a system automatically processes and generates results without human intervention.

[0592] "Auto Send" is a feature that automatically sends data such as reports generated by the system to designated recipients.

[0593] MODE FOR CARRYING OUT THE INVENTION

[0594] An embodiment of the present invention will now be described. This system is centered around a server, and efficiently collects, analyzes, and generates reports on customer feedback in physical stores via smart devices. This allows store managers and staff to understand customer feedback in real time and respond quickly.

[0595] System Overview

[0596] Hardware and software used

[0597] Hardware:

[0598] Smartphone

[0599] tablet

[0600] Smart Glasses

[0601] software:

[0602] iOS / Android app

[0603] Smart Glasses App

[0604] Server (Amazon Web Services, etc.)

[0605] Generative AI model (GPT-4)

[0606] Program processing description

[0607] 1. Data Collection:

[0608] Terminal: Smart devices (smartphones, tablets, smart glasses) installed in the store provide a feedback input form for customers. The feedback data entered by customers is sent to the server. In addition, external posting data about the store is collected using SNS APIs (e.g., Instagram API and Twitter API).

[0609] Example: Customers can enter feedback via in-store tablets, such as:

[0610] "The staff were amazing! I wish they'd hire more staff so the cashiers don't get too crowded."

[0611] 2. Data Preprocessing:

[0612] Server: Text cleans the collected data, removing unnecessary characters, tags, and duplicate data, as well as filtering out inappropriate content.

[0613] Example: Remove emojis and HTML tags to generate clean text data.

[0614] 3. Data Summary:

[0615] Server: Feeds pre-processed data into a generative AI model (GPT-4) to summarize key topics, trends, customer requests and complaints.

[0616] Example prompt sentence:

[0617] Summarize the following customer feedback:

[0618] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[0619] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[0620] 4. Report Generation:

[0621] Server: Based on the summarized information, the server visualizes store performance and customer opinions, automatically generating reports with charts and statistics, which are saved in PDF and HTML formats.

[0622] Example: The report includes the advantages and disadvantages of the product or service, the items most mentioned by customers, and suggestions for improvement.

[0623] 5. Report Delivery:

[0624] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback with staff using the notification system.

[0625] Example: Every day, an updated report is emailed to the store manager's smartphone and also notified via an internal notification system.

[0626] This enables efficient collection and analysis of customer feedback and rapid service improvement.

[0627] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0628] Program processing steps

[0629] Step 1: Data collection

[0630] input:

[0631] Customer feedback input from in-store smart devices

[0632] External posting data from SNS API

[0633] Specific behavior:

[0634] Terminals: Smart devices (smartphones, tablets, smart glasses) installed in the store collect customer feedback and send it to the server.

[0635] Server: Uses APIs to collect store-related post data from social media and external data sources (e.g., Instagram API and Twitter API) and stores it in temporary storage.

[0636] output:

[0637] Feedback data and SNS posting data stored in temporary storage

[0638] Step 2: Data Preprocessing

[0639] input:

[0640] Feedback data and SNS posting data stored in temporary storage

[0641] Specific behavior:

[0642] Server: Cleans the collected data, removing unnecessary characters, tags, and duplicate data, and filters out inappropriate content, using regular expressions (RegEx) and text analysis tools.

[0643] output:

[0644] Clean text data

[0645] Step 3: Summarize the data

[0646] input:

[0647] Clean text data

[0648] Specific behavior:

[0649] Server: Inputs the preprocessed data into a generative AI model (GPT-4) to summarize important information and trends, and provides appropriate instructions to the model using prompts.

[0650] Example prompt sentence:

[0651] Summarize the following customer feedback:

[0652] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[0653] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[0654] output:

[0655] Summarized feedback information

[0656] Step 4: Generate a report

[0657] input:

[0658] Summarized feedback information

[0659] Specific behavior:

[0660] Server: Based on the summarized information, the server automatically generates reports containing charts and statistics that visualize store performance and customer opinions. Reports are saved in PDF and HTML formats.

[0661] output:

[0662] PDF and HTML reports

[0663] Step 5: Report Delivery

[0664] input:

[0665] PDF and HTML reports

[0666] Specific behavior:

[0667] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback using an internal notification system.

[0668] output:

[0669] Reports and notifications sent to store managers and staff's smart devices

[0670] By implementing each processing step in this manner, it becomes possible to efficiently collect and analyze customer feedback and quickly improve services.

[0671] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0672] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, preprocesses it, summarizes it using a generative AI model, and then recognizes user emotions using an emotion engine. It then automatically generates a report and delivers it to service personnel. This allows for a deeper understanding of customer feedback, and enables efficient collection, analysis, and reporting at low cost.

[0673] System Overview

[0674] This system is centered around a server and includes the following main functions:

[0675] 1. Collecting data from external sources

[0676] The server collects customer feedback data from external sources such as app stores, social media, and customer support systems using APIs or scraping techniques. For example, Twitter APIs can be used to automatically retrieve product posts.

[0677] 2. Data Preprocessing

[0678] The server preprocesses the collected customer feedback data. Preprocessing includes text cleaning, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[0679] 3. Data Summarization

[0680] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[0681] 4. Emotion Recognition by Emotion Engine

[0682] The server inputs the summarized data into an emotion engine to analyze the user's emotions. The emotion engine uses natural language processing technology to recognize the customer's emotions (e.g., joy, anger, sadness, etc.) from the text data.

[0683] 5. Automatic report generation

[0684] The server automatically generates reports based on the summarized data and the results of sentiment analysis by the sentiment engine. The reports include a summary of customer feedback along with the results of the sentiment analysis. A customized report template is used depending on the needs of each service. For example, a report in PDF format is generated that categorizes the summarized customer feedback into product advantages and disadvantages and includes graphs and statistics that reflect the sentiment analysis results.

[0685] 6. Report Distribution

[0686] The server automatically delivers the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[0687] Specific examples

[0688] A specific example of this system is shown below.

[0689] Voice of the Customer Data Collection Example

[0690] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[0691] Data Preprocessing Example

[0692] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[0693] Data Summarization Example

[0694] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[0695] Emotion Recognition Example

[0696] The server inputs the data summarized by the generative AI model into an emotion engine to recognize customer emotions, for example, extracting "joy" from the text "This product is very good."

[0697] Report Generation Example

[0698] The server automatically generates a detailed product report based on the summarized information and emotion recognition results, in PDF format, including charts and graphs.

[0699] Report Delivery Example

[0700] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[0701] In this way, this system efficiently collects and analyzes customer feedback, understands customer emotions through sentiment analysis, and promptly provides reports, thereby supporting service improvements and rapid response.

[0702] The processing flow will be explained below.

[0703] Step 1:

[0704] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[0705] Step 2:

[0706] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[0707] Step 3:

[0708] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[0709] Step 4:

[0710] The server cleans the raw data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and profanity) from the text data, for example by using regular expressions to remove emojis and tags.

[0711] Step 5:

[0712] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, improving the accuracy of the analysis. For example, it generates hash values ​​to remove duplicate posts and combines data with the same hash value.

[0713] Step 6:

[0714] The server initializes and configures the generative AI model. The generative AI model (e.g., GPT-4) uses specific settings and prompts for summarizing the data, such as the length of the summary or the priority of certain keywords.

[0715] Step 7:

[0716] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates concise summaries, such as identifying the characteristics and issues frequently mentioned by customers.

[0717] Step 8:

[0718] The server inputs the data summarized by the generative AI model into the emotion engine to recognize the user's emotions. The emotion engine uses natural language processing technology to analyze customer emotions (e.g., joy, anger, sadness, etc.) from the text data. For example, it extracts "joy" from the text "This product is very good."

[0719] Step 9:

[0720] The server automatically generates reports based on the summarized data and the sentiment analysis results from the sentiment engine. The server then inserts the summarized information and sentiment analysis results into pre-configured report templates to create customized reports tailored to the needs of each service. For example, the server visually displays the sentiment analysis results as charts and graphs.

[0721] Step 10:

[0722] The server saves the generated report and renders the document in a specified format, such as PDF or HTML. For example, the report can be saved in PDF format and made available as an email attachment.

[0723] Step 11:

[0724] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report to, for example, obtains the latest email address from the list of representatives.

[0725] Step 12:

[0726] The server adds a report summary to the body of the email and attaches the generated report document, giving a concise understanding of the email's contents. For example, it might insert a simple message such as "This week's customer feedback report is attached."

[0727] Step 13:

[0728] The server sends emails to the relevant personnel and sends notifications to an internal messaging system, so that all relevant personnel receive the latest report information, for example, by sending notifications to an internal chat tool.

[0729] Example 2

[0730] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0731] There is a need to collect, analyze, and report customer opinions effectively and efficiently, but traditional methods require a great deal of time and effort to process large amounts of data. It is also difficult to accurately grasp customer sentiment, which can delay service improvements and prompt responses. This can lead to lower customer satisfaction and a loss of trust in companies.

[0732] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0733] In this invention, the server includes means for automatically collecting customer opinion data from an external data source, means for preprocessing the collected customer opinion data, means for generating an AI model that summarizes the preprocessed customer opinion data, means for analyzing the summarized data and recognizing customer emotions, means for automatically generating a report based on the summarized data and the emotion recognition results, and means for delivering the generated report to a person in charge. This makes it possible to efficiently collect and analyze customer opinions, accurately grasp customer emotions, and quickly provide them as a report.

[0734] "External Data Sources" refers to data available from publicly available databases on the internet, social media platforms, online stores, and other digital environments.

[0735] "Customer Opinion Data" means customer ratings, feedback, reviews, comments, and other textual opinions about products and services.

[0736] "Automated collection means" refers to technology designed to collect data continuously, using programmable means, without human intervention.

[0737] "Preprocessing means" refers to technologies used to process collected data, such as by cleaning text, removing duplicate data, and filtering inappropriate content, before analyzing the data.

[0738] "Generative AI model means" refers to a system or program that uses artificial intelligence technology to extract important information from input data and generate a summary statement.

[0739] "Means for recognizing emotions" refers to technology that uses natural language processing technology to analyze customer emotions from text data and assign specific emotional labels (e.g., joy, anger, sadness, etc.).

[0740] "Means for automatically generating reports" refers to a system or program that automatically generates reports based on summarized data and emotion recognition results using predefined templates.

[0741] "Delivery means" refers to a system or program for sending generated reports to appropriate personnel via email or internal messaging systems.

[0742] A specific embodiment for implementing this invention will be described below. This system automatically collects customer opinion data from external data sources, preprocesses it, summarizes it using a generative AI model, recognizes customer emotions through an emotion engine, and automatically generates a report based on the results and delivers it to the person in charge.

[0743] System Overview

[0744] This system is centered around the server and includes the following main functions:

[0745] Data collection from external data sources

[0746] The server uses databases and social media platforms publicly available on the Internet as external data sources. Specific examples include collecting data using the Twitter API and Facebook API. For example, every morning at 2:00, the server automatically retrieves tweets related to "product name" via the Twitter API and stores them in temporary storage.

[0747] Data Preprocessing

[0748] The server pre-processes the collected data, which includes text cleaning (removing unnecessary characters and tags), removing duplicate data, and filtering inappropriate content. Regular expressions and filtering algorithms are used to remove emojis and HTML tags, and then the clean data is generated.

[0749] Data Summarization

[0750] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to extract and summarize important information. For example, the server passes text data to a GPT-4 model and receives text summarizing key topics and customer opinions as output.

[0751] emotion recognition

[0752] The server inputs the summarized data into an emotion engine to analyze customer emotions. The emotion engine uses natural language processing technology to assign emotion labels such as "happiness," "anger," and "sadness" to the text. For example, it extracts "happiness" from the text "This product is very good."

[0753] Report Generation

[0754] The server automatically generates a report based on the summarized data and emotion recognition results. The report is created using a template and includes product advantages and disadvantages, as well as customer sentiment analysis in the form of charts and graphs. The generated report is saved in PDF or HTML format.

[0755] Report Distribution

[0756] The server automatically delivers the generated report to the person in charge by attaching the report to an email using the SMTP protocol and sending it to a specified email address. For example, the server could send the latest report to the person in charge's mailbox every Monday.

[0757] Specific examples

[0758] A specific example of this system is shown below.

[0759] Customer Opinion Data Collection Example

[0760] Every morning at 2am, the server uses the Facebook API to collect posts about a specific product and stores them in temporary storage.

[0761] Specific examples of data preprocessing

[0762] The server then removes unnecessary characters and tags from the collected posts, filters out duplicates and inappropriate content, and uses regular expressions and filtering algorithms to generate a clean dataset.

[0763] Examples of data summarization

[0764] The server feeds the preprocessed data into a GPT-4 model, which generates text summarizing key topics and trends.

[0765] Specific examples of emotion recognition

[0766] The server inputs the summaries generated by the generative AI model into an emotion engine to analyze customer emotions, for example, extracting the emotion "joy" from the text "This product is very good."

[0767] Example of report generation

[0768] The server automatically generates a detailed report based on the summarized data and emotion recognition results, which is saved in PDF format and includes charts and graphs.

[0769] Example of report distribution

[0770] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[0771] This format makes it possible to efficiently collect customer opinions, conduct detailed analysis, and quickly provide the results in report format.

[0772] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0773] Step 1: Collect data from external data sources

[0774] The server collects customer opinion data from external data sources. Specifically, every morning at 2:00 AM, the server retrieves posts related to specific keywords or hashtags using the Twitter API or Facebook API. The collected data is stored in temporary storage in JSON format.

[0775] Input: External data source (Twitter API, Facebook API, etc.)

[0776] Data processing: API requests, data acquisition, conversion to JSON format

[0777] Output: JSON format data stored in temporary storage

[0778] Step 2: Preprocessing the data

[0779] The server preprocesses the collected data. First, it reads the JSON data and removes unnecessary characters and tags (e.g., emojis, HTML tags). Second, it detects and removes duplicate data. Third, it filters out inappropriate content.

[0780] Input: JSON format data stored in temporary storage

[0781] Data operations: text cleaning, de-duplicating data, filtering

[0782] Output: A preprocessed, clean dataset

[0783] Step 3: Summarize the data

[0784] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate a summary. Specifically, the server passes the clean text data to GPT-4 and receives a summary that extracts important topics and claims.

[0785] Input: Preprocessed clean dataset

[0786] Data calculation: Extraction of important information and generation of summaries using generative AI models

[0787] Output: Generated summary

[0788] Step 4: Emotion Recognition

[0789] The server inputs the summary sentence into the emotion engine for emotion analysis. The emotion engine uses natural language processing technology to identify the customer's emotion (e.g., joy, anger, sadness) from the text.

[0790] Input: Generated summary

[0791] Data Computing: Sentiment Analysis with Natural Language Processing

[0792] Output: Sentiment analysis results with emotion labels

[0793] Step 5: Generate a report

[0794] The server automatically generates a report based on the summary and the results of the sentiment analysis. The report is created based on a pre-prepared template and includes the summary and the results of the sentiment analysis as charts and graphs. The generated report is saved in PDF or HTML format.

[0795] Input: Summary and sentiment analysis results

[0796] Data calculation: Insert data into templates, automatically generate reports

[0797] Output: Report in PDF or HTML format

[0798] Step 6: Report Distribution

[0799] The server automatically delivers the generated report to the person in charge by attaching the report to an email and sending it to the person's email address. It also notifies the person in charge via the internal messaging system.

[0800] Input: Report in PDF or HTML format

[0801] Data calculation: Email composition and sending using the SMTP protocol

[0802] Output: Reports sent to the assignee's mailbox and notifications in the internal messaging system

[0803] Through these steps, this system is able to efficiently collect and analyze customer opinion data and quickly provide it as a report.

[0804] (Application example 2)

[0805] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0806] Autonomous vehicles require efficient collection and analysis of passenger feedback and the ability to improve vehicle performance and user experience based on that feedback. However, traditional methods often require manual collection of feedback, and data analysis and emotion recognition are ineffective, resulting in limited real-time performance and accuracy. Additionally, there is a lack of a way to quickly share reports generated based on feedback with relevant parties. This can delay the implementation of prompt improvements based on feedback.

[0807] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automatically collecting customer voice data from an external data source, means for preprocessing the collected customer voice data, generative AI model means for summarizing the preprocessed customer voice data, emotion engine means for analyzing emotions based on the summarized data, means for automatically generating a report based on the analyzed data, and means for distributing the generated report to a vehicle manager or developer of the autonomous vehicle. This makes it possible to collect and analyze passenger feedback in real time and quickly distribute the results to relevant parties.

[0808] "External data sources" are sources of data obtained from outside sources such as social media, customer support systems, app stores, etc.

[0809] "Voice of Customer Data" is text data that includes feedback, opinions, and ratings expressed by customers about products and services.

[0810] "Preprocessing" is a series of data cleansing steps that remove unnecessary characters and tags from raw data and filter out duplicate data and inappropriate content.

[0811] A "generative AI model" is an artificial intelligence model used to extract and summarize key topics and trends from large amounts of data, such as GPT-4.

[0812] An "emotion engine" is an engine that uses natural language processing technology to analyze customer emotions (joy, anger, sadness, etc.) from text data.

[0813] A "report" is a document generated based on summarized voice of customer data and sentiment analysis results, and may include graphs and statistics.

[0814] "Vehicle managers and developers" are people or positions involved in the operation, maintenance, and development of autonomous vehicles who implement improvement measures based on feedback data.

[0815] System Overview

[0816] A system for implementing this invention automatically collects customer voice-of-mouth data from external data sources, summarizes it using a generative AI model and an emotion engine, and performs sentiment analysis. It then automatically generates reports based on the analysis results and distributes them to fleet managers and developers. This allows feedback on autonomous vehicles to be efficiently collected and analyzed and quickly shared with relevant parties.

[0817] Hardware and software used

[0818] 1. Hardware:

[0819] Server (for data collection and processing)

[0820] Smartphone (for collecting feedback)

[0821] 2. Software:

[0822] Requests: An HTTP request library for gathering data from external data sources.

[0823] OpenAI API: Generate summaries of data using generative AI models (e.g., GPT-4)

[0824] EmotionEngine: A natural language processing engine for emotion analysis

[0825] ReportLab: A library for saving automatically generated reports in PDF format.

[0826] Data processing and calculation

[0827] 1. Data Collection

[0828] The server automatically collects customer feedback data from external data sources (e.g., social media, customer support systems) via APIs, for example, by sending an HTTP request to an API endpoint and temporarily storing the retrieved data.

[0829] 2. Data Preprocessing

[0830] The collected customer feedback data is preprocessed by the server, which removes unnecessary characters and emojis, deletes HTML tags, and filters out duplicate data to generate clean text data.

[0831] 3. Data Summary

[0832] The preprocessed text data is then fed into a generative AI model (e.g., GPT-4) to summarize the key information. Generative AI models extract the most important topics and trends from the vast amount of data.

[0833] 4. Emotion analysis

[0834] The summarized text data is input to an emotion engine, which analyzes the customer's emotions (e.g., joy, anger, sadness, etc.). The emotion analysis results show the distribution of emotions for the customer's feedback.

[0835] 5. Report Generation and Delivery

[0836] Based on the analyzed data, the server automatically generates a report in PDF format. The report includes summary information, sentiment analysis results, graphs, and statistical information. The generated report is automatically distributed to vehicle managers and developers. For example, the report can be sent by email every week.

[0837] Specific examples

[0838] For example, suppose a passenger posts feedback such as, "This car is very comfortable, but the air conditioning doesn't work well." This feedback can be summarized as, "The vehicle is comfortable, but I'm dissatisfied with the air conditioning performance," and sentiment analysis can extract "happiness" and "dissatisfaction."

[0839] Prompt Sentence Examples

[0840] "Please summarize the following text:

[0841] This car is very comfortable, but the air conditioning doesn't work well.

[0842] By using the above system, customer feedback can be efficiently collected and analyzed, and used to improve the quality of autonomous vehicles.

[0843] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0844] Step 1:

[0845] Data collection

[0846] The server automatically collects customer feedback data from external data sources (e.g., social media or customer support systems). Specifically, the server uses an API to send an HTTP request and temporarily stores the data obtained as an API response. For example, a request is made to an API endpoint of a social media service to obtain post data containing keywords such as "vehicle," "comfort," and "air conditioning." This data is often stored in JSON format. The input is raw data obtained from the external data source, and the output is unprocessed data stored in temporary storage.

[0847] Step 2:

[0848] Data Preprocessing

[0849] The collected customer voice data is pre-processed by the server. Specifically, it removes inappropriate characters (e.g., emojis, HTML tags) and duplicate data, and normalizes the text. For example, "This car is very comfortable." The feedback "This car is very comfortable, but the air conditioning doesn't work well 🎉" is transformed into "This car is very comfortable, but the air conditioning doesn't work well." The input is raw data, and the output is clean text data.

[0850] Step 3:

[0851] Data Summary

[0852] The preprocessed customer voice data is input into a generative AI model by the server. This AI model summarizes the text using, for example, GPT-4. Specifically, it extracts key topics and trends from large amounts of feedback data and outputs them as summarized text. The prompt includes the instruction "Please summarize the following text:\n[text data]". The input is clean text data, and the output is summarized text.

[0853] Step 4:

[0854] Emotion analysis

[0855] The server inputs the summarized text data into an emotion engine, which analyzes the customer's emotions. Specifically, the data is classified into emotion categories such as "happiness," "anger," and "sadness," and a determination is made as to which emotion the text best expresses. For example, "happiness" and "dissatisfaction" are extracted from the text "This car is very comfortable, but the air conditioning doesn't work well." The input is the summarized text, and the output is the emotion analysis results.

[0856] Step 5:

[0857] Report Generation

[0858] Based on the analysis results, the server automatically generates a report. Specifically, it uses a library such as ReportLab to create a PDF report. The report includes summarized text, sentiment analysis results, graphs, and statistical information. For example, the report includes the original feedback in the "Product Benefits" section. The input is the summarized data and sentiment analysis results, and the output is a completed report file.

[0859] Step 6:

[0860] Report Distribution

[0861] The generated reports are automatically distributed by the server to vehicle managers and developers. Specifically, reports are sent to relevant parties using email or an internal messaging system. For example, the latest report is automatically sent at a fixed date and time each week. The input is a completed report file, and the output is a report delivered to the relevant parties' mailboxes.

[0862] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0863] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0864] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0865] [Third embodiment]

[0866] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0867] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0868] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0869] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0870] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0871] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0872] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0873] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0874] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0875] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0876] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0877] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0878] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, performs preprocessing, summarizes it using a generative AI model, automatically generates a report based on the summarized data, and finally delivers the report to service personnel. This allows for the collection, analysis, and reporting of customer feedback at low cost and efficiently.

[0879] System Overview

[0880] This system is centered around a server and includes the following main functions:

[0881] 1. Collecting data from external sources

[0882] The server collects customer feedback data from external data sources such as app stores, social media, customer support systems, etc. This collection can be done using APIs or scraping techniques, for example, using the Twitter API to automatically retrieve posts about the product.

[0883] 2. Data Preprocessing

[0884] The server preprocesses the collected customer feedback data. Preprocessing includes text cleaning, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[0885] 3. Data Summarization

[0886] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[0887] 4. Automatic report generation

[0888] The server automatically generates reports based on the summarized data, using report templates customized to the needs of each service. For example, it could categorize summarized customer testimonials into product advantages and disadvantages and generate a PDF report with graphs and statistics.

[0889] 5. Report Distribution

[0890] The server automatically delivers the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[0891] Specific examples

[0892] A specific example of this system is shown below.

[0893] Voice of the Customer Data Collection Example

[0894] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[0895] Data Preprocessing Example

[0896] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[0897] Data Summarization Example

[0898] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[0899] Report Generation Example

[0900] The server automatically generates a detailed product report based on the summarized information, in PDF format, including charts and graphs.

[0901] Report Delivery Example

[0902] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[0903] In this way, this system efficiently collects and analyzes customer feedback and promptly provides reports, thereby supporting service improvements and rapid response.

[0904] The processing flow will be explained below.

[0905] Step 1:

[0906] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[0907] Step 2:

[0908] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[0909] Step 3:

[0910] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[0911] Step 4:

[0912] The server cleans the text data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and inappropriate language) from the raw data.

[0913] Step 5:

[0914] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, which improves the accuracy of the analysis.

[0915] Step 6:

[0916] The server initializes and configures the generative AI model, which (e.g., GPT-4) uses specific configuration values ​​and prompts to summarize the data.

[0917] Step 7:

[0918] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates a concise summary.

[0919] Step 8:

[0920] The server automatically generates reports based on the summarized data, inserting the summarized information into pre-configured report templates to create customized reports tailored to the needs of each service.

[0921] Step 9:

[0922] The server saves the generated report and formats the document in the specified format, such as PDF or HTML.

[0923] Step 10:

[0924] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report.

[0925] Step 11:

[0926] The server adds a report summary to the body of the email and attaches the generated report document, allowing you to understand the contents of the email concisely.

[0927] Step 12:

[0928] The server sends emails to personnel and sends notifications to an internal messaging system, ensuring that all relevant personnel receive the latest report information.

[0929] Example 1

[0930] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0931] Traditional methods for collecting, analyzing, and reporting customer feedback data required a great deal of time and effort, making it difficult to efficiently collect and analyze information. Furthermore, because much of the processing was manual, it was difficult to maintain data accuracy and consistency. Traditional methods were particularly difficult to use when large amounts of data needed to be collected and analyzed in a short amount of time.

[0932] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0933] In this invention, the server includes means for automatically collecting customer voice data from external data sources, means for preprocessing the collected customer voice data, a generative AI model means for summarizing the preprocessed customer voice data, means for instructing the generative AI model to perform summarization using a prompt sentence, means for automatically generating a report in PDF format based on the summarized data, and means for delivering the generated report to a service representative via email or an internal message system. This makes it possible to efficiently and accurately collect and analyze customer voice data and quickly provide information to a service representative.

[0934] "External Data Sources" refers to sources of data obtained from various platforms and services on the Internet.

[0935] "Customer voice data" refers to text data such as opinions, impressions, and evaluations expressed by users about products and services.

[0936] "Preprocessing" refers to the process of removing unnecessary information from collected data and preparing it in a format suitable for analysis.

[0937] A "generative AI model" refers to an artificial intelligence program that learns from large amounts of data and performs natural language processing and text generation.

[0938] A "prompt sentence" refers to input text that instructs a generative AI model on specific actions and processing details.

[0939] "PDF format" is an abbreviation for Portable Document Format, and refers to a file format for electronically storing and displaying documents and image information.

[0940] "Mail" is an abbreviation for electronic mail and refers to a means of communication for exchanging text and files over the Internet.

[0941] An "internal messaging system" refers to a dedicated communication system for exchanging information and messages within an organization.

[0942] "Service representative" refers to a person who handles customer service and support tasks.

[0943] "API" stands for Application Programming Interface and refers to an interface for exchanging data and functions between different software applications.

[0944] This section describes an embodiment of the present invention. This system is primarily comprised of a server and has the functions of collecting customer feedback data from external data sources, preprocessing and summarizing it, and automatically generating and distributing reports. This system makes it possible to efficiently collect, analyze, and report customer feedback.

[0945] System configuration

[0946] The system consists of the following major hardware and software components:

[0947] Server: The central unit that processes and analyzes data. A general-purpose server equipped with a high-performance CPU and large memory capacity is used.

[0948] Network Interface: A network connection for communicating with external data sources.

[0949] Generative AI models: Generative AI models such as GPT-4 are used as models for natural language processing.

[0950] API: Used to retrieve data from external data sources. Examples include the Facebook API and Twitter API.

[0951] Mail Server: Email sending facility for delivering reports to service personnel.

[0952] Internal messaging system: A dedicated communication system for sharing information within an organization.

[0953] Data collection

[0954] The server runs a scheduled task every morning at 2:00 AM to collect customer feedback data from external data sources using APIs. For example, it uses the Facebook API to collect posts related to "New Product X." By sending an API request, data that matches the specified criteria is retrieved.

[0955] Data Preprocessing

[0956] The collected data is pre-processed on the server, which includes removing unnecessary characters and tags, filtering duplicate data, and eliminating inappropriate content. This process provides a clean dataset suitable for analysis.

[0957] Data Summarization

[0958] The preprocessed data is then input into a generative AI model, which uses prompts to provide specific summarization instructions to the model. For example, a prompt might be, "Summarize the main customer complaints about new product X." The generative AI model then extracts key information from the input data and provides a summary result.

[0959] Automatic report generation

[0960] The server automatically generates a PDF report based on the summarized data, including graphs and statistics based on the summary content, and formats the report according to an automatically generated template.

[0961] Report distribution

[0962] The generated reports are delivered from the server to service personnel via email from the mail server and via the internal messaging system, allowing personnel to receive timely feedback from customers.

[0963] Specific examples

[0964] For example, the server collects posts about "New Product X" using the Facebook API at 2:00 a.m. every morning and temporarily stores them in a database. It then performs data preprocessing and summarizes them by inputting a prompt to the generative AI model: "Summarize the main customer complaints about New Product X." Finally, it generates a PDF report based on the summary results and sends it to the service representative via email.

[0965] In this way, the system automates the entire process from data collection to report distribution, enabling efficient analysis and reporting of customer feedback.

[0966] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0967] Step 1:

[0968] Data collection

[0969] The server runs a scheduled task every morning at 2am, using APIs to collect voice of customer data from external data sources.

[0970] Input: Facebook API key, search query (e.g. "New Product X").

[0971] Specific operation: The server sends an API request to retrieve data based on the specified conditions, and the retrieved data is temporarily stored in a database.

[0972] Output: The raw data collected (e.g. Facebook posts).

[0973] Step 2:

[0974] Data Preprocessing

[0975] The server pre-processes the collected raw data.

[0976] Input: Raw data collected.

[0977] What it does: The server uses regular expressions to remove unwanted characters and tags, and also filters out duplicate data and spam posts.

[0978] Output: A clean dataset.

[0979] Step 3:

[0980] Data Summarization

[0981] The server inputs the preprocessed data into a generative AI model to generate summaries.

[0982] Input: A clean dataset, a prompt statement (e.g., "Summarize the main customer complaints about new product X").

[0983] Specific operation: The server sends prompt sentences to the generative AI model (GPT-4) to generate a summary of the data.

[0984] Output: Summarized text data.

[0985] Step 4:

[0986] Automatic report generation

[0987] The server automatically generates a PDF report based on the summarized data.

[0988] Input: Abstracted text data.

[0989] What it does: The server embeds summary information into a report template, adds graphs and statistics, and generates a PDF report.

[0990] Output: Generated PDF report.

[0991] Step 5:

[0992] Report distribution

[0993] The server delivers the generated report to the service representative.

[0994] Input: PDF report, service representative's email address or internal messaging system user ID.

[0995] What it does: The server uses the SMTP library to send emails and notify through its internal messaging system.

[0996] Output: The delivered report.

[0997] (Application example 1)

[0998] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0999] In today's highly competitive market, brick-and-mortar stores need to quickly and efficiently collect customer feedback and use it to improve their services. Traditional feedback collection methods are manual, time-consuming, and labor-intensive, and analyzing data and generating reports is difficult. In particular, there is a need for an integrated system that utilizes smart devices such as smartphones, tablets, and smart glasses to efficiently collect feedback and effectively analyze it.

[1000] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1001] In this invention, the server includes means for automatically collecting customer feedback data from external data sources, means for preprocessing the collected customer feedback data, means for generating an AI model that summarizes the preprocessed customer feedback data, means for automatically generating a report based on the summarized data, means for distributing the generated report to a service representative, means for collecting customer feedback via a smart device, and means for automatically sending the generated report to a store manager or staff member, thereby enabling efficient collection and analysis of customer feedback and rapid service improvement.

[1002] Definitions of important words

[1003] "External data sources" are external data providers from which customer feedback data can be obtained, such as app stores, social media, and customer support systems.

[1004] "Voice of customer data" refers to text data that includes feedback, opinions, and impressions provided by customers regarding products and services.

[1005] "Preprocessing" refers to a series of processes that remove unnecessary characters and tags from collected text data, delete duplicates, and filter inappropriate content.

[1006] A "generative AI model" is an artificial intelligence model used to extract and summarize important information from collected and pre-processed data. A specific example is GPT-4.

[1007] A "report" is a document generated from summarized data that visualizes and presents product or service performance, trends, and areas for improvement.

[1008] "Smart devices" are portable electronic devices with advanced functions, such as smartphones, tablets, and smart glasses.

[1009] "Store Manager" means the person responsible for the operation and management of the physical store.

[1010] "Staff" refers to employees who deal with customers and perform other duties at the store.

[1011] "API" stands for Application Programming Interface, a standardized means of exchanging data and functions between different software programs.

[1012] "Automatic generation" is a process in which a system automatically processes and generates results without human intervention.

[1013] "Auto Send" is a feature that automatically sends data such as reports generated by the system to designated recipients.

[1014] MODE FOR CARRYING OUT THE INVENTION

[1015] An embodiment of the present invention will now be described. This system is centered around a server, and efficiently collects, analyzes, and generates reports on customer feedback in physical stores via smart devices. This allows store managers and staff to understand customer feedback in real time and respond quickly.

[1016] System Overview

[1017] Hardware and software used

[1018] Hardware:

[1019] Smartphone

[1020] tablet

[1021] Smart Glasses

[1022] software:

[1023] iOS / Android app

[1024] Smart Glasses App

[1025] Server (Amazon Web Services, etc.)

[1026] Generative AI model (GPT-4)

[1027] Program processing description

[1028] 1. Data Collection:

[1029] Terminal: Smart devices (smartphones, tablets, smart glasses) installed in the store provide a feedback input form for customers. The feedback data entered by customers is sent to the server. In addition, external posting data about the store is collected using SNS APIs (e.g., Instagram API and Twitter API).

[1030] Example: Customers can enter feedback via in-store tablets, such as:

[1031] "The staff were amazing! I wish they'd hire more staff so the cashiers don't get too crowded."

[1032] 2. Data Preprocessing:

[1033] Server: Text cleans the collected data, removing unnecessary characters, tags, and duplicate data, as well as filtering out inappropriate content.

[1034] Example: Remove emojis and HTML tags to generate clean text data.

[1035] 3. Data Summary:

[1036] Server: Feeds pre-processed data into a generative AI model (GPT-4) to summarize key topics, trends, customer requests and complaints.

[1037] Example prompt sentence:

[1038] Summarize the following customer feedback:

[1039] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[1040] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[1041] 4. Report Generation:

[1042] Server: Based on the summarized information, the server visualizes store performance and customer opinions, automatically generating reports with charts and statistics, which are saved in PDF and HTML formats.

[1043] Example: The report includes the advantages and disadvantages of the product or service, the items most mentioned by customers, and suggestions for improvement.

[1044] 5. Report Delivery:

[1045] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback with staff using the notification system.

[1046] Example: Every day, an updated report is emailed to the store manager's smartphone and also notified via an internal notification system.

[1047] This enables efficient collection and analysis of customer feedback and rapid service improvement.

[1048] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1049] Program processing steps

[1050] Step 1: Data collection

[1051] input:

[1052] Customer feedback input from in-store smart devices

[1053] External posting data from SNS API

[1054] Specific behavior:

[1055] Terminals: Smart devices (smartphones, tablets, smart glasses) installed in the store collect customer feedback and send it to the server.

[1056] Server: Uses APIs to collect store-related post data from social media and external data sources (e.g., Instagram API and Twitter API) and stores it in temporary storage.

[1057] output:

[1058] Feedback data and SNS posting data stored in temporary storage

[1059] Step 2: Data Preprocessing

[1060] input:

[1061] Feedback data and SNS posting data stored in temporary storage

[1062] Specific behavior:

[1063] Server: Cleans the collected data, removing unnecessary characters, tags, and duplicate data, and filters out inappropriate content, using regular expressions (RegEx) and text analysis tools.

[1064] output:

[1065] Clean text data

[1066] Step 3: Summarize the data

[1067] input:

[1068] Clean text data

[1069] Specific behavior:

[1070] Server: Inputs the preprocessed data into a generative AI model (GPT-4) to summarize important information and trends, and provides appropriate instructions to the model using prompts.

[1071] Example prompt sentence:

[1072] Summarize the following customer feedback:

[1073] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[1074] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[1075] output:

[1076] Summarized feedback information

[1077] Step 4: Generate a report

[1078] input:

[1079] Summarized feedback information

[1080] Specific behavior:

[1081] Server: Based on the summarized information, the server automatically generates reports containing charts and statistics that visualize store performance and customer opinions. Reports are saved in PDF and HTML formats.

[1082] output:

[1083] PDF and HTML reports

[1084] Step 5: Report Delivery

[1085] input:

[1086] PDF and HTML reports

[1087] Specific behavior:

[1088] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback using an internal notification system.

[1089] output:

[1090] Reports and notifications sent to store managers and staff's smart devices

[1091] By implementing each processing step in this manner, it becomes possible to efficiently collect and analyze customer feedback and quickly improve services.

[1092] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1093] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, preprocesses it, summarizes it using a generative AI model, and then recognizes user emotions using an emotion engine. It then automatically generates a report and delivers it to service personnel. This allows for a deeper understanding of customer feedback, and enables efficient collection, analysis, and reporting at low cost.

[1094] System Overview

[1095] This system is centered around a server and includes the following main functions:

[1096] 1. Collecting data from external sources

[1097] The server collects customer feedback data from external sources such as app stores, social media, and customer support systems using APIs or scraping techniques. For example, Twitter APIs can be used to automatically retrieve product posts.

[1098] 2. Data Preprocessing

[1099] The server preprocesses the collected customer feedback data. Preprocessing includes text cleaning, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[1100] 3. Data Summarization

[1101] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[1102] 4. Emotion Recognition by Emotion Engine

[1103] The server inputs the summarized data into an emotion engine to analyze the user's emotions. The emotion engine uses natural language processing technology to recognize the customer's emotions (e.g., joy, anger, sadness, etc.) from the text data.

[1104] 5. Automatic report generation

[1105] The server automatically generates reports based on the summarized data and the results of sentiment analysis by the sentiment engine. The reports include a summary of customer feedback along with the results of the sentiment analysis. A customized report template is used depending on the needs of each service. For example, a report in PDF format is generated that categorizes the summarized customer feedback into product advantages and disadvantages and includes graphs and statistics that reflect the sentiment analysis results.

[1106] 6. Report Distribution

[1107] The server automatically delivers the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[1108] Specific examples

[1109] A specific example of this system is shown below.

[1110] Voice of the Customer Data Collection Example

[1111] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[1112] Data Preprocessing Example

[1113] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[1114] Data Summarization Example

[1115] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[1116] Emotion Recognition Example

[1117] The server inputs the data summarized by the generative AI model into an emotion engine to recognize customer emotions, for example, extracting "joy" from the text "This product is very good."

[1118] Report Generation Example

[1119] The server automatically generates a detailed product report based on the summarized information and emotion recognition results, in PDF format, including charts and graphs.

[1120] Report Delivery Example

[1121] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[1122] In this way, this system efficiently collects and analyzes customer feedback, understands customer emotions through sentiment analysis, and promptly provides reports, thereby supporting service improvements and rapid response.

[1123] The processing flow will be explained below.

[1124] Step 1:

[1125] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[1126] Step 2:

[1127] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[1128] Step 3:

[1129] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[1130] Step 4:

[1131] The server cleans the raw data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and profanity) from the text data, for example by using regular expressions to remove emojis and tags.

[1132] Step 5:

[1133] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, improving the accuracy of the analysis. For example, it generates hash values ​​to remove duplicate posts and combines data with the same hash value.

[1134] Step 6:

[1135] The server initializes and configures the generative AI model. The generative AI model (e.g., GPT-4) uses specific settings and prompts for summarizing the data, such as the length of the summary or the priority of certain keywords.

[1136] Step 7:

[1137] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates concise summaries, such as identifying the characteristics and issues frequently mentioned by customers.

[1138] Step 8:

[1139] The server inputs the data summarized by the generative AI model into the emotion engine to recognize the user's emotions. The emotion engine uses natural language processing technology to analyze customer emotions (e.g., joy, anger, sadness, etc.) from the text data. For example, it extracts "joy" from the text "This product is very good."

[1140] Step 9:

[1141] The server automatically generates reports based on the summarized data and the sentiment analysis results from the sentiment engine. The server then inserts the summarized information and sentiment analysis results into pre-configured report templates to create customized reports tailored to the needs of each service. For example, the server visually displays the sentiment analysis results as charts and graphs.

[1142] Step 10:

[1143] The server saves the generated report and renders the document in a specified format, such as PDF or HTML. For example, the report can be saved in PDF format and made available as an email attachment.

[1144] Step 11:

[1145] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report to, for example, obtains the latest email address from the list of representatives.

[1146] Step 12:

[1147] The server adds a report summary to the body of the email and attaches the generated report document, giving a concise understanding of the email's contents. For example, it might insert a simple message such as "This week's customer feedback report is attached."

[1148] Step 13:

[1149] The server sends emails to the relevant personnel and sends notifications to an internal messaging system, so that all relevant personnel receive the latest report information, for example, by sending notifications to an internal chat tool.

[1150] Example 2

[1151] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1152] There is a need to collect, analyze, and report customer opinions effectively and efficiently, but traditional methods require a great deal of time and effort to process large amounts of data. It is also difficult to accurately grasp customer sentiment, which can delay service improvements and prompt responses. This can lead to lower customer satisfaction and a loss of trust in companies.

[1153] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1154] In this invention, the server includes means for automatically collecting customer opinion data from an external data source, means for preprocessing the collected customer opinion data, means for generating an AI model that summarizes the preprocessed customer opinion data, means for analyzing the summarized data and recognizing customer emotions, means for automatically generating a report based on the summarized data and the emotion recognition results, and means for delivering the generated report to a person in charge. This makes it possible to efficiently collect and analyze customer opinions, accurately grasp customer emotions, and quickly provide them as a report.

[1155] "External Data Sources" refers to data available from publicly available databases on the internet, social media platforms, online stores, and other digital environments.

[1156] "Customer Opinion Data" means customer ratings, feedback, reviews, comments, and other textual opinions about products and services.

[1157] "Automated collection means" refers to technology designed to collect data continuously, using programmable means, without human intervention.

[1158] "Preprocessing means" refers to technologies used to process collected data, such as by cleaning text, removing duplicate data, and filtering inappropriate content, before analyzing the data.

[1159] "Generative AI model means" refers to a system or program that uses artificial intelligence technology to extract important information from input data and generate a summary statement.

[1160] "Means for recognizing emotions" refers to technology that uses natural language processing technology to analyze customer emotions from text data and assign specific emotional labels (e.g., joy, anger, sadness, etc.).

[1161] "Means for automatically generating reports" refers to a system or program that automatically generates reports based on summarized data and emotion recognition results using predefined templates.

[1162] "Delivery means" refers to a system or program for sending generated reports to appropriate personnel via email or internal messaging systems.

[1163] A specific embodiment for implementing this invention will be described below. This system automatically collects customer opinion data from external data sources, preprocesses it, summarizes it using a generative AI model, recognizes customer emotions through an emotion engine, and automatically generates a report based on the results and delivers it to the person in charge.

[1164] System Overview

[1165] This system is centered around the server and includes the following main functions:

[1166] Data collection from external data sources

[1167] The server uses databases and social media platforms publicly available on the Internet as external data sources. Specific examples include collecting data using the Twitter API and Facebook API. For example, every morning at 2:00, the server automatically retrieves tweets related to "product name" via the Twitter API and stores them in temporary storage.

[1168] Data Preprocessing

[1169] The server pre-processes the collected data, which includes text cleaning (removing unnecessary characters and tags), removing duplicate data, and filtering inappropriate content. Regular expressions and filtering algorithms are used to remove emojis and HTML tags, and then the clean data is generated.

[1170] Data Summarization

[1171] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to extract and summarize important information. For example, the server passes text data to a GPT-4 model and receives text summarizing key topics and customer opinions as output.

[1172] emotion recognition

[1173] The server inputs the summarized data into an emotion engine to analyze customer emotions. The emotion engine uses natural language processing technology to assign emotion labels such as "happiness," "anger," and "sadness" to the text. For example, it extracts "happiness" from the text "This product is very good."

[1174] Report Generation

[1175] The server automatically generates a report based on the summarized data and emotion recognition results. The report is created using a template and includes product advantages and disadvantages, as well as customer sentiment analysis in the form of charts and graphs. The generated report is saved in PDF or HTML format.

[1176] Report Distribution

[1177] The server automatically delivers the generated report to the person in charge by attaching the report to an email using the SMTP protocol and sending it to a specified email address. For example, the server could send the latest report to the person in charge's mailbox every Monday.

[1178] Specific examples

[1179] A specific example of this system is shown below.

[1180] Customer Opinion Data Collection Example

[1181] Every morning at 2am, the server uses the Facebook API to collect posts about a specific product and stores them in temporary storage.

[1182] Specific examples of data preprocessing

[1183] The server then removes unnecessary characters and tags from the collected posts, filters out duplicates and inappropriate content, and uses regular expressions and filtering algorithms to generate a clean dataset.

[1184] Examples of data summarization

[1185] The server feeds the preprocessed data into a GPT-4 model, which generates text summarizing key topics and trends.

[1186] Specific examples of emotion recognition

[1187] The server inputs the summaries generated by the generative AI model into an emotion engine to analyze customer emotions, for example, extracting the emotion "joy" from the text "This product is very good."

[1188] Example of report generation

[1189] The server automatically generates a detailed report based on the summarized data and emotion recognition results, which is saved in PDF format and includes charts and graphs.

[1190] Example of report distribution

[1191] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[1192] This format makes it possible to efficiently collect customer opinions, conduct detailed analysis, and quickly provide the results in report format.

[1193] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1194] Step 1: Collect data from external data sources

[1195] The server collects customer opinion data from external data sources. Specifically, every morning at 2:00 AM, the server retrieves posts related to specific keywords or hashtags using the Twitter API or Facebook API. The collected data is stored in temporary storage in JSON format.

[1196] Input: External data source (Twitter API, Facebook API, etc.)

[1197] Data processing: API requests, data acquisition, conversion to JSON format

[1198] Output: JSON format data stored in temporary storage

[1199] Step 2: Preprocessing the data

[1200] The server preprocesses the collected data. First, it reads the JSON data and removes unnecessary characters and tags (e.g., emojis, HTML tags). Second, it detects and removes duplicate data. Third, it filters out inappropriate content.

[1201] Input: JSON format data stored in temporary storage

[1202] Data operations: text cleaning, de-duplicating data, filtering

[1203] Output: A preprocessed, clean dataset

[1204] Step 3: Summarize the data

[1205] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate a summary. Specifically, the server passes the clean text data to GPT-4 and receives a summary that extracts important topics and claims.

[1206] Input: Preprocessed clean dataset

[1207] Data calculation: Extraction of important information and generation of summaries using generative AI models

[1208] Output: Generated summary

[1209] Step 4: Emotion Recognition

[1210] The server inputs the summary sentence into the emotion engine for emotion analysis. The emotion engine uses natural language processing technology to identify the customer's emotion (e.g., joy, anger, sadness) from the text.

[1211] Input: Generated summary

[1212] Data Computing: Sentiment Analysis with Natural Language Processing

[1213] Output: Sentiment analysis results with emotion labels

[1214] Step 5: Generate a report

[1215] The server automatically generates a report based on the summary and the results of the sentiment analysis. The report is created based on a pre-prepared template and includes the summary and the results of the sentiment analysis as charts and graphs. The generated report is saved in PDF or HTML format.

[1216] Input: Summary and sentiment analysis results

[1217] Data calculation: Insert data into templates, automatically generate reports

[1218] Output: Report in PDF or HTML format

[1219] Step 6: Report Distribution

[1220] The server automatically delivers the generated report to the person in charge by attaching the report to an email and sending it to the person's email address. It also notifies the person in charge via the internal messaging system.

[1221] Input: Report in PDF or HTML format

[1222] Data calculation: Email composition and sending using the SMTP protocol

[1223] Output: Reports sent to the assignee's mailbox and notifications in the internal messaging system

[1224] Through these steps, this system is able to efficiently collect and analyze customer opinion data and quickly provide it as a report.

[1225] (Application example 2)

[1226] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1227] Autonomous vehicles require efficient collection and analysis of passenger feedback and the ability to improve vehicle performance and user experience based on that feedback. However, traditional methods often require manual collection of feedback, and data analysis and emotion recognition are ineffective, resulting in limited real-time performance and accuracy. Additionally, there is a lack of a way to quickly share reports generated based on feedback with relevant parties. This can delay the implementation of prompt improvements based on feedback.

[1228] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automatically collecting customer voice data from an external data source, means for preprocessing the collected customer voice data, generative AI model means for summarizing the preprocessed customer voice data, emotion engine means for analyzing emotions based on the summarized data, means for automatically generating a report based on the analyzed data, and means for distributing the generated report to a vehicle manager or developer of the autonomous vehicle. This makes it possible to collect and analyze passenger feedback in real time and quickly distribute the results to relevant parties.

[1229] "External data sources" are sources of data obtained from outside sources such as social media, customer support systems, app stores, etc.

[1230] "Voice of Customer Data" is text data that includes feedback, opinions, and ratings expressed by customers about products and services.

[1231] "Preprocessing" is a series of data cleansing steps that remove unnecessary characters and tags from raw data and filter out duplicate data and inappropriate content.

[1232] A "generative AI model" is an artificial intelligence model used to extract and summarize key topics and trends from large amounts of data, such as GPT-4.

[1233] An "emotion engine" is an engine that uses natural language processing technology to analyze customer emotions (joy, anger, sadness, etc.) from text data.

[1234] A "report" is a document generated based on summarized voice of customer data and sentiment analysis results, and may include graphs and statistics.

[1235] "Vehicle managers and developers" are people or positions involved in the operation, maintenance, and development of autonomous vehicles who implement improvement measures based on feedback data.

[1236] System Overview

[1237] A system for implementing this invention automatically collects customer voice-of-mouth data from external data sources, summarizes it using a generative AI model and an emotion engine, and performs sentiment analysis. It then automatically generates reports based on the analysis results and distributes them to fleet managers and developers. This allows feedback on autonomous vehicles to be efficiently collected and analyzed and quickly shared with relevant parties.

[1238] Hardware and software used

[1239] 1. Hardware:

[1240] Server (for data collection and processing)

[1241] Smartphone (for collecting feedback)

[1242] 2. Software:

[1243] Requests: An HTTP request library for gathering data from external data sources.

[1244] OpenAI API: Generate summaries of data using generative AI models (e.g., GPT-4)

[1245] EmotionEngine: A natural language processing engine for emotion analysis

[1246] ReportLab: A library for saving automatically generated reports in PDF format.

[1247] Data processing and calculation

[1248] 1. Data Collection

[1249] The server automatically collects customer feedback data from external data sources (e.g., social media, customer support systems) via APIs, for example, by sending an HTTP request to an API endpoint and temporarily storing the retrieved data.

[1250] 2. Data Preprocessing

[1251] The collected customer feedback data is preprocessed by the server, which removes unnecessary characters and emojis, deletes HTML tags, and filters out duplicate data to generate clean text data.

[1252] 3. Data Summary

[1253] The preprocessed text data is then fed into a generative AI model (e.g., GPT-4) to summarize the key information. Generative AI models extract the most important topics and trends from the vast amount of data.

[1254] 4. Emotion analysis

[1255] The summarized text data is input to an emotion engine, which analyzes the customer's emotions (e.g., joy, anger, sadness, etc.). The emotion analysis results show the distribution of emotions for the customer's feedback.

[1256] 5. Report Generation and Delivery

[1257] Based on the analyzed data, the server automatically generates a report in PDF format. The report includes summary information, sentiment analysis results, graphs, and statistical information. The generated report is automatically distributed to vehicle managers and developers. For example, the report can be sent by email every week.

[1258] Specific examples

[1259] For example, suppose a passenger posts feedback such as, "This car is very comfortable, but the air conditioning doesn't work well." This feedback can be summarized as, "The vehicle is comfortable, but I'm dissatisfied with the air conditioning performance," and sentiment analysis can extract "happiness" and "dissatisfaction."

[1260] Prompt Sentence Examples

[1261] "Please summarize the following text:

[1262] This car is very comfortable, but the air conditioning doesn't work well.

[1263] By using the above system, customer feedback can be efficiently collected and analyzed, and used to improve the quality of autonomous vehicles.

[1264] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1265] Step 1:

[1266] Data collection

[1267] The server automatically collects customer feedback data from external data sources (e.g., social media or customer support systems). Specifically, the server uses an API to send an HTTP request and temporarily stores the data obtained as an API response. For example, a request is made to an API endpoint of a social media service to obtain post data containing keywords such as "vehicle," "comfort," and "air conditioning." This data is often stored in JSON format. The input is raw data obtained from the external data source, and the output is unprocessed data stored in temporary storage.

[1268] Step 2:

[1269] Data Preprocessing

[1270] The collected customer voice data is pre-processed by the server. Specifically, it removes inappropriate characters (e.g., emojis, HTML tags) and duplicate data, and normalizes the text. For example, "This car is very comfortable." The feedback "This car is very comfortable, but the air conditioning doesn't work well 🎉" is transformed into "This car is very comfortable, but the air conditioning doesn't work well." The input is raw data, and the output is clean text data.

[1271] Step 3:

[1272] Data Summary

[1273] The preprocessed customer voice data is input into a generative AI model by the server. This AI model summarizes the text using, for example, GPT-4. Specifically, it extracts key topics and trends from large amounts of feedback data and outputs them as summarized text. The prompt includes the instruction "Please summarize the following text:\n[text data]". The input is clean text data, and the output is summarized text.

[1274] Step 4:

[1275] Emotion analysis

[1276] The server inputs the summarized text data into an emotion engine, which analyzes the customer's emotions. Specifically, the data is classified into emotion categories such as "happiness," "anger," and "sadness," and a determination is made as to which emotion the text best expresses. For example, "happiness" and "dissatisfaction" are extracted from the text "This car is very comfortable, but the air conditioning doesn't work well." The input is the summarized text, and the output is the emotion analysis results.

[1277] Step 5:

[1278] Report Generation

[1279] Based on the analysis results, the server automatically generates a report. Specifically, it uses a library such as ReportLab to create a PDF report. The report includes summarized text, sentiment analysis results, graphs, and statistical information. For example, the report includes the original feedback in the "Product Benefits" section. The input is the summarized data and sentiment analysis results, and the output is a completed report file.

[1280] Step 6:

[1281] Report Distribution

[1282] The generated reports are automatically distributed by the server to vehicle managers and developers. Specifically, reports are sent to relevant parties using email or an internal messaging system. For example, the latest report is automatically sent at a fixed date and time each week. The input is a completed report file, and the output is a report delivered to the relevant parties' mailboxes.

[1283] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1284] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1285] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1286] [Fourth embodiment]

[1287] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1288] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1289] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1290] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1291] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1292] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1293] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1294] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1295] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1296] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1297] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1298] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1299] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1300] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, performs preprocessing, summarizes it using a generative AI model, automatically generates a report based on the summarized data, and finally delivers the report to service personnel. This allows for the collection, analysis, and reporting of customer feedback at low cost and efficiently.

[1301] System Overview

[1302] This system is centered around a server and includes the following main functions:

[1303] 1. Collecting data from external sources

[1304] The server collects customer feedback data from external data sources such as app stores, social media, customer support systems, etc. This collection can be done using APIs or scraping techniques, for example, using the Twitter API to automatically retrieve posts about the product.

[1305] 2. Data Preprocessing

[1306] The server preprocesses the collected customer feedback data. Preprocessing includes text cleaning, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[1307] 3. Data Summarization

[1308] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[1309] 4. Automatic report generation

[1310] The server automatically generates reports based on the summarized data, using report templates customized to the needs of each service. For example, it could categorize summarized customer testimonials into product advantages and disadvantages and generate a PDF report with graphs and statistics.

[1311] 5. Report Distribution

[1312] The server automatically delivers the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[1313] Specific examples

[1314] A specific example of this system is shown below.

[1315] Voice of the Customer Data Collection Example

[1316] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[1317] Data Preprocessing Example

[1318] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[1319] Data Summarization Example

[1320] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[1321] Report Generation Example

[1322] The server automatically generates a detailed product report based on the summarized information, in PDF format, including charts and graphs.

[1323] Report Delivery Example

[1324] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[1325] In this way, this system efficiently collects and analyzes customer feedback and promptly provides reports, thereby supporting service improvements and rapid response.

[1326] The processing flow will be explained below.

[1327] Step 1:

[1328] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[1329] Step 2:

[1330] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[1331] Step 3:

[1332] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[1333] Step 4:

[1334] The server cleans the text data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and inappropriate language) from the raw data.

[1335] Step 5:

[1336] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, which improves the accuracy of the analysis.

[1337] Step 6:

[1338] The server initializes and configures the generative AI model, which (e.g., GPT-4) uses specific configuration values ​​and prompts to summarize the data.

[1339] Step 7:

[1340] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates a concise summary.

[1341] Step 8:

[1342] The server automatically generates reports based on the summarized data, inserting the summarized information into pre-configured report templates to create customized reports tailored to the needs of each service.

[1343] Step 9:

[1344] The server saves the generated report and formats the document in the specified format, such as PDF or HTML.

[1345] Step 10:

[1346] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report.

[1347] Step 11:

[1348] The server adds a report summary to the body of the email and attaches the generated report document, allowing you to understand the contents of the email concisely.

[1349] Step 12:

[1350] The server sends emails to personnel and sends notifications to an internal messaging system, ensuring that all relevant personnel receive the latest report information.

[1351] Example 1

[1352] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1353] Traditional methods for collecting, analyzing, and reporting customer feedback data required a great deal of time and effort, making it difficult to efficiently collect and analyze information. Furthermore, because much of the processing was manual, it was difficult to maintain data accuracy and consistency. Traditional methods were particularly difficult to use when large amounts of data needed to be collected and analyzed in a short amount of time.

[1354] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1355] In this invention, the server includes means for automatically collecting customer voice data from external data sources, means for preprocessing the collected customer voice data, a generative AI model means for summarizing the preprocessed customer voice data, means for instructing the generative AI model to perform summarization using a prompt sentence, means for automatically generating a report in PDF format based on the summarized data, and means for delivering the generated report to a service representative via email or an internal message system. This makes it possible to efficiently and accurately collect and analyze customer voice data and quickly provide information to a service representative.

[1356] "External Data Sources" refers to sources of data obtained from various platforms and services on the Internet.

[1357] "Customer voice data" refers to text data such as opinions, impressions, and evaluations expressed by users about products and services.

[1358] "Preprocessing" refers to the process of removing unnecessary information from collected data and preparing it in a format suitable for analysis.

[1359] A "generative AI model" refers to an artificial intelligence program that learns from large amounts of data and performs natural language processing and text generation.

[1360] A "prompt sentence" refers to input text that instructs a generative AI model on specific actions and processing details.

[1361] "PDF format" is an abbreviation for Portable Document Format, and refers to a file format for electronically storing and displaying documents and image information.

[1362] "Mail" is an abbreviation for electronic mail and refers to a means of communication for exchanging text and files over the Internet.

[1363] An "internal messaging system" refers to a dedicated communication system for exchanging information and messages within an organization.

[1364] "Service representative" refers to a person who handles customer service and support tasks.

[1365] "API" stands for Application Programming Interface and refers to an interface for exchanging data and functions between different software applications.

[1366] This section describes an embodiment of the present invention. This system is primarily comprised of a server and has the functions of collecting customer feedback data from external data sources, preprocessing and summarizing it, and automatically generating and distributing reports. This system makes it possible to efficiently collect, analyze, and report customer feedback.

[1367] System configuration

[1368] The system consists of the following major hardware and software components:

[1369] Server: The central unit that processes and analyzes data. A general-purpose server equipped with a high-performance CPU and large memory capacity is used.

[1370] Network Interface: A network connection for communicating with external data sources.

[1371] Generative AI models: Generative AI models such as GPT-4 are used as models for natural language processing.

[1372] API: Used to retrieve data from external data sources. Examples include the Facebook API and Twitter API.

[1373] Mail Server: Email sending facility for delivering reports to service personnel.

[1374] Internal messaging system: A dedicated communication system for sharing information within an organization.

[1375] Data collection

[1376] The server runs a scheduled task every morning at 2:00 AM to collect customer feedback data from external data sources using APIs. For example, it uses the Facebook API to collect posts related to "New Product X." By sending an API request, data that matches the specified criteria is retrieved.

[1377] Data Preprocessing

[1378] The collected data is pre-processed on the server, which includes removing unnecessary characters and tags, filtering duplicate data, and eliminating inappropriate content. This process provides a clean dataset suitable for analysis.

[1379] Data Summarization

[1380] The preprocessed data is then input into a generative AI model, which uses prompts to provide specific summarization instructions to the model. For example, a prompt might be, "Summarize the main customer complaints about new product X." The generative AI model then extracts key information from the input data and provides a summary result.

[1381] Automatic report generation

[1382] The server automatically generates a PDF report based on the summarized data, including graphs and statistics based on the summary content, and formats the report according to an automatically generated template.

[1383] Report distribution

[1384] The generated reports are delivered from the server to service personnel via email from the mail server and via the internal messaging system, allowing personnel to receive timely feedback from customers.

[1385] Specific examples

[1386] For example, the server collects posts about "New Product X" using the Facebook API at 2:00 a.m. every morning and temporarily stores them in a database. It then performs data preprocessing and summarizes them by inputting a prompt to the generative AI model: "Summarize the main customer complaints about New Product X." Finally, it generates a PDF report based on the summary results and sends it to the service representative via email.

[1387] In this way, the system automates the entire process from data collection to report distribution, enabling efficient analysis and reporting of customer feedback.

[1388] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1389] Step 1:

[1390] Data collection

[1391] The server runs a scheduled task every morning at 2am, using APIs to collect voice of customer data from external data sources.

[1392] Input: Facebook API key, search query (e.g. "New Product X").

[1393] Specific operation: The server sends an API request to retrieve data based on the specified conditions, and the retrieved data is temporarily stored in a database.

[1394] Output: The raw data collected (e.g. Facebook posts).

[1395] Step 2:

[1396] Data Preprocessing

[1397] The server pre-processes the collected raw data.

[1398] Input: Raw data collected.

[1399] What it does: The server uses regular expressions to remove unwanted characters and tags, and also filters out duplicate data and spam posts.

[1400] Output: A clean dataset.

[1401] Step 3:

[1402] Data Summarization

[1403] The server inputs the preprocessed data into a generative AI model to generate summaries.

[1404] Input: A clean dataset, a prompt statement (e.g., "Summarize the main customer complaints about new product X").

[1405] Specific operation: The server sends prompt sentences to the generative AI model (GPT-4) to generate a summary of the data.

[1406] Output: Summarized text data.

[1407] Step 4:

[1408] Automatic report generation

[1409] The server automatically generates a PDF report based on the summarized data.

[1410] Input: Abstracted text data.

[1411] What it does: The server embeds summary information into a report template, adds graphs and statistics, and generates a PDF report.

[1412] Output: Generated PDF report.

[1413] Step 5:

[1414] Report distribution

[1415] The server delivers the generated report to the service representative.

[1416] Input: PDF report, service representative's email address or internal messaging system user ID.

[1417] What it does: The server uses the SMTP library to send emails and notify through its internal messaging system.

[1418] Output: The delivered report.

[1419] (Application example 1)

[1420] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1421] In today's highly competitive market, brick-and-mortar stores need to quickly and efficiently collect customer feedback and use it to improve their services. Traditional feedback collection methods are manual, time-consuming, and labor-intensive, and analyzing data and generating reports is difficult. In particular, there is a need for an integrated system that utilizes smart devices such as smartphones, tablets, and smart glasses to efficiently collect feedback and effectively analyze it.

[1422] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1423] In this invention, the server includes means for automatically collecting customer feedback data from external data sources, means for preprocessing the collected customer feedback data, means for generating an AI model that summarizes the preprocessed customer feedback data, means for automatically generating a report based on the summarized data, means for distributing the generated report to a service representative, means for collecting customer feedback via a smart device, and means for automatically sending the generated report to a store manager or staff member, thereby enabling efficient collection and analysis of customer feedback and rapid service improvement.

[1424] Definitions of important words

[1425] "External data sources" are external data providers from which customer feedback data can be obtained, such as app stores, social media, and customer support systems.

[1426] "Voice of customer data" refers to text data that includes feedback, opinions, and impressions provided by customers regarding products and services.

[1427] "Preprocessing" refers to a series of processes that remove unnecessary characters and tags from collected text data, delete duplicates, and filter inappropriate content.

[1428] A "generative AI model" is an artificial intelligence model used to extract and summarize important information from collected and pre-processed data. A specific example is GPT-4.

[1429] A "report" is a document generated from summarized data that visualizes and presents product or service performance, trends, and areas for improvement.

[1430] "Smart devices" are portable electronic devices with advanced functions, such as smartphones, tablets, and smart glasses.

[1431] "Store Manager" means the person responsible for the operation and management of the physical store.

[1432] "Staff" refers to employees who deal with customers and perform other duties at the store.

[1433] "API" stands for Application Programming Interface, a standardized means of exchanging data and functions between different software programs.

[1434] "Automatic generation" is a process in which a system automatically processes and generates results without human intervention.

[1435] "Auto Send" is a feature that automatically sends data such as reports generated by the system to designated recipients.

[1436] MODE FOR CARRYING OUT THE INVENTION

[1437] An embodiment of the present invention will now be described. This system is centered around a server, and efficiently collects, analyzes, and generates reports on customer feedback in physical stores via smart devices. This allows store managers and staff to understand customer feedback in real time and respond quickly.

[1438] System Overview

[1439] Hardware and software used

[1440] Hardware:

[1441] Smartphone

[1442] tablet

[1443] Smart Glasses

[1444] software:

[1445] iOS / Android app

[1446] Smart Glasses App

[1447] Server (Amazon Web Services, etc.)

[1448] Generative AI model (GPT-4)

[1449] Program processing description

[1450] 1. Data Collection:

[1451] Terminal: Smart devices (smartphones, tablets, smart glasses) installed in the store provide a feedback input form for customers. The feedback data entered by customers is sent to the server. In addition, external posting data about the store is collected using SNS APIs (e.g., Instagram API and Twitter API).

[1452] Example: Customers can enter feedback via in-store tablets, such as:

[1453] "The staff were amazing! I wish they'd hire more staff so the cashiers don't get too crowded."

[1454] 2. Data Preprocessing:

[1455] Server: Text cleans the collected data, removing unnecessary characters, tags, and duplicate data, as well as filtering out inappropriate content.

[1456] Example: Remove emojis and HTML tags to generate clean text data.

[1457] 3. Data Summary:

[1458] Server: Feeds pre-processed data into a generative AI model (GPT-4) to summarize key topics, trends, customer requests and complaints.

[1459] Example prompt sentence:

[1460] Summarize the following customer feedback:

[1461] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[1462] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[1463] 4. Report Generation:

[1464] Server: Based on the summarized information, the server visualizes store performance and customer opinions, automatically generating reports with charts and statistics, which are saved in PDF and HTML formats.

[1465] Example: The report includes the advantages and disadvantages of the product or service, the items most mentioned by customers, and suggestions for improvement.

[1466] 5. Report Delivery:

[1467] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback with staff using the notification system.

[1468] Example: Every day, an updated report is emailed to the store manager's smartphone and also notified via an internal notification system.

[1469] This enables efficient collection and analysis of customer feedback and rapid service improvement.

[1470] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1471] Program processing steps

[1472] Step 1: Data collection

[1473] input:

[1474] Customer feedback input from in-store smart devices

[1475] External posting data from SNS API

[1476] Specific behavior:

[1477] Terminals: Smart devices (smartphones, tablets, smart glasses) installed in the store collect customer feedback and send it to the server.

[1478] Server: Uses APIs to collect store-related post data from social media and external data sources (e.g., Instagram API and Twitter API) and stores it in temporary storage.

[1479] output:

[1480] Feedback data and SNS posting data stored in temporary storage

[1481] Step 2: Data Preprocessing

[1482] input:

[1483] Feedback data and SNS posting data stored in temporary storage

[1484] Specific behavior:

[1485] Server: Cleans the collected data, removing unnecessary characters, tags, and duplicate data, and filters out inappropriate content, using regular expressions (RegEx) and text analysis tools.

[1486] output:

[1487] Clean text data

[1488] Step 3: Summarize the data

[1489] input:

[1490] Clean text data

[1491] Specific behavior:

[1492] Server: Inputs the preprocessed data into a generative AI model (GPT-4) to summarize important information and trends, and provides appropriate instructions to the model using prompts.

[1493] Example prompt sentence:

[1494] Summarize the following customer feedback:

[1495] Customer feedback: "The staff were amazing! I wish they'd hire more staff so the registers aren't overcrowded."

[1496] Instructions: Summarize the main points of the feedback and extract areas for improvement.

[1497] output:

[1498] Summarized feedback information

[1499] Step 4: Generate a report

[1500] input:

[1501] Summarized feedback information

[1502] Specific behavior:

[1503] Server: Based on the summarized information, the server automatically generates reports containing charts and statistics that visualize store performance and customer opinions. Reports are saved in PDF and HTML formats.

[1504] output:

[1505] PDF and HTML reports

[1506] Step 5: Report Delivery

[1507] input:

[1508] PDF and HTML reports

[1509] Specific behavior:

[1510] Server: Automatically sends generated reports to store managers and staff's smart devices. Share the latest feedback using an internal notification system.

[1511] output:

[1512] Reports and notifications sent to store managers and staff's smart devices

[1513] By implementing each processing step in this manner, it becomes possible to efficiently collect and analyze customer feedback and quickly improve services.

[1514] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1515] This section describes an embodiment of the present invention. This system automatically collects customer feedback data from external data sources, preprocesses it, summarizes it using a generative AI model, and then recognizes user emotions using an emotion engine. It then automatically generates a report and delivers it to service personnel. This allows for a deeper understanding of customer feedback, and enables efficient collection, analysis, and reporting at low cost.

[1516] System Overview

[1517] This system is centered around a server and includes the following main functions:

[1518] 1. Collecting data from external sources

[1519] The server collects customer feedback data from external sources such as app stores, social media, and customer support systems using APIs or scraping techniques. For example, Twitter APIs can be used to automatically retrieve product posts.

[1520] 2. Data Preprocessing

[1521] The server preprocesses the collected customer feedback data. Preprocessing includes text cleaning, removing duplicate data, and filtering inappropriate content. For example, the server removes unnecessary data (e.g., emojis and HTML tags) from the posts and formats the text data.

[1522] 3. Data Summarization

[1523] The server then inputs the preprocessed data into a generative AI model, such as GPT-4, which extracts and summarizes important information. This model extracts key topics and trends from the vast amount of data.

[1524] 4. Emotion Recognition by Emotion Engine

[1525] The server inputs the summarized data into an emotion engine to analyze the user's emotions. The emotion engine uses natural language processing technology to recognize the customer's emotions (e.g., joy, anger, sadness, etc.) from the text data.

[1526] 5. Automatic report generation

[1527] The server automatically generates reports based on the summarized data and the results of sentiment analysis by the sentiment engine. The reports include a summary of customer feedback along with the results of the sentiment analysis. A customized report template is used depending on the needs of each service. For example, a report in PDF format is generated that categorizes the summarized customer feedback into product advantages and disadvantages and includes graphs and statistics that reflect the sentiment analysis results.

[1528] 6. Report Distribution

[1529] The server automatically delivers the generated reports to the service personnel via email or an internal messaging system, for example, sending the latest report to the personnel's mailbox every Monday.

[1530] Specific examples

[1531] A specific example of this system is shown below.

[1532] Voice of the Customer Data Collection Example

[1533] The server collects posts about a specific product using the Facebook API every morning at 2:00 a.m. The data is stored in temporary storage.

[1534] Data Preprocessing Example

[1535] From the collected posts, the server removes unnecessary characters and tags, filters out duplicates and inappropriate content, and after this step, a clean dataset is generated.

[1536] Data Summarization Example

[1537] The server feeds the clean dataset into a generative AI model that summarizes the product's advantages and disadvantages, as well as the topics most mentioned by customers.

[1538] Emotion Recognition Example

[1539] The server inputs the data summarized by the generative AI model into an emotion engine to recognize customer emotions, for example, extracting "joy" from the text "This product is very good."

[1540] Report Generation Example

[1541] The server automatically generates a detailed product report based on the summarized information and emotion recognition results, in PDF format, including charts and graphs.

[1542] Report Delivery Example

[1543] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[1544] In this way, this system efficiently collects and analyzes customer feedback, understands customer emotions through sentiment analysis, and promptly provides reports, thereby supporting service improvements and rapid response.

[1545] The processing flow will be explained below.

[1546] Step 1:

[1547] The server runs regularly scheduled tasks, triggering APIs and scraping scripts. For example, the server collects the latest voice of customer data from each data source at 2 AM every day.

[1548] Step 2:

[1549] The server sends requests to each data source (e.g., Facebook API, Twitter API, customer support system API) to obtain the latest feedback data. When using scraping technology, customer reviews and comments are extracted from web pages.

[1550] Step 3:

[1551] The server stores the acquired raw data in a temporary storage, which is used for uniform handling of the data during subsequent pre-processing steps.

[1552] Step 4:

[1553] The server cleans the raw data by removing unnecessary characters and tags (e.g., emojis, HTML tags, and profanity) from the text data, for example by using regular expressions to remove emojis and tags.

[1554] Step 5:

[1555] The server removes duplicate entries from the preprocessed data and filters out meaningless entries and spam, improving the accuracy of the analysis. For example, it generates hash values ​​to remove duplicate posts and combines data with the same hash value.

[1556] Step 6:

[1557] The server initializes and configures the generative AI model. The generative AI model (e.g., GPT-4) uses specific settings and prompts for summarizing the data, such as the length of the summary or the priority of certain keywords.

[1558] Step 7:

[1559] The server inputs the cleaned customer feedback data into a generative AI model, which then summarizes the data. The model extracts key topics and trends from the vast amount of data and generates concise summaries, such as identifying the characteristics and issues frequently mentioned by customers.

[1560] Step 8:

[1561] The server inputs the data summarized by the generative AI model into the emotion engine to recognize the user's emotions. The emotion engine uses natural language processing technology to analyze customer emotions (e.g., joy, anger, sadness, etc.) from the text data. For example, it extracts "joy" from the text "This product is very good."

[1562] Step 9:

[1563] The server automatically generates reports based on the summarized data and the sentiment analysis results from the sentiment engine. The server then inserts the summarized information and sentiment analysis results into pre-configured report templates to create customized reports tailored to the needs of each service. For example, the server visually displays the sentiment analysis results as charts and graphs.

[1564] Step 10:

[1565] The server saves the generated report and renders the document in a specified format, such as PDF or HTML. For example, the report can be saved in PDF format and made available as an email attachment.

[1566] Step 11:

[1567] The server initializes the email sending settings and obtains the email addresses of each service representative to send the report to, for example, obtains the latest email address from the list of representatives.

[1568] Step 12:

[1569] The server adds a report summary to the body of the email and attaches the generated report document, giving a concise understanding of the email's contents. For example, it might insert a simple message such as "This week's customer feedback report is attached."

[1570] Step 13:

[1571] The server sends emails to the relevant personnel and sends notifications to an internal messaging system, so that all relevant personnel receive the latest report information, for example, by sending notifications to an internal chat tool.

[1572] Example 2

[1573] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1574] There is a need to collect, analyze, and report customer opinions effectively and efficiently, but traditional methods require a great deal of time and effort to process large amounts of data. It is also difficult to accurately grasp customer sentiment, which can delay service improvements and prompt responses. This can lead to lower customer satisfaction and a loss of trust in companies.

[1575] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1576] In this invention, the server includes means for automatically collecting customer opinion data from an external data source, means for preprocessing the collected customer opinion data, means for generating an AI model that summarizes the preprocessed customer opinion data, means for analyzing the summarized data and recognizing customer emotions, means for automatically generating a report based on the summarized data and the emotion recognition results, and means for delivering the generated report to a person in charge. This makes it possible to efficiently collect and analyze customer opinions, accurately grasp customer emotions, and quickly provide them as a report.

[1577] "External Data Sources" refers to data available from publicly available databases on the internet, social media platforms, online stores, and other digital environments.

[1578] "Customer Opinion Data" means customer ratings, feedback, reviews, comments, and other textual opinions about products and services.

[1579] "Automated collection means" refers to technology designed to collect data continuously, using programmable means, without human intervention.

[1580] "Preprocessing means" refers to technologies used to process collected data, such as by cleaning text, removing duplicate data, and filtering inappropriate content, before analyzing the data.

[1581] "Generative AI model means" refers to a system or program that uses artificial intelligence technology to extract important information from input data and generate a summary statement.

[1582] "Means for recognizing emotions" refers to technology that uses natural language processing technology to analyze customer emotions from text data and assign specific emotional labels (e.g., joy, anger, sadness, etc.).

[1583] "Means for automatically generating reports" refers to a system or program that automatically generates reports based on summarized data and emotion recognition results using predefined templates.

[1584] "Delivery means" refers to a system or program for sending generated reports to appropriate personnel via email or internal messaging systems.

[1585] A specific embodiment for implementing this invention will be described below. This system automatically collects customer opinion data from external data sources, preprocesses it, summarizes it using a generative AI model, recognizes customer emotions through an emotion engine, and automatically generates a report based on the results and delivers it to the person in charge.

[1586] System Overview

[1587] This system is centered around the server and includes the following main functions:

[1588] Data collection from external data sources

[1589] The server uses databases and social media platforms publicly available on the Internet as external data sources. Specific examples include collecting data using the Twitter API and Facebook API. For example, every morning at 2:00, the server automatically retrieves tweets related to "product name" via the Twitter API and stores them in temporary storage.

[1590] Data Preprocessing

[1591] The server pre-processes the collected data, which includes text cleaning (removing unnecessary characters and tags), removing duplicate data, and filtering inappropriate content. Regular expressions and filtering algorithms are used to remove emojis and HTML tags, and then the clean data is generated.

[1592] Data Summarization

[1593] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to extract and summarize important information. For example, the server passes text data to a GPT-4 model and receives text summarizing key topics and customer opinions as output.

[1594] emotion recognition

[1595] The server inputs the summarized data into an emotion engine to analyze customer emotions. The emotion engine uses natural language processing technology to assign emotion labels such as "happiness," "anger," and "sadness" to the text. For example, it extracts "happiness" from the text "This product is very good."

[1596] Report Generation

[1597] The server automatically generates a report based on the summarized data and emotion recognition results. The report is created using a template and includes product advantages and disadvantages, as well as customer sentiment analysis in the form of charts and graphs. The generated report is saved in PDF or HTML format.

[1598] Report Distribution

[1599] The server automatically delivers the generated report to the person in charge by attaching the report to an email using the SMTP protocol and sending it to a specified email address. For example, the server could send the latest report to the person in charge's mailbox every Monday.

[1600] Specific examples

[1601] A specific example of this system is shown below.

[1602] Customer Opinion Data Collection Example

[1603] Every morning at 2am, the server uses the Facebook API to collect posts about a specific product and stores them in temporary storage.

[1604] Specific examples of data preprocessing

[1605] The server then removes unnecessary characters and tags from the collected posts, filters out duplicates and inappropriate content, and uses regular expressions and filtering algorithms to generate a clean dataset.

[1606] Examples of data summarization

[1607] The server feeds the preprocessed data into a GPT-4 model, which generates text summarizing key topics and trends.

[1608] Specific examples of emotion recognition

[1609] The server inputs the summaries generated by the generative AI model into an emotion engine to analyze customer emotions, for example, extracting the emotion "joy" from the text "This product is very good."

[1610] Example of report generation

[1611] The server automatically generates a detailed report based on the summarized data and emotion recognition results, which is saved in PDF format and includes charts and graphs.

[1612] Example of report distribution

[1613] The server sends the generated PDF report to the service representative's email address and also notifies them via an internal messaging system.

[1614] This format makes it possible to efficiently collect customer opinions, conduct detailed analysis, and quickly provide the results in report format.

[1615] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1616] Step 1: Collect data from external data sources

[1617] The server collects customer opinion data from external data sources. Specifically, at 2:00 a.m. every morning, the server retrieves posts related to specific keywords or hashtags using the Twitter API or Facebook API. The collected data is stored in temporary storage in JSON format.

[1618] Input: External data source (Twitter API, Facebook API, etc.)

[1619] Data processing: API requests, data acquisition, conversion to JSON format

[1620] Output: JSON format data stored in temporary storage

[1621] Step 2: Preprocessing the data

[1622] The server preprocesses the collected data. First, it reads the JSON data and removes unnecessary characters and tags (e.g., emojis, HTML tags). Second, it detects and removes duplicate data. Third, it filters out inappropriate content.

[1623] Input: JSON format data stored in temporary storage

[1624] Data operations: text cleaning, de-duplicating data, filtering

[1625] Output: A preprocessed, clean dataset

[1626] Step 3: Summarize the data

[1627] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate a summary. Specifically, the server passes the clean text data to GPT-4 and receives a summary that extracts important topics and claims.

[1628] Input: Preprocessed clean dataset

[1629] Data calculation: Extraction of important information and generation of summaries using generative AI models

[1630] Output: Generated summary

[1631] Step 4: Emotion Recognition

[1632] The server inputs the summary sentence into the emotion engine for emotion analysis. The emotion engine uses natural language processing technology to identify the customer's emotion (e.g., joy, anger, sadness) from the text.

[1633] Input: Generated summary

[1634] Data Computing: Sentiment Analysis with Natural Language Processing

[1635] Output: Sentiment analysis results with emotion labels

[1636] Step 5: Generate a report

[1637] The server automatically generates a report based on the summary and the results of the sentiment analysis. The report is created based on a pre-prepared template and includes the summary and the results of the sentiment analysis as charts and graphs. The generated report is saved in PDF or HTML format.

[1638] Input: Summary and sentiment analysis results

[1639] Data calculation: Insert data into templates, automatically generate reports

[1640] Output: Report in PDF or HTML format

[1641] Step 6: Report Distribution

[1642] The server automatically delivers the generated report to the person in charge by attaching the report to an email and sending it to the person's email address. It also notifies the person in charge via the internal messaging system.

[1643] Input: Report in PDF or HTML format

[1644] Data calculation: Email composition and sending using the SMTP protocol

[1645] Output: Reports sent to the assignee's mailbox and notifications in the internal messaging system

[1646] Through these steps, this system is able to efficiently collect and analyze customer opinion data and quickly provide it as a report.

[1647] (Application example 2)

[1648] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1649] Autonomous vehicles require efficient collection and analysis of passenger feedback and the ability to improve vehicle performance and user experience based on that feedback. However, traditional methods often require manual collection of feedback, and data analysis and emotion recognition are ineffective, resulting in limited real-time performance and accuracy. Additionally, there is a lack of a way to quickly share reports generated based on feedback with relevant parties. This can delay the implementation of prompt improvements based on feedback.

[1650] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automatically collecting customer voice data from an external data source, means for preprocessing the collected customer voice data, generative AI model means for summarizing the preprocessed customer voice data, emotion engine means for analyzing emotions based on the summarized data, means for automatically generating a report based on the analyzed data, and means for distributing the generated report to a vehicle manager or developer of the autonomous vehicle. This makes it possible to collect and analyze passenger feedback in real time and quickly distribute the results to relevant parties.

[1651] "External data sources" are sources of data obtained from outside sources such as social media, customer support systems, app stores, etc.

[1652] "Voice of Customer Data" is text data that includes feedback, opinions, and ratings expressed by customers about products and services.

[1653] "Preprocessing" is a series of data cleansing steps that remove unnecessary characters and tags from raw data and filter out duplicate data and inappropriate content.

[1654] A "generative AI model" is an artificial intelligence model used to extract and summarize key topics and trends from large amounts of data, such as GPT-4.

[1655] An "emotion engine" is an engine that uses natural language processing technology to analyze customer emotions (joy, anger, sadness, etc.) from text data.

[1656] A "report" is a document generated based on summarized voice of customer data and sentiment analysis results, and may include graphs and statistics.

[1657] "Vehicle managers and developers" are people or positions involved in the operation, maintenance, and development of autonomous vehicles who implement improvement measures based on feedback data.

[1658] System Overview

[1659] A system for implementing this invention automatically collects customer voice-of-mouth data from external data sources, summarizes it using a generative AI model and an emotion engine, and performs sentiment analysis. It then automatically generates reports based on the analysis results and distributes them to fleet managers and developers. This allows feedback on autonomous vehicles to be efficiently collected and analyzed and quickly shared with relevant parties.

[1660] Hardware and software used

[1661] 1. Hardware:

[1662] Server (for data collection and processing)

[1663] Smartphone (for collecting feedback)

[1664] 2. Software:

[1665] Requests: An HTTP request library for gathering data from external data sources.

[1666] OpenAI API: Generate summaries of data using generative AI models (e.g., GPT-4)

[1667] EmotionEngine: A natural language processing engine for emotion analysis

[1668] ReportLab: A library for saving automatically generated reports in PDF format.

[1669] Data processing and calculation

[1670] 1. Data Collection

[1671] The server automatically collects customer feedback data from external data sources (e.g., social media, customer support systems) via APIs, for example, by sending an HTTP request to an API endpoint and temporarily storing the retrieved data.

[1672] 2. Data Preprocessing

[1673] The collected customer feedback data is preprocessed by the server, which removes unnecessary characters and emojis, deletes HTML tags, and filters out duplicate data to generate clean text data.

[1674] 3. Data Summary

[1675] The preprocessed text data is then fed into a generative AI model (e.g., GPT-4) to summarize the key information. Generative AI models extract the most important topics and trends from the vast amount of data.

[1676] 4. Emotion analysis

[1677] The summarized text data is input to an emotion engine, which analyzes the customer's emotions (e.g., joy, anger, sadness, etc.). The emotion analysis results show the distribution of emotions for the customer's feedback.

[1678] 5. Report Generation and Delivery

[1679] Based on the analyzed data, the server automatically generates a report in PDF format. The report includes summary information, sentiment analysis results, graphs, and statistical information. The generated report is automatically distributed to vehicle managers and developers. For example, the report can be sent by email every week.

[1680] Specific examples

[1681] For example, suppose a passenger posts feedback such as, "This car is very comfortable, but the air conditioning doesn't work well." This feedback can be summarized as, "The vehicle is comfortable, but I'm dissatisfied with the air conditioning performance," and sentiment analysis can extract "happiness" and "dissatisfaction."

[1682] Prompt Sentence Examples

[1683] "Please summarize the following text:

[1684] This car is very comfortable, but the air conditioning doesn't work well.

[1685] By using the above system, customer feedback can be efficiently collected and analyzed, and used to improve the quality of autonomous vehicles.

[1686] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1687] Step 1:

[1688] Data collection

[1689] The server automatically collects customer feedback data from external data sources (e.g., social media or customer support systems). Specifically, the server uses an API to send an HTTP request and temporarily stores the data obtained as an API response. For example, a request is made to an API endpoint of a social media service to obtain post data containing keywords such as "vehicle," "comfort," and "air conditioning." This data is often stored in JSON format. The input is raw data obtained from the external data source, and the output is unprocessed data stored in temporary storage.

[1690] Step 2:

[1691] Data Preprocessing

[1692] The collected customer voice data is pre-processed by the server. Specifically, it removes inappropriate characters (e.g., emojis, HTML tags) and duplicate data, and normalizes the text. For example, "This car is very comfortable." The feedback "This car is very comfortable, but the air conditioning doesn't work well 🎉" is transformed into "This car is very comfortable, but the air conditioning doesn't work well." The input is raw data, and the output is clean text data.

[1693] Step 3:

[1694] Data Summary

[1695] The preprocessed customer voice data is input into a generative AI model by the server. This AI model summarizes the text using, for example, GPT-4. Specifically, it extracts key topics and trends from large amounts of feedback data and outputs them as summarized text. The prompt includes the instruction "Please summarize the following text:\n[text data]". The input is clean text data, and the output is summarized text.

[1696] Step 4:

[1697] Emotion analysis

[1698] The server inputs the summarized text data into an emotion engine, which analyzes the customer's emotions. Specifically, the data is classified into emotion categories such as "happiness," "anger," and "sadness," and a determination is made as to which emotion the text best expresses. For example, "happiness" and "dissatisfaction" are extracted from the text "This car is very comfortable, but the air conditioning doesn't work well." The input is the summarized text, and the output is the emotion analysis results.

[1699] Step 5:

[1700] Report Generation

[1701] Based on the analysis results, the server automatically generates a report. Specifically, it uses a library such as ReportLab to create a PDF report. The report includes summarized text, sentiment analysis results, graphs, and statistical information. For example, the report includes the original feedback in the "Product Benefits" section. The input is the summarized data and sentiment analysis results, and the output is a completed report file.

[1702] Step 6:

[1703] Report Distribution

[1704] The generated reports are automatically distributed by the server to vehicle managers and developers. Specifically, reports are sent to relevant parties using email or an internal messaging system. For example, the latest report is automatically sent at a fixed date and time each week. The input is a completed report file, and the output is a report delivered to the relevant parties' mailboxes.

[1705] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1706] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1707] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1708] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1709] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1710] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1711] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1712] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1713] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1714] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1715] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1716] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1717] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1718] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1719] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1720] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1721] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1722] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1723] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1724] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1725] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1726] The following is further disclosed regarding the above embodiment.

[1727] (Claim 1)

[1728] A means of automatically collecting voice of customer data from external data sources; and

[1729] A means for pre-processing the collected customer voice data;

[1730] a generative AI model means for summarizing the preprocessed voice-of-customer data; and

[1731] A means for automatically generating reports based on the summarized data;

[1732] means for delivering the generated report to a service representative;

[1733] A system including:

[1734] (Claim 2)

[1735] 10. The system of claim 1, further comprising means for utilizing an API to collect voice-of-the-customer data from an external data source.

[1736] (Claim 3)

[1737] 10. The system of claim 1, further comprising means for saving the report generated based on the summarized data in PDF and HTML formats.

[1738] "Example 1"

[1739] (Claim 1)

[1740] A means of automatically collecting voice of customer data from external data sources; and

[1741] A means for pre-processing the collected customer voice data;

[1742] a generative AI model means for summarizing the preprocessed voice-of-customer data; and

[1743] a means for instructing the generative AI model to summarize using prompt sentences;

[1744] A means to automatically generate PDF reports based on the summarized data,

[1745] a means for delivering the generated report to service personnel via email or an internal messaging system;

[1746] A system including:

[1747] (Claim 2)

[1748] 10. The system of claim 1, further comprising means for collecting voice-of-customer data from an external data source utilizing an API.

[1749] (Claim 3)

[1750] 10. The system of claim 1, further comprising means for saving reports generated based on the summarized data in PDF format and in a format suitable for an internal messaging system.

[1751] "Application Example 1"

[1752] Claims

[1753] (Claim 1)

[1754] A means of automatically collecting voice of customer data from external data sources; and

[1755] A means for pre-processing the collected customer voice data;

[1756] a generative AI model means for summarizing the preprocessed voice-of-customer data; and

[1757] A means for automatically generating reports based on the summarized data;

[1758] means for delivering the generated report to a service representative;

[1759] A means of collecting customer feedback via smart devices;

[1760] A means to automatically send generated reports to store managers and staff,

[1761] A system including:

[1762] (Claim 2)

[1763] 10. The system of claim 1, further comprising means for utilizing an API to collect voice-of-the-customer data from an external data source.

[1764] (Claim 3)

[1765] 10. The system of claim 1, further comprising means for saving the report generated based on the summarized data in PDF and HTML formats.

[1766] "Example 2: Combining Emotion Engines"

[1767] (Claim 1)

[1768] a means for automatically collecting customer opinion data from external data sources; and

[1769] a means for preprocessing the collected customer opinion data;

[1770] a generative AI model means for summarizing the pre-processed customer opinion data;

[1771] A means of analyzing the summarized data and recognizing customer sentiment;

[1772] means for automatically generating a report based on the summarized data and emotion recognition results;

[1773] a means for distributing the generated reports to the appropriate personnel;

[1774] A system including:

[1775] (Claim 2)

[1776] 10. The system of claim 1, further comprising means for utilizing an API to collect customer opinion data from an external data source.

[1777] (Claim 3)

[1778] 10. The system of claim 1, further comprising means for saving the report generated based on the summarized data and emotion recognition results in PDF and HTML formats.

[1779] "Application example 2 when combining emotion engines"

[1780] (Claim 1)

[1781] A means of automatically collecting voice of customer data from external data sources; and

[1782] A means for pre-processing the collected customer voice data;

[1783] a generative AI model means for summarizing the preprocessed voice-of-customer data; and

[1784] an emotion engine means for analyzing emotions based on the summarized data;

[1785] A means for automatically generating reports based on the analyzed data;

[1786] A means for distributing the generated reports to fleet managers and developers of autonomous vehicles; and

[1787] A system including:

[1788] (Claim 2)

[1789] 10. The system of claim 1, further comprising means for utilizing an API to collect voice-of-the-customer data from an external data source.

[1790] (Claim 3)

[1791] 10. The system of claim 1, further comprising means for saving the report generated based on the summarized data and the analyzed emotion data in PDF and HTML formats. [Explanation of symbols]

[1792] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of automatically collecting voice of customer data from external data sources; and A means for pre-processing the collected customer voice data; a generative AI model means for summarizing the preprocessed voice-of-customer data; and A means for automatically generating reports based on the summarized data; means for delivering the generated report to a service representative; A system including:

2. 10. The system of claim 1, further comprising means for utilizing an API to collect voice-of-the-customer data from external data sources.

3. 10. The system of claim 1, further comprising means for saving reports generated based on the summarized data in PDF and HTML formats.

Images