system

Abstract

We provide the system. [Solution] Means for collecting survey results from customers or employees, A means of cleaning and preprocessing the collected questionnaire data, A method for passing pre-processed data to a generative artificial intelligence system to perform filtering, A means for aggregating filtering results and performing analysis based on them, A means of generating a report based on the analysis results and notifying administrators or relevant parties, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional questionnaire aggregation system, it is difficult to efficiently process a large amount of questionnaire data and lead to insights useful for analysis. In particular, manually filtering text data such as free responses and performing sentiment analysis is time-consuming and labor-intensive, so an efficient method is required. In addition, missing value imputation and data cleaning are often performed manually, which has an adverse effect on data quality degradation and the reliability of analysis results. To solve these problems, a system that can automate the entire process from data collection to analysis and report creation is necessary.

Means for Solving the Problems

[0005] The present invention relates to a system for efficiently collecting and processing survey results from customers or employees and automatically reporting the analysis results. The system includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for aggregating the filtering results and performing analysis based thereon, and means for generating a report based on the analysis results and notifying administrators or relevant parties.

[0006] Furthermore, the generative artificial intelligence includes means for classifying the emotions expressed in survey responses and means for transmitting the classified emotion data, thereby enabling automatic filtering of open-ended text data. The preprocessing means also includes means for imputing missing values ​​in the survey data, thereby improving data quality. With this configuration, the entire process from survey data collection to report generation can be automated, efficiently providing high-quality analysis results.

[0007] "Survey results" refer to data such as opinions, evaluations, and feedback provided by customers or employees.

[0008] "Collection" refers to the process of importing survey results into a database or similar system.

[0009] "Preprocessing" is the process of imputing missing values ​​and correcting outliers to make data easier to analyze.

[0010] "Generative artificial intelligence" refers to advanced programs that perform generation and classification tasks using natural language processing techniques.

[0011] "Filtering" is the process of classifying or selecting data according to specific criteria.

[0012] "Aggregation" is the process of statistically summarizing filtered data.

[0013] "Analysis" is a process of deriving useful insights based on aggregated data.

[0014] "Report" is a document that summarizes the analysis results in an easy-to-understand format.

[0015] "Administrator" is a user who operates and supervises the system.

[0016] "Cleaning" refers to the process performed to maintain the consistency and accuracy of data.

[0017] "Notification" is a process of informing interested parties of the analysis results or reports.

Brief Explanation of Drawings

[0018] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10]Shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention is a system that efficiently collects survey results from customers or employees, cleans them, filters them using generative artificial intelligence, and analyzes them, ultimately generating a report.

[0040] System Overview

[0041] This system includes means for collecting survey results, means for pre-processing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[0042] Program processing

[0043] The program's processing can be explained in natural language as follows:

[0044] 1. Data Collection

[0045] Users enter information through a survey form. This survey form operates on a web browser or a dedicated application.

[0046] The terminal sends this input data to the server. The transmitted data is received by the server and initially stored.

[0047] 2. Data preprocessing

[0048] The server stores the collected survey data in a database. Then, it extracts the data for preprocessing.

[0049] The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[0050] 3. Filtering using generative artificial intelligence

[0051] The server passes the cleaned data to a generative artificial intelligence (AI). This AI, for example, analyzes open-ended text data and classifies the emotions expressed as "positive" or "negative."

[0052] The generative artificial intelligence returns the filtered results. These results are then saved again on the server side.

[0053] 4. Summary and Analysis of Results

[0054] The server aggregates the filtering results and performs a detailed analysis based on them. For example, it calculates the percentage of positive responses and the percentage of negative responses.

[0055] Furthermore, statistical analysis will be conducted on specific question items and by department.

[0056] 5. Report generation and notification

[0057] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts.

[0058] The server generates reports and notifies the administrator. These notifications are sent via email or a dashboard.

[0059] Specific example

[0060] For example, consider a case where a company conducts a survey to investigate employee satisfaction. In this survey, employees answer questions about their work environment and job satisfaction.

[0061] 1. Questionnaire response

[0062] Users (employees) access a survey form and answer questions about the work environment.

[0063] The device sends the response data to the server.

[0064] 2. Storing in a database

[0065] The server stores the received survey data in a database.

[0066] 3. Data Cleaning

[0067] The server checks for missing or outlier values ​​and fills in the data as needed.

[0068] 4. AI-based filtering

[0069] The server passes the text response to a generative artificial intelligence system, which then classifies its content as positive or negative.

[0070] 5. Summary and Analysis of Results

[0071] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[0072] 6. Report generation and notification

[0073] The server generates a report containing the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[0074] In this way, this system can efficiently collect, preprocess, and analyze survey results from customers and employees, and provide the results as reports.

[0075] The following describes the processing flow.

[0076] Step 1:

[0077] The user fills out the survey form.

[0078] Users (customers or employees) respond to surveys via online forms or applications.

[0079] Enter your answers for each question, and then click the "Submit" button.

[0080] Step 2:

[0081] The device sends data to the server.

[0082] The device (PC or smartphone) collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[0083] The data sent is often in formats such as JSON or XML.

[0084] Step 3:

[0085] The server receives the data.

[0086] The server temporarily stores the received survey data in a buffer.

[0087] Next, the data is converted into a format suitable for storage in the database.

[0088] Step 4:

[0089] The server saves data to the database.

[0090] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[0091] The inserted data is recorded in the table.

[0092] Step 5:

[0093] The server cleans up the data.

[0094] The server reads the stored data and cleans it.

[0095] Specifically, missing values ​​are detected and either imputed with the mean or removed as invalid data. Outliers are checked and corrected in the same way.

[0096] Step 6:

[0097] The server passes data to the generative artificial intelligence.

[0098] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[0099] The data to be submitted will include, in particular, open-ended text data.

[0100] Step 7:

[0101] Generative artificial intelligence filters the data.

[0102] Generative artificial intelligence analyzes received data and filters it according to specific criteria.

[0103] For example, classify users' emotions as either "positive" or "negative."

[0104] Step 8:

[0105] The server receives the results from the generative artificial intelligence.

[0106] The server receives the filtering results returned by the generative artificial intelligence.

[0107] The received data is stored back into the buffer.

[0108] Step 9:

[0109] The server aggregates the filtered data.

[0110] The server extracts the filtering results stored in the buffer and performs statistical aggregation.

[0111] For example, count the number of positive and negative responses and calculate the percentage of each.

[0112] Step 10:

[0113] The server analyzes the data.

[0114] The server analyzes the aggregated data and performs detailed statistical analysis.

[0115] For example, you could calculate the average satisfaction score for each department and identify which departments receive high ratings.

[0116] Step 11:

[0117] The server generates the report.

[0118] The server generates a report based on the analysis results.

[0119] The report should be presented in a visually clear format, including text summaries, graphs, and charts.

[0120] Step 12:

[0121] The server notifies the administrator of the report.

[0122] The server sends a notification to the administrator's terminal to send the generated report.

[0123] The notification should include a link to the report, making it easily accessible to administrators.

[0124] The above explains in detail the program processing of this system, which includes a series of processes from collecting survey data from users, filtering and analyzing it using generative artificial intelligence, and notifying administrators of reports.

[0125] (Example 1)

[0126] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0127] Traditional survey analysis systems faced challenges such as the time and effort required for data preprocessing, sentiment classification, result analysis, and report generation. Furthermore, they suffered from accuracy issues in sentiment classification and insufficient visualization of analysis results. This made it difficult for administrators to quickly and accurately obtain useful information for making appropriate decisions.

[0128] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0129] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for aggregating the filtering results and analyzing the proportion of positive and negative emotions, and means for generating a report including a visual chart based on the analysis results and notifying the administrator or relevant parties. This automates everything from data pre-processing to emotion classification, result analysis, and visual report generation, enabling administrators to obtain information quickly and accurately.

[0130] "Customer or employee survey results" refers to data from surveys conducted by a company or organization among its customers or employees.

[0131] "Means of collection" refers to the technical means of receiving survey results and storing them in a database or storage.

[0132] "Methods for cleaning and preprocessing" refer to processes and technical means for removing missing or outlier values ​​from collected data and converting it into an analyzable format.

[0133] "Generative artificial intelligence" refers to AI models used to perform tasks such as natural language processing and sentiment analysis.

[0134] "Means of filtering" refers to technical means of inputting pre-processed data into a generative artificial intelligence system and classifying or selecting the data according to specific criteria.

[0135] "Means of aggregation and analysis" refer to technical means of statistically aggregating filtered data and analyzing patterns and trends.

[0136] A "report including visual charts" is a report that presents analysis results in a visual format such as bar graphs or pie charts.

[0137] "Means of notifying administrators or stakeholders" refers to technical means of notifying stakeholders of generated reports via email or dashboards.

[0138] "Methods for imputing missing values" refer to technical means of filling in missing values ​​in a dataset based on statistical methods or known data.

[0139] "Methods for correcting outliers" refer to technical means that detect values ​​in a dataset that deviate significantly from the normal range and correct them to bring them back within an appropriate range.

[0140] This invention relates to a system that efficiently collects survey results from customers or employees, performs preprocessing, filtering using generative artificial intelligence, analysis, and report generation. In this system, the entire process, from data collection to report notification, is automated.

[0141] System configuration and hardware / software used

[0142] 1. Data Collection

[0143] Users access a dedicated survey form and enter their answers to the questions. This survey form operates on a web browser (e.g., Google Chrome®, Mozilla Firefox) or a dedicated application.

[0144] The terminal transmits the survey data entered by the user to the server in real time. The HTTPS protocol is used for this communication to ensure security.

[0145] 2. Data preprocessing

[0146] The server stores the received survey data in a temporary storage area and then formally saves it to a database (e.g., MySQL®, PostgreSQL).

[0147] The server checks the data extracted from the database for missing or outlier values ​​and cleans it as needed. Missing values ​​are imputed using the median or mean, and outliers are detected and a warning is issued.

[0148] 3. Filtering using generative artificial intelligence

[0149] The server passes the cleaned data to a generative artificial intelligence (e.g., OpenAI® GP T-3) for sentiment classification. Specifically, it classifies the data as "positive" or "negative" through text analysis.

[0150] The classification results returned by the generative artificial intelligence are saved to the server, and metadata necessary for analysis (analysis date and time, model version, etc.) is also added.

[0151] 4. Summary and Analysis of Results

[0152] The server aggregates and analyzes the filtered data. This includes calculating the percentage of positive and negative responses, and performing detailed statistical analysis on specific questions.

[0153] The analytical methods used include regression analysis and clustering.

[0154] 5. Report generation and notification

[0155] The server generates a report that includes visual charts (e.g., bar graphs, pie charts) based on the analysis results.

[0156] The server notifies administrators or relevant parties of the generated reports. These notifications are sent via email or a dedicated dashboard.

[0157] Examples of specific cases and prompt statements

[0158] For example, if a company conducts a survey to investigate employee satisfaction, this system will operate as follows:

[0159] Users (employees) access a survey form and answer questions about the work environment.

[0160] The device sends the response data to the server.

[0161] The server stores the received survey data in a database and performs checks and imputations for missing or outlier values.

[0162] The server passes the text responses to a generative artificial intelligence system, which then classifies their content as positive or negative.

[0163] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[0164] The server generates a report containing the analysis results and notifies the administrator.

[0165] Examples of prompts to input into a generative AI model:

[0166] Please classify the following open-ended text data as positive or negative.

[0167] Answer 1: "The workplace atmosphere is very good, but the workload is too much."

[0168] Answer 2: "My boss is kind and helpful. I am very satisfied."

[0169] Answer 3: "The work is boring and unstimulating. Improvement is needed."

[0170] In this way, the system efficiently collects, preprocesses, and analyzes survey results, and provides the results as a report.

[0171] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0172] Step 1:

[0173] User survey input

[0174] Users access a dedicated survey form and answer the questions. This survey form operates on a web browser (e.g., Google Chrome, Mozilla Firefox) or a dedicated application.

[0175] Input: User survey response data

[0176] Output: Survey response data sent to the server via the terminal.

[0177] Step 2:

[0178] Sending data from the terminal to the server

[0179] The terminal sends the survey data entered by the user to the server using the HTTPS protocol.

[0180] Input: User survey response data

[0181] Output: Survey response data received by the server

[0182] Step 3:

[0183] Data reception and temporary storage by the server

[0184] The server receives the survey data sent from the terminal and saves it to a temporary storage area. A database system (e.g., MySQL, PostgreSQL) is used for this storage.

[0185] Input: Survey response data sent from the device.

[0186] Output: Data stored in the temporary storage area

[0187] Step 4:

[0188] Data preprocessing by the server

[0189] The server extracts survey data from the database and checks for missing or outlier values. Missing values ​​are imputed using the median or mean, and outliers are detected and corrected.

[0190] Input: Data from temporary storage area

[0191] Output: Cleaned and pre-processed data

[0192] Step 5:

[0193] Data transmission and filtering by the server to the generative artificial intelligence.

[0194] The server passes the pre-processed data to the generative artificial intelligence. The generative AI uses the prompt "Classify the following text as positive or negative" to categorize the emotions.

[0195] Input: Cleaned and pre-processed data

[0196] Output: Filtering results (positive, negative) returned by the generative artificial intelligence.

[0197] Step 6:

[0198] Server saving of filtering results

[0199] The server stores the filtering results received from the generative artificial intelligence in a database. At this time, metadata such as the analysis date and time and model version are also recorded.

[0200] Input: Filtered results from generative artificial intelligence

[0201] Output: Filtering results stored in the database

[0202] Step 7:

[0203] Server-based aggregation and analysis of filtering results

[0204] The server aggregates the filtering results and calculates the percentage of positive and negative responses. It also performs detailed statistical analysis for specific questions and departments.

[0205] Input: Database containing filtered results

[0206] Output: Aggregated and analyzed data

[0207] Step 8:

[0208] Server-based report generation and notification

[0209] The server generates a report containing visual charts from the aggregated and analyzed results. The generated report is notified to administrators or relevant parties. Notifications are made via email or a dedicated dashboard.

[0210] Input: Aggregated and analyzed data

[0211] Output: Report notified to administrator or relevant party

[0212] Through the steps described above, this system can efficiently automate and perform a series of processes, from data collection to report notification.

[0213] (Application Example 1)

[0214] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0215] Traditional survey systems struggle not only to collect feedback from customers and employees, but also to efficiently clean, analyze, and quickly implement countermeasures based on that feedback. Furthermore, the time-consuming process of data aggregation and analysis makes it difficult to immediately reflect the insights gained from the feedback. Therefore, there is a need for real-time service improvement and rapid problem resolution.

[0216] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0217] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, and means for sending the generated report to an administrator using electronic messages. This makes it possible to efficiently carry out everything from collecting and analyzing survey results to generating and notifying reports.

[0218] "Customer or employee" refers to a user who provides survey responses to this system.

[0219] "Survey results" refers to the content of responses given by customers or employees to a survey.

[0220] "Means of collection" refers to the methods and equipment used to obtain survey results and transmit them to a server.

[0221] "Methods for cleaning and preprocessing" refer to methods and equipment that fill in missing or outlier values ​​in collected survey data, making the data analyzable.

[0222] "Generative artificial intelligence" refers to intelligent systems that analyze text data using natural language processing and machine learning techniques.

[0223] "Means of filtering" refers to methods and devices that use generative artificial intelligence to analyze survey data, classify information, or extract information based on specific conditions.

[0224] "Means for aggregating filtering results and performing analysis based on them" refers to methods and equipment for compiling the results analyzed by generative artificial intelligence and performing statistical analysis and evaluation.

[0225] "Means of generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for organizing analysis results, creating reports in visual or text format, and informing administrators or relevant parties via email or other means.

[0226] "Means of sending to an administrator using electronic messages" refers to methods or devices for sending generated reports to an administrator via email or other digital communication means.

[0227] The present invention is a system for efficiently collecting survey results from customers or employees, cleaning, filtering using generative artificial intelligence, and analyzing the results, and finally generating a report. This system includes means for collecting survey results, means for preprocessing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[0228] System Overview

[0229] 1. Data Collection

[0230] Users (customers or employees) answer the survey via a smartphone application. This survey form operates on a web browser or a dedicated application. The device sends this input data to the server, where it is received and initially stored.

[0231] 2. Data preprocessing

[0232] The server stores the collected survey data in a database. It then extracts the data for preprocessing. The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[0233] 3. Filtering using generative artificial intelligence

[0234] The server passes the cleaned data to a generative artificial intelligence (AI). This AI analyzes, for example, open-ended text data and classifies the emotions expressed as "positive" or "negative." The generative AI returns the filtered results, which are then stored again on the server.

[0235] 4. Summary and Analysis of Results

[0236] The server aggregates the filtering results and performs detailed analysis based on them. For example, it calculates the percentage of positive and negative responses. Furthermore, it also performs statistical analysis for specific questions or by department.

[0237] 5. Report generation and notification

[0238] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts. The server sends the generated report to the administrator via electronic message.

[0239] Specific examples

[0240] For example, consider a case where a customer satisfaction survey is conducted at a physical store. In this survey, customers answer questions about things like "the atmosphere of the store" and "the service provided by the staff."

[0241] 1. Questionnaire response

[0242] Users (customers) access a smartphone app and answer questions about the store's atmosphere and staff service.

[0243] 2. Storing in a database

[0244] The terminal sends the response data to the server. The server stores the received survey data in a database.

[0245] 3. Data Cleaning

[0246] The server checks for missing or outlier values ​​and fills in the data as needed.

[0247] 4. AI-based filtering

[0248] The server passes the text responses to a generative artificial intelligence system, which then classifies the content as either "positive" or "negative." An example of a prompt is: "We'd love to hear your feedback: How did you feel about the service provided by our staff?"

[0249] 5. Summary and Analysis of Results

[0250] The server aggregates the classification results and analyzes trends in customer satisfaction with the store's atmosphere, for example.

[0251] 6. Report generation and notification

[0252] The server generates a report containing the analysis results and notifies the administrator via email. This report is used to consider ways to improve store services.

[0253] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0254] Step 1:

[0255] Users access the survey form and answer the questions. Specifically, customers or employees use a smartphone application to answer questions entered into the survey form using text or multiple-choice options. The entered data (e.g., free-text responses, multiple-choice answers, etc.) is reflected in the survey form.

[0256] Step 2:

[0257] The terminal sends the survey results entered by the user to the server. The transmitted data is received by the server for subsequent processing and initially stored in the database. The input is the survey response data, and the output is the raw data stored in the database.

[0258] Step 3:

[0259] The server preprocesses the survey data stored in the database. Specifically, it imputes missing values ​​and corrects outliers. This results in cleaned data. The input is the stored raw data, and the output is the cleaned data.

[0260] Step 4:

[0261] The server passes the cleaned data to a generative artificial intelligence (AI) for filtering. The AI ​​uses a natural language processing model to analyze the text data and classify the emotion of each response as either "positive" or "negative." In this process, the generative AI model receives text data as input and outputs the result of the emotion classification. The input is the cleaned data, and the output is the result of the emotion classification.

[0262] Step 5:

[0263] The server aggregates data classified by generative artificial intelligence and performs overall trend analysis and statistical analysis. Specifically, it calculates the percentage of positive and negative responses. The input is the result data of sentiment classification, and the output is the aggregated and statistically analyzed results.

[0264] Step 6:

[0265] The server generates a report based on the aggregated and analyzed results and sends it to the administrator via electronic message. The report includes visual information such as graphs and charts. Specifically, the process involves organizing the analysis results, converting them into a visual format, and sending them to the administrator via email. The input is the aggregated and analyzed results, and the output is the report sent via email.

[0266] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0267] This invention is a system that collects survey results from customers or employees, cleans the data, filters it using generative artificial intelligence, and combines it with an emotion engine to efficiently analyze the data and generate reports.

[0268] System Overview

[0269] This system utilizes an emotion engine, in particular, to recognize user emotions, streamlining the entire process from survey data collection and analysis to report generation. The system includes the following:

[0270] 1. Data collection methods:

[0271] A method for collecting survey results from users. This is done via online forms or dedicated apps.

[0272] 2. Pretreatment means:

[0273] Means for cleaning the collected data and performing pretreatment. This includes complementing missing values and correcting outliers.

[0274] 3. Filtering means by generative artificial intelligence:

[0275] Means for passing the preprocessed data to generative artificial intelligence and causing sentiment classification to be performed.

[0276] 4. Sentiment engine:

[0277] Means for analyzing the voice, expression, and text data during the user's questionnaire response and recognizing sentiment in real time.

[0278] 5. Analysis means:

[0279] Means for aggregating the filtered data and performing analysis based on it.

[0280] 6. Report generation and notification means:

[0281] Means for generating a report based on the analysis results and notifying the administrator or relevant personnel.

[0282] Program processing

[0283] Describing the program processing in natural language is as follows:

[0284] 1. Data collection

[0285] The user accesses the questionnaire form and answers each question. This form operates on a web browser or an application.

[0286] The terminal collects the data input by the user and transmits it to the server.

[0287] ]> ​2. Emotional analysis using an emotion engine

[0288] The device uses its camera and microphone to capture facial expressions and voice while the user is responding, and transmits this information to the emotion engine in real time.

[0289] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state.

[0290] 3. Receiving and cleaning data

[0291] The server stores the received survey data in a database and performs preprocessing. During this process, missing values ​​are imputed and outliers are corrected.

[0292] 4. Filtering using generative artificial intelligence

[0293] The server passes the pre-processed data to a generative artificial intelligence system, which performs sentiment classification of the text data. The system then receives the results, categorized as positive, negative, neutral, etc.

[0294] 5. Summary and Analysis of Results

[0295] The server aggregates the filtering results and performs a detailed analysis. For example, it calculates the percentage of positive and negative responses. Real-time sentiment data from the sentiment engine is also used in the analysis.

[0296] 6. Report generation and notification

[0297] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements.

[0298] The server notifies the administrator of the reports it generates, making them easily accessible to the administrator.

[0299] Specific example

[0300] For example, when conducting an employee satisfaction survey:

[0301] 1. Questionnaire response

[0302] The user answers the questionnaire items. At the same time, the user's facial expression and voice data are acquired using the camera and microphone.

[0303] 2. Real-time emotion analysis

[0304] The emotion engine analyzes the facial expressions and voices in real time to recognize the user's emotional state.

[0305] 3. Storage and cleaning in the database

[0306] The server stores the received questionnaire data in the database and cleans the missing values and outliers.

[0307] 4. Filtering by generative AI

[0308] The server passes the text data to the generative AI for emotion classification. For example, it classifies into "satisfied", "dissatisfied", etc.

[0309] 5. Aggregation and analysis of results

[0310] The server aggregates the filtering results and the analysis results of the emotion engine, and analyzes the trends for each department and the overall satisfaction.

[0311] 6. Report generation and notification

[0312] The server creates a report based on the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[0313] As described above, this system can provide more detailed and reliable insights by recognizing the user's emotions in real time and using them for the analysis of the questionnaire results.

[0314] The following describes the processing flow.

[0315] Step 1:

[0316] The user fills out the survey form.

[0317] Users (customers or employees) answer survey questions via online forms or applications.

[0318] Enter detailed answers for each question and click the "Submit" button.

[0319] Step 2:

[0320] The device sends data to the server.

[0321] The device collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[0322] The data is sent to the server in formats such as JSON or XML.

[0323] Step 3:

[0324] The device acquires the user's voice and facial expression data.

[0325] The device uses its camera and microphone to capture the user's real-time voice and facial expression data.

[0326] This data is sent to the emotion engine.

[0327] Step 4:

[0328] The emotion engine analyzes voice and facial expression data.

[0329] The emotion engine analyzes acquired voice and facial expression data to recognize the user's emotional state (e.g., positive, negative, neutral).

[0330] Send the recognition results to the server.

[0331] Step 5:

[0332] The server receives and stores the survey data.

[0333] The server stores the survey data sent from the terminal in a buffer for temporary storage.

[0334] Convert the data to a format suitable for storage in a database.

[0335] Step 6:

[0336] The server saves data to the database.

[0337] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[0338] The inserted data is recorded in the survey data table.

[0339] Step 7:

[0340] The server cleans the survey data.

[0341] The server reads the stored data and cleans it.

[0342] Missing values ​​are detected and either imputed with the mean or excluded as invalid data. Outliers are checked and corrected in the same way.

[0343] Step 8:

[0344] The server passes data to the generative artificial intelligence.

[0345] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[0346] In particular, submit the text data of your open-ended responses.

[0347] Step 9:

[0348] Generative artificial intelligence filters the data.

[0349] Generative artificial intelligence analyzes received data and classifies emotions according to specific criteria.

[0350] For example, text responses are classified as either "positive" or "negative."

[0351] Step 10:

[0352] The server receives the results from the generative artificial intelligence.

[0353] The server receives the filtering results returned by the generative artificial intelligence.

[0354] The received data is stored back into the buffer.

[0355] Step 11:

[0356] The server aggregates the filtering results.

[0357] The server extracts filtered data and performs statistical aggregation.

[0358] Count the number of positive and negative responses, and calculate the percentage of each.

[0359] Step 12:

[0360] The server integrates the results from the emotion engine.

[0361] The server integrates the real-time emotion recognition results received from the emotion engine with the analysis results of the survey data.

[0362] This means that not only text responses but also the user's emotional state during the response process will be used for analysis.

[0363] Step 13:

[0364] The server analyzes the data.

[0365] The server performs detailed statistical analysis based on the aggregated results.

[0366] For example, calculate the average satisfaction score for each department and analyze specific demographic factors.

[0367] Step 14:

[0368] The server generates the report.

[0369] The server generates a report based on the analysis results.

[0370] The report should include visually clear formats such as text summaries, graphs, and charts.

[0371] Step 15:

[0372] The server notifies the administrator of the report.

[0373] The server will notify the administrator of the generated report via email or a dashboard.

[0374] Include a link that allows administrators to access the report.

[0375] In summary, the system's program processing is executed as a series of processes, from collecting survey data, to real-time analysis by the emotion engine, filtering by generative artificial intelligence, data analysis, and finally, sending reports to administrators.

[0376] (Example 2)

[0377] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0378] Traditional survey systems struggled to accurately grasp users' emotional states, making detailed emotion-based analysis difficult. Furthermore, missing or outlier data in the collected responses reduced the accuracy of the analysis. While generative artificial intelligence-based emotion classification was employed, the lack of real-time emotion analysis undermined the reliability of the results.

[0379] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0380] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial expressions and voice using an emotion engine that recognizes the emotional state of the user at the time of response, means for aggregating the filtering results and performing analysis based on them, and means for generating a report based on the analysis results and notifying the administrator or relevant parties. This enables detailed and reliable data analysis that incorporates real-time emotion analysis of the user.

[0381] "Customer or employee" refers to an individual who uses the system to respond to a survey.

[0382] "Survey results" refer to the response data entered by customers or employees through a survey form.

[0383] "Means of collection" refers to methods and devices for obtaining survey results from users via online forms, dedicated applications, etc.

[0384] "Methods for cleaning and preprocessing" refer to data preprocessing techniques and methods for imputing missing values ​​and correcting outliers in collected survey data.

[0385] "Generative artificial intelligence" refers to artificial intelligence models that analyze collected text data and perform sentiment classification and other classifications.

[0386] "Means for performing filtering" refers to methods or devices for passing pre-processed data to a generative artificial intelligence system to perform filtering, such as classifying the sentiment of the data.

[0387] An "emotion engine" is a system or software that analyzes the user's facial expressions and voice data in real time when they respond to questions, and recognizes the user's emotional state.

[0388] "Means of aggregating and performing analysis based on that data" refers to technologies and methods for aggregating results obtained from emotion engines and generative artificial intelligence, and then performing detailed analysis based on that data.

[0389] "Means for generating reports and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating those reports to administrators or relevant parties via email or notification systems.

[0390] This invention is a system that collects survey results from customers or employees, performs data cleaning, filters using generative artificial intelligence (AI models), and combines this with an emotion engine. This allows for efficient data analysis and report generation.

[0391] The system uses the following hardware and software:

[0392] Web browser or dedicated application: Software on a device used by the user to answer the survey.

[0393] Server: The central computer responsible for data collection, database storage, data preprocessing, analysis using generative AI models, emotion engine management, aggregation, analysis, and report generation.

[0394] Database: A system for storing collected survey data and analysis results.

[0395] Emotion Engine: Software that processes facial and voice data obtained from cameras and microphones in order to analyze the user's emotions in real time.

[0396] Generative artificial intelligence: Natural language processing (NLP) models used to classify emotions from survey responses. Examples include AI models such as GPT-4 (registered trademark).

[0397] Python's Pandas library: Used for data cleaning (imputing missing values ​​and correcting outliers).

[0398] Matplotlib and ReportLab are used to generate reports that include graphs and charts.

[0399] Specific examples of implementation

[0400] For example, let's consider the case of conducting an employee satisfaction survey.

[0401] 1. Users access the survey form via a web browser or dedicated application and answer the survey. At this time, the device's camera and microphone are turned on to capture facial expressions and voice data.

[0402] 2. The terminal transmits the user's input and acquired facial expression / voice data to the server. The HTTPS protocol is used in this process.

[0403] 3. The server saves the received survey data to a database and preprocesses the data using the Python Pandas library. Preprocessing includes imputing missing values ​​and correcting outliers.

[0404] 4. The pre-processed data is input into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. For example, responses are classified as positive, negative, or neutral.

[0405] 5. The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server.

[0406] 6. The server aggregates the filtering results from the generative artificial intelligence and the data obtained from the emotion engine, and performs a detailed analysis. This analysis includes the percentage of positive responses, the percentage of negative responses, etc.

[0407] 7. The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, etc. These are generated using Python's Matplotlib or ReportLab.

[0408] 8. The server notifies the administrator of the generated report. Email or a dedicated notification system may be used for notification.

[0409] Examples of prompts for generative AI models

[0410] "Classify the emotions from the submitted survey responses into positive, negative, and neutral, and calculate the ratio of each. Furthermore, perform a comprehensive emotional analysis considering facial expression and voice data, and output the results as a report."

[0411] The above describes the embodiments for carrying out the present invention. This system achieves more detailed and reliable data analysis by incorporating real-time sentiment analysis of the user.

[0412] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0413] Step 1:

[0414] Data collection

[0415] Users access the survey form via a web browser or a dedicated application and fill it out. This includes answering questions and providing free-form text.

[0416] The terminal collects survey data entered by the user and sends it to the server. The data is structured in JSON format and sent using an HTTP POST request.

[0417] Input: User survey response data (text).

[0418] Output: Structured data (JSON format) sent to the server.

[0419] Step 2:

[0420] Real-time sentiment analysis

[0421] The device uses its camera and microphone to capture facial expressions and voice while the user is filling out a survey, and transmits this information to the emotion engine in real time.

[0422] The emotion engine analyzes the user's voice and facial expression data to recognize their emotional state (e.g., joy, sadness, anger). This analysis uses voice emotion recognition algorithms and facial expression recognition algorithms.

[0423] Input: User's facial expression data, voice data.

[0424] Output: Analyzed emotion data (e.g., joy, sadness, anger, etc.).

[0425] Step 3:

[0426] Receiving and cleaning data

[0427] The server receives survey data sent from the terminal and stores it in the database. Afterward, it performs data preprocessing. This preprocessing includes imputing missing values ​​and correcting outliers. The data is cleaned using the Python Pandas library.

[0428] Input: Survey data sent from the device.

[0429] Output: Clean data after preprocessing.

[0430] Step 4:

[0431] Filtering by generative artificial intelligence

[0432] The server inputs pre-processed data into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. The model analyzes the text data and classifies each response as positive, negative, neutral, etc.

[0433] Input: Clean text data.

[0434] Output: Classified sentiment data (positive, negative, neutral).

[0435] Step 5:

[0436] Summary and analysis of results

[0437] The server aggregates data obtained from generative artificial intelligence and emotion engines and performs detailed analysis. Specifically, it calculates the percentage of positive and negative responses. This analysis uses SQL queries and Python libraries (e.g., Pandas, Matplotlib).

[0438] Input: Classified sentiment data, real-time analyzed sentiment data.

[0439] Output: Aggregated results and detailed analysis results (e.g., positive response rate).

[0440] Step 6:

[0441] Report generation and notification

[0442] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements. Python's ReportLab and Matplotlib are used to generate the report. The generated report is notified to the administrator via email or a dedicated notification system.

[0443] Input: Detailed analysis results.

[0444] Output: Generated report (PDF or HTML format), notification email.

[0445] (Application Example 2)

[0446] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0447] Traditional survey systems often fail to adequately consider emotional factors when collecting feedback from customers and employees, resulting in superficial analysis of the results. Furthermore, data cleaning and filtering are frequently performed manually, making efficient data analysis difficult. Consequently, it becomes challenging to quickly and accurately obtain information that is useful to managers and stakeholders. In store feedback systems, there is a need for methods that analyze customer emotions in real time and classify and analyze emotional data using generative AI.

[0448] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0449] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial and voice data collected during pre-processing to recognize real-time emotions, means for aggregating the filtering results and real-time emotion data and performing analysis based on them, and means for generating a report based on the analysis results and notifying administrators or relevant parties. This enables detailed and reliable feedback analysis that takes customer emotions into account, allowing administrators and relevant parties to quickly obtain useful information.

[0450] "Means of collecting survey results" refers to methods or equipment used to collect responses from customers or employees.

[0451] "Methods for cleaning and preprocessing data" refer to methods and techniques for preparing collected data for analysis by imputing or correcting missing or outlier values.

[0452] "Methods for passing data to a generative artificial intelligence to perform filtering" refer to methods or systems for inputting pre-processed data into a generative artificial intelligence and having it classify it into appropriate emotions or categories.

[0453] "Means for analyzing facial and voice data to recognize emotions in real time" refers to technologies and devices that use collected facial and voice data to analyze and recognize a user's emotional state in real time.

[0454] "Means for aggregating data and performing analysis based on it" refers to methods and systems that integrate and aggregate filtered data and real-time sentiment data to perform statistical and sentimental analysis.

[0455] "Means for generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating them to administrators or relevant parties.

[0456] This invention is an advanced system for efficiently collecting and analyzing feedback from customers or employees. This system operates using a combination of hardware and software and includes the following means:

[0457] hardware

[0458] 1. Smart devices

[0459] Smartphones and tablets are used for collecting survey responses. These devices have built-in cameras and microphones, allowing them to collect user facial expressions and audio data.

[0460] 2. Server

[0461] A server equipped with a database and analysis engine. This enables data preprocessing, filtering by generative artificial intelligence, real-time sentiment analysis, data aggregation and analysis, and report generation and notification.

[0462] software

[0463] 1. Emotional Engine

[0464] The system analyzes the user's emotions in real time using facial expression analysis libraries (e.g., "OpenFace," "Affectiva") and voice emotion analysis software (e.g., "IBM Watson® Tone Analyzer").

[0465] 2. Generative Artificial Intelligence

[0466] Generative artificial intelligence such as BERT and GPT-3(registered trademark)(OpenAI) will be used to classify the sentiment of text data from survey responses.

[0467] 3. Database

[0468] We use MySQL or PostgreSQL to store and manage the collected and pre-processed data.

[0469] 4. Analytics

[0470] We will use Python and its libraries (e.g., pandas, numpy, matplotlib) to analyze data and generate reports.

[0471] 5. Notification System

[0472] Reports are sent to administrators and relevant parties using email services (e.g., AWS® SES) or messaging services (e.g., Twilio).

[0473] System operation

[0474] Data collection and cleaning

[0475] The terminal collects survey responses from customers and employees. In addition to the text data entered by the user, it also collects facial expressions and voice data using the camera and microphone. The collected data is sent to a server where missing values ​​are imputed and outliers are corrected.

[0476] Filtering by generative artificial intelligence

[0477] The pre-processed data is passed to a generative artificial intelligence (AI) for emotion classification. The AI ​​classifies the data into positive, negative, neutral, etc.

[0478] Real-time sentiment analysis

[0479] The collected facial and voice data is analyzed in real time by an emotion engine to recognize the user's emotional state.

[0480] Data aggregation and analysis

[0481] The filtering results and real-time sentiment data are integrated by the server for detailed analysis. For example, by statistically analyzing the satisfaction and dissatisfaction levels of each product, areas for improvement in the store can be identified.

[0482] Report generation and notification

[0483] Based on the analysis results, a report is generated that includes text summaries, graphs, charts, and other elements. This report is then sent to administrators and relevant parties through a notification system.

[0484] Specific example

[0485] Examples of prompt statements include the following:

[0486] Please generate an emotional analysis report for product "A". Please perform the analysis based on the following data set. We would like to know the ratio of positive, negative, and neutral responses.

[0487] In this way, this system enables detailed feedback analysis, including the actual emotions of customers, allowing managers and stakeholders to quickly identify areas for improvement.

[0488] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0489] Step 1:

[0490] Data collection

[0491] When collecting survey results entered by customers or employees, the device also uses its camera and microphone to simultaneously acquire facial expressions and voice data.

[0492] Input: Customer or employee survey text, facial expression data, audio data

[0493] Output: Collected questionnaire text, facial expression data, audio data

[0494] Step 2:

[0495] Data transmission

[0496] The device transmits the collected survey text, facial expression data, and audio data to the server.

[0497] Input: Collected questionnaire text, facial expression data, audio data

[0498] Output: Survey text, facial expression data, and audio data sent to the server.

[0499] Step 3:

[0500] Data cleaning and preprocessing

[0501] The server fills in missing values ​​and corrects outliers from the received data. For example, it calculates fill-in values ​​for questions with missing answers and corrects extremely abnormal values.

[0502] Input: Survey text, facial expression data, and audio data sent from the device.

[0503] Output: Cleaned and pre-processed questionnaire text, facial expression data, audio data

[0504] Step 4:

[0505] Filtering by generative artificial intelligence

[0506] The server inputs pre-processed data into a generative artificial intelligence (e.g., GPT-3) to perform sentiment classification. The generative AI classifies the data into positive, negative, or neutral and returns the result.

[0507] Input: Preprocessed survey text

[0508] Output: Emotionally classified survey data (positive, negative, neutral)

[0509] Step 5:

[0510] Real-time sentiment analysis

[0511] The server analyzes facial and voice data collected using facial expression analysis libraries and voice emotion analysis software in real time to recognize the user's emotional state.

[0512] Input: Preprocessed facial expression data and audio data

[0513] Output: Recognized real-time sentiment data

[0514] Step 6:

[0515] Data aggregation and analysis

[0516] The server integrates and aggregates the filtering results and real-time sentiment data, and performs statistical and emotional analysis. For example, it calculates the percentage of positive emotions, negative emotions, and neutral emotions.

[0517] Input: Sentiment-classified survey data, real-time sentiment data

[0518] Output: Aggregated and analyzed results (e.g., sentiment ratio, trend analysis, etc.)

[0519] Step 7:

[0520] Report generation

[0521] The server generates a report that includes text summaries, graphs, and charts based on the aggregated and analyzed results.

[0522] Input: Aggregation and analysis results

[0523] Output: Report (text summary, graphs, charts, etc.)

[0524] Step 8:

[0525] notification

[0526] The server uses email or messaging services to notify administrators or relevant parties of the reports it generates.

[0527] Input: Generated report

[0528] Output: Notification to administrators or relevant parties (email, message, etc.)

[0529] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0530] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0531] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0532] [Second Embodiment]

[0533] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0534] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0535] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0536] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0537] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0538] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0539] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0540] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0541] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0542] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0543] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0544] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0545] This invention is a system that efficiently collects survey results from customers or employees, cleans them, filters them using generative artificial intelligence, and analyzes them, ultimately generating a report.

[0546] System Overview

[0547] This system includes means for collecting survey results, means for pre-processing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[0548] Program processing

[0549] The program's processing can be explained in natural language as follows:

[0550] 1. Data Collection

[0551] Users enter information through a survey form. This survey form operates on a web browser or a dedicated application.

[0552] The terminal sends this input data to the server. The transmitted data is received by the server and initially stored.

[0553] 2. Data preprocessing

[0554] The server stores the collected survey data in a database. Then, it extracts the data for preprocessing.

[0555] The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[0556] 3. Filtering using generative artificial intelligence

[0557] The server passes the cleaned data to a generative artificial intelligence (AI). This AI, for example, analyzes open-ended text data and classifies the emotions expressed as "positive" or "negative."

[0558] The generative artificial intelligence returns the filtered results. These results are then saved again on the server side.

[0559] 4. Summary and Analysis of Results

[0560] The server aggregates the filtering results and performs a detailed analysis based on them. For example, it calculates the percentage of positive responses and the percentage of negative responses.

[0561] Furthermore, statistical analysis will be conducted on specific question items and by department.

[0562] 5. Report generation and notification

[0563] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts.

[0564] The server generates reports and notifies the administrator. These notifications are sent via email or a dashboard.

[0565] Specific example

[0566] For example, consider a case where a company conducts a survey to investigate employee satisfaction. In this survey, employees answer questions about their work environment and job satisfaction.

[0567] 1. Questionnaire response

[0568] Users (employees) access a survey form and answer questions about the work environment.

[0569] The device sends the response data to the server.

[0570] 2. Storing in a database

[0571] The server stores the received survey data in a database.

[0572] 3. Data Cleaning

[0573] The server checks for missing or outlier values ​​and fills in the data as needed.

[0574] 4. AI-based filtering

[0575] The server passes the text response to a generative artificial intelligence system, which then classifies its content as positive or negative.

[0576] 5. Summary and Analysis of Results

[0577] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[0578] 6. Report generation and notification

[0579] The server generates a report containing the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[0580] In this way, this system can efficiently collect, preprocess, and analyze survey results from customers and employees, and provide the results as reports.

[0581] The following describes the processing flow.

[0582] Step 1:

[0583] The user fills out the survey form.

[0584] Users (customers or employees) respond to surveys via online forms or applications.

[0585] Enter your answers for each question, and then click the "Submit" button.

[0586] Step 2:

[0587] The device sends data to the server.

[0588] The device (PC or smartphone) collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[0589] The data sent is often in formats such as JSON or XML.

[0590] Step 3:

[0591] The server receives the data.

[0592] The server temporarily stores the received survey data in a buffer.

[0593] Next, the data is converted into a format suitable for storage in the database.

[0594] Step 4:

[0595] The server saves data to the database.

[0596] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[0597] The inserted data is recorded in the table.

[0598] Step 5:

[0599] The server cleans up the data.

[0600] The server reads the stored data and cleans it.

[0601] Specifically, missing values ​​are detected and either imputed with the mean or removed as invalid data. Outliers are checked and corrected in the same way.

[0602] Step 6:

[0603] The server passes data to the generative artificial intelligence.

[0604] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[0605] The data to be submitted will include, in particular, open-ended text data.

[0606] Step 7:

[0607] Generative artificial intelligence filters the data.

[0608] Generative artificial intelligence analyzes received data and filters it according to specific criteria.

[0609] For example, classify users' emotions as either "positive" or "negative."

[0610] Step 8:

[0611] The server receives the results from the generative artificial intelligence.

[0612] The server receives the filtering results returned by the generative artificial intelligence.

[0613] The received data is stored back into the buffer.

[0614] Step 9:

[0615] The server aggregates the filtered data.

[0616] The server extracts the filtering results stored in the buffer and performs statistical aggregation.

[0617] For example, count the number of positive and negative responses and calculate the percentage of each.

[0618] Step 10:

[0619] The server analyzes the data.

[0620] The server analyzes the aggregated data and performs detailed statistical analysis.

[0621] For example, you could calculate the average satisfaction score for each department and identify which departments receive high ratings.

[0622] Step 11:

[0623] The server generates the report.

[0624] The server generates a report based on the analysis results.

[0625] The report should be presented in a visually clear format, including text summaries, graphs, and charts.

[0626] Step 12:

[0627] The server notifies the administrator of the report.

[0628] The server sends a notification to the administrator's terminal to send the generated report.

[0629] The notification should include a link to the report, making it easily accessible to administrators.

[0630] The above explains in detail the program processing of this system, which includes a series of processes from collecting survey data from users, filtering and analyzing it using generative artificial intelligence, and notifying administrators of reports.

[0631] (Example 1)

[0632] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0633] Traditional survey analysis systems faced challenges such as the time and effort required for data preprocessing, sentiment classification, result analysis, and report generation. Furthermore, they suffered from accuracy issues in sentiment classification and insufficient visualization of analysis results. This made it difficult for administrators to quickly and accurately obtain useful information for making appropriate decisions.

[0634] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0635] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for aggregating the filtering results and analyzing the proportion of positive and negative emotions, and means for generating a report including a visual chart based on the analysis results and notifying the administrator or relevant parties. This automates everything from data pre-processing to emotion classification, result analysis, and visual report generation, enabling administrators to obtain information quickly and accurately.

[0636] "Customer or employee survey results" refers to data from surveys conducted by a company or organization among its customers or employees.

[0637] "Means of collection" refers to the technical means of receiving survey results and storing them in a database or storage.

[0638] "Methods for cleaning and preprocessing" refer to processes and technical means for removing missing or outlier values ​​from collected data and converting it into an analyzable format.

[0639] "Generative artificial intelligence" refers to AI models used to perform tasks such as natural language processing and sentiment analysis.

[0640] "Means of filtering" refers to technical means of inputting pre-processed data into a generative artificial intelligence system and classifying or selecting the data according to specific criteria.

[0641] "Means of aggregation and analysis" refer to technical means of statistically aggregating filtered data and analyzing patterns and trends.

[0642] A "report including visual charts" is a report that presents analysis results in a visual format such as bar graphs or pie charts.

[0643] "Means of notifying administrators or stakeholders" refers to technical means of notifying stakeholders of generated reports via email or dashboards.

[0644] "Methods for imputing missing values" refer to technical means of filling in missing values ​​in a dataset based on statistical methods or known data.

[0645] "Methods for correcting outliers" refer to technical means that detect values ​​in a dataset that deviate significantly from the normal range and correct them to bring them back within an appropriate range.

[0646] This invention relates to a system that efficiently collects survey results from customers or employees, performs preprocessing, filtering using generative artificial intelligence, analysis, and report generation. In this system, the entire process, from data collection to report notification, is automated.

[0647] System configuration and hardware / software used

[0648] 1. Data Collection

[0649] Users access a dedicated survey form and enter their answers to the questions. This survey form operates on a web browser (e.g., Google Chrome, Mozilla Firefox) or a dedicated application.

[0650] The terminal transmits the survey data entered by the user to the server in real time. The HTTPS protocol is used for this communication to ensure security.

[0651] 2. Data preprocessing

[0652] The server stores the received survey data in a temporary storage area and then formally saves it to a database (e.g., MySQL, PostgreSQL).

[0653] The server checks the data extracted from the database for missing or outlier values ​​and cleans it as needed. Missing values ​​are imputed using the median or mean, and outliers are detected and a warning is issued.

[0654] 3. Filtering using generative artificial intelligence

[0655] The server passes the cleaned data to a generative artificial intelligence (e.g., OpenAI GP T-3) for sentiment classification. Specifically, it classifies the data as either "positive" or "negative" through text analysis.

[0656] The classification results returned by the generative artificial intelligence are saved to the server, and metadata necessary for analysis (analysis date and time, model version, etc.) is also added.

[0657] 4. Summary and Analysis of Results

[0658] The server aggregates and analyzes the filtered data. This includes calculating the percentage of positive and negative responses, and performing detailed statistical analysis on specific questions.

[0659] The analytical methods used include regression analysis and clustering.

[0660] 5. Report generation and notification

[0661] The server generates a report that includes visual charts (e.g., bar graphs, pie charts) based on the analysis results.

[0662] The server notifies administrators or relevant parties of the generated reports. These notifications are sent via email or a dedicated dashboard.

[0663] Examples of specific cases and prompt statements

[0664] For example, if a company conducts a survey to investigate employee satisfaction, this system will operate as follows:

[0665] Users (employees) access a survey form and answer questions about the work environment.

[0666] The device sends the response data to the server.

[0667] The server stores the received survey data in a database and performs checks and imputations for missing or outlier values.

[0668] The server passes the text responses to a generative artificial intelligence system, which then classifies their content as positive or negative.

[0669] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[0670] The server generates a report containing the analysis results and notifies the administrator.

[0671] Examples of prompts to input into a generative AI model:

[0672] Please classify the following open-ended text data as positive or negative.

[0673] Answer 1: "The workplace atmosphere is very good, but the workload is too much."

[0674] Answer 2: "My boss is kind and helpful. I am very satisfied."

[0675] Answer 3: "The work is boring and unstimulating. Improvement is needed."

[0676] In this way, the system efficiently collects, preprocesses, and analyzes survey results, and provides the results as a report.

[0677] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0678] Step 1:

[0679] User survey input

[0680] Users access a dedicated survey form and answer the questions. This survey form operates on a web browser (e.g., Google Chrome, Mozilla Firefox) or a dedicated application.

[0681] Input: User survey response data

[0682] Output: Survey response data sent to the server via the terminal.

[0683] Step 2:

[0684] Sending data from the terminal to the server

[0685] The terminal sends the survey data entered by the user to the server using the HTTPS protocol.

[0686] Input: User survey response data

[0687] Output: Survey response data received by the server

[0688] Step 3:

[0689] Data reception and temporary storage by the server

[0690] The server receives the survey data sent from the terminal and saves it to a temporary storage area. A database system (e.g., MySQL, PostgreSQL) is used for this storage.

[0691] Input: Survey response data sent from the device.

[0692] Output: Data stored in the temporary storage area

[0693] Step 4:

[0694] Data preprocessing by the server

[0695] The server extracts survey data from the database and checks for missing or outlier values. Missing values ​​are imputed using the median or mean, and outliers are detected and corrected.

[0696] Input: Data from temporary storage area

[0697] Output: Cleaned and pre-processed data

[0698] Step 5:

[0699] Data transmission and filtering by the server to the generative artificial intelligence.

[0700] The server passes the pre-processed data to the generative artificial intelligence. The generative AI uses the prompt "Classify the following text as positive or negative" to categorize the emotions.

[0701] Input: Cleaned and pre-processed data

[0702] Output: Filtering results (positive, negative) returned by the generative artificial intelligence.

[0703] Step 6:

[0704] Server saving of filtering results

[0705] The server stores the filtering results received from the generative artificial intelligence in a database. At this time, metadata such as the analysis date and time and model version are also recorded.

[0706] Input: Filtered results from generative artificial intelligence

[0707] Output: Filtering results stored in the database

[0708] Step 7:

[0709] Server-based aggregation and analysis of filtering results

[0710] The server aggregates the filtering results and calculates the percentage of positive and negative responses. It also performs detailed statistical analysis for specific questions and departments.

[0711] Input: Database containing filtered results

[0712] Output: Aggregated and analyzed data

[0713] Step 8:

[0714] Server-based report generation and notification

[0715] The server generates a report containing visual charts from the aggregated and analyzed results. The generated report is notified to administrators or relevant parties. Notifications are made via email or a dedicated dashboard.

[0716] Input: Aggregated and analyzed data

[0717] Output: Report notified to administrator or relevant party

[0718] Through the steps described above, this system can efficiently automate and perform a series of processes, from data collection to report notification.

[0719] (Application Example 1)

[0720] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0721] Traditional survey systems struggle not only to collect feedback from customers and employees, but also to efficiently clean, analyze, and quickly implement countermeasures based on that feedback. Furthermore, the time-consuming process of data aggregation and analysis makes it difficult to immediately reflect the insights gained from the feedback. Therefore, there is a need for real-time service improvement and rapid problem resolution.

[0722] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0723] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, and means for sending the generated report to an administrator using electronic messages. This makes it possible to efficiently carry out everything from collecting and analyzing survey results to generating and notifying reports.

[0724] "Customer or employee" refers to a user who provides survey responses to this system.

[0725] "Survey results" refers to the content of responses given by customers or employees to a survey.

[0726] "Means of collection" refers to the methods and equipment used to obtain survey results and transmit them to a server.

[0727] "Methods for cleaning and preprocessing" refer to methods and equipment that fill in missing or outlier values ​​in collected survey data, making the data analyzable.

[0728] "Generative artificial intelligence" refers to intelligent systems that analyze text data using natural language processing and machine learning techniques.

[0729] "Means of filtering" refers to methods and devices that use generative artificial intelligence to analyze survey data, classify information, or extract information based on specific conditions.

[0730] "Means for aggregating filtering results and performing analysis based on them" refers to methods and equipment for compiling the results analyzed by generative artificial intelligence and performing statistical analysis and evaluation.

[0731] "Means of generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for organizing analysis results, creating reports in visual or text format, and informing administrators or relevant parties via email or other means.

[0732] "Means of sending to an administrator using electronic messages" refers to methods or devices for sending generated reports to an administrator via email or other digital communication means.

[0733] The present invention is a system for efficiently collecting survey results from customers or employees, cleaning, filtering using generative artificial intelligence, and analyzing the results, and finally generating a report. This system includes means for collecting survey results, means for preprocessing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[0734] System Overview

[0735] 1. Data Collection

[0736] Users (customers or employees) answer the survey via a smartphone application. This survey form operates on a web browser or a dedicated application. The device sends this input data to the server, where it is received and initially stored.

[0737] 2. Data preprocessing

[0738] The server stores the collected survey data in a database. It then extracts the data for preprocessing. The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[0739] 3. Filtering using generative artificial intelligence

[0740] The server passes the cleaned data to a generative artificial intelligence (AI). This AI analyzes, for example, open-ended text data and classifies the emotions expressed as "positive" or "negative." The generative AI returns the filtered results, which are then stored again on the server.

[0741] 4. Summary and Analysis of Results

[0742] The server aggregates the filtering results and performs detailed analysis based on them. For example, it calculates the percentage of positive and negative responses. Furthermore, it also performs statistical analysis for specific questions or by department.

[0743] 5. Report generation and notification

[0744] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts. The server sends the generated report to the administrator via electronic message.

[0745] Specific examples

[0746] For example, consider a case where a customer satisfaction survey is conducted at a physical store. In this survey, customers answer questions about things like "the atmosphere of the store" and "the service provided by the staff."

[0747] 1. Questionnaire response

[0748] Users (customers) access a smartphone app and answer questions about the store's atmosphere and staff service.

[0749] 2. Storing in a database

[0750] The terminal sends the response data to the server. The server stores the received survey data in a database.

[0751] 3. Data Cleaning

[0752] The server checks for missing or outlier values ​​and fills in the data as needed.

[0753] 4. AI-based filtering

[0754] The server passes the text responses to a generative artificial intelligence system, which then classifies the content as either "positive" or "negative." An example of a prompt is: "We'd love to hear your feedback: How did you feel about the service provided by our staff?"

[0755] 5. Summary and Analysis of Results

[0756] The server aggregates the classification results and analyzes trends in customer satisfaction with the store's atmosphere, for example.

[0757] 6. Report generation and notification

[0758] The server generates a report containing the analysis results and notifies the administrator via email. This report is used to consider ways to improve store services.

[0759] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0760] Step 1:

[0761] Users access the survey form and answer the questions. Specifically, customers or employees use a smartphone application to answer questions entered into the survey form using text or multiple-choice options. The entered data (e.g., free-text responses, multiple-choice answers, etc.) is reflected in the survey form.

[0762] Step 2:

[0763] The terminal sends the survey results entered by the user to the server. The transmitted data is received by the server for subsequent processing and initially stored in the database. The input is the survey response data, and the output is the raw data stored in the database.

[0764] Step 3:

[0765] The server preprocesses the survey data stored in the database. Specifically, it imputes missing values ​​and corrects outliers. This results in cleaned data. The input is the stored raw data, and the output is the cleaned data.

[0766] Step 4:

[0767] The server passes the cleaned data to a generative artificial intelligence (AI) for filtering. The AI ​​uses a natural language processing model to analyze the text data and classify the emotion of each response as either "positive" or "negative." In this process, the generative AI model receives text data as input and outputs the result of the emotion classification. The input is the cleaned data, and the output is the result of the emotion classification.

[0768] Step 5:

[0769] The server aggregates data classified by generative artificial intelligence and performs overall trend analysis and statistical analysis. Specifically, it calculates the percentage of positive and negative responses. The input is the result data of sentiment classification, and the output is the aggregated and statistically analyzed results.

[0770] Step 6:

[0771] The server generates a report based on the aggregated and analyzed results and sends it to the administrator via electronic message. The report includes visual information such as graphs and charts. Specifically, the process involves organizing the analysis results, converting them into a visual format, and sending them to the administrator via email. The input is the aggregated and analyzed results, and the output is the report sent via email.

[0772] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0773] This invention is a system that collects survey results from customers or employees, cleans the data, filters it using generative artificial intelligence, and combines it with an emotion engine to efficiently analyze the data and generate reports.

[0774] System Overview

[0775] This system utilizes an emotion engine, in particular, to recognize user emotions, streamlining the entire process from survey data collection and analysis to report generation. The system includes the following:

[0776] 1. Data collection methods:

[0777] A method for collecting survey results from users. This is done via online forms or dedicated apps.

[0778] 2. Pre-treatment means:

[0779] A method for cleaning and preprocessing collected data. This includes imputing missing values ​​and correcting outliers.

[0780] 3. Filtering methods using generative artificial intelligence:

[0781] A method of passing pre-processed data to a generative artificial intelligence system to perform emotion classification.

[0782] 4. Emotional Engine:

[0783] A method for analyzing user voice, facial expressions, and text data from survey responses to recognize emotions in real time.

[0784] 5. Analysis methods:

[0785] A method for aggregating filtered data and performing analysis based on that data.

[0786] 6. Report generation and notification methods:

[0787] A means of generating a report based on the analysis results and notifying administrators or relevant parties.

[0788] Program processing

[0789] The program's processing can be explained in natural language as follows:

[0790] 1. Data Collection

[0791] Users access the survey form and answer each question. This form operates within a web browser or application.

[0792] The terminal collects data entered by the user and sends it to the server.

[0793] 2. Emotional analysis using an emotion engine

[0794] The device uses its camera and microphone to capture facial expressions and voice while the user is responding, and transmits this information to the emotion engine in real time.

[0795] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state.

[0796] 3. Receiving and cleaning data

[0797] The server stores the received survey data in a database and performs preprocessing. During this process, missing values ​​are imputed and outliers are corrected.

[0798] 4. Filtering using generative artificial intelligence

[0799] The server passes the pre-processed data to a generative artificial intelligence system, which performs sentiment classification of the text data. The system then receives the results, categorized as positive, negative, neutral, etc.

[0800] 5. Summary and Analysis of Results

[0801] The server aggregates the filtering results and performs a detailed analysis. For example, it calculates the percentage of positive and negative responses. Real-time sentiment data from the sentiment engine is also used in the analysis.

[0802] 6. Report generation and notification

[0803] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements.

[0804] The server notifies the administrator of the reports it generates, making them easily accessible to the administrator.

[0805] Specific example

[0806] For example, when conducting an employee satisfaction survey:

[0807] 1. Questionnaire response

[0808] The user answers a series of questions. Simultaneously, the user's facial expressions and voice data are captured using the camera and microphone.

[0809] 2. Real-time sentiment analysis

[0810] The emotion engine analyzes facial expressions and voice in real time to recognize the user's emotional state.

[0811] 3. Storing in the database and cleaning

[0812] The server stores the received survey data in a database and cleans up any missing or outlier values.

[0813] 4. Filtering using generative artificial intelligence

[0814] The server passes text data to a generative artificial intelligence system, which then performs emotion classification. For example, it might classify the data as "satisfied" or "dissatisfied."

[0815] 5. Summary and Analysis of Results

[0816] The server aggregates the filtering results and the analysis results of the emotion engine to analyze trends for each department and overall satisfaction levels.

[0817] 6. Report generation and notification

[0818] The server generates a report based on the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[0819] As described above, this system can recognize users' emotions in real time and utilize this information in the analysis of survey results, thereby providing more detailed and reliable insights.

[0820] The following describes the processing flow.

[0821] Step 1:

[0822] The user fills out the survey form.

[0823] Users (customers or employees) answer survey questions via online forms or applications.

[0824] Enter detailed answers for each question and click the "Submit" button.

[0825] Step 2:

[0826] The device sends data to the server.

[0827] The device collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[0828] The data is sent to the server in formats such as JSON or XML.

[0829] Step 3:

[0830] The device acquires the user's voice and facial expression data.

[0831] The device uses its camera and microphone to capture the user's real-time voice and facial expression data.

[0832] This data is sent to the emotion engine.

[0833] Step 4:

[0834] The emotion engine analyzes voice and facial expression data.

[0835] The emotion engine analyzes acquired voice and facial expression data to recognize the user's emotional state (e.g., positive, negative, neutral).

[0836] Send the recognition results to the server.

[0837] Step 5:

[0838] The server receives and stores the survey data.

[0839] The server stores the survey data sent from the terminal in a buffer for temporary storage.

[0840] Convert the data to a format suitable for storage in a database.

[0841] Step 6:

[0842] The server saves data to the database.

[0843] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[0844] The inserted data is recorded in the survey data table.

[0845] Step 7:

[0846] The server cleans the survey data.

[0847] The server reads the stored data and cleans it.

[0848] Missing values ​​are detected and either imputed with the mean or excluded as invalid data. Outliers are checked and corrected in the same way.

[0849] Step 8:

[0850] The server passes data to the generative artificial intelligence.

[0851] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[0852] In particular, submit the text data of your open-ended responses.

[0853] Step 9:

[0854] Generative artificial intelligence filters the data.

[0855] Generative artificial intelligence analyzes received data and classifies emotions according to specific criteria.

[0856] For example, text responses are classified as either "positive" or "negative."

[0857] Step 10:

[0858] The server receives the results from the generative artificial intelligence.

[0859] The server receives the filtering results returned by the generative artificial intelligence.

[0860] The received data is stored back into the buffer.

[0861] Step 11:

[0862] The server aggregates the filtering results.

[0863] The server extracts filtered data and performs statistical aggregation.

[0864] Count the number of positive and negative responses, and calculate the percentage of each.

[0865] Step 12:

[0866] The server integrates the results from the emotion engine.

[0867] The server integrates the real-time emotion recognition results received from the emotion engine with the analysis results of the survey data.

[0868] This means that not only text responses but also the user's emotional state during the response process will be used for analysis.

[0869] Step 13:

[0870] The server analyzes the data.

[0871] The server performs detailed statistical analysis based on the aggregated results.

[0872] For example, calculate the average satisfaction score for each department and analyze specific demographic factors.

[0873] Step 14:

[0874] The server generates the report.

[0875] The server generates a report based on the analysis results.

[0876] The report should include visually clear formats such as text summaries, graphs, and charts.

[0877] Step 15:

[0878] The server notifies the administrator of the report.

[0879] The server will notify the administrator of the generated report via email or a dashboard.

[0880] Include a link that allows administrators to access the report.

[0881] In summary, the system's program processing is executed as a series of processes, from collecting survey data, to real-time analysis by the emotion engine, filtering by generative artificial intelligence, data analysis, and finally, sending reports to administrators.

[0882] (Example 2)

[0883] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0884] Traditional survey systems struggled to accurately grasp users' emotional states, making detailed emotion-based analysis difficult. Furthermore, missing or outlier data in the collected responses reduced the accuracy of the analysis. While generative artificial intelligence-based emotion classification was employed, the lack of real-time emotion analysis undermined the reliability of the results.

[0885] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0886] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial expressions and voice using an emotion engine that recognizes the emotional state of the user at the time of response, means for aggregating the filtering results and performing analysis based on them, and means for generating a report based on the analysis results and notifying the administrator or relevant parties. This enables detailed and reliable data analysis that incorporates real-time emotion analysis of the user.

[0887] "Customer or employee" refers to an individual who uses the system to respond to a survey.

[0888] "Survey results" refer to the response data entered by customers or employees through a survey form.

[0889] "Means of collection" refers to methods and devices for obtaining survey results from users via online forms, dedicated applications, etc.

[0890] "Methods for cleaning and preprocessing" refer to data preprocessing techniques and methods for imputing missing values ​​and correcting outliers in collected survey data.

[0891] "Generative artificial intelligence" refers to artificial intelligence models that analyze collected text data and perform sentiment classification and other classifications.

[0892] "Means for performing filtering" refers to methods or devices for passing pre-processed data to a generative artificial intelligence system to perform filtering, such as classifying the sentiment of the data.

[0893] An "emotion engine" is a system or software that analyzes the user's facial expressions and voice data in real time when they respond to questions, and recognizes the user's emotional state.

[0894] "Means of aggregating and performing analysis based on that data" refers to technologies and methods for aggregating results obtained from emotion engines and generative artificial intelligence, and then performing detailed analysis based on that data.

[0895] "Means for generating reports and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating those reports to administrators or relevant parties via email or notification systems.

[0896] This invention is a system that collects survey results from customers or employees, performs data cleaning, filters using generative artificial intelligence (AI models), and combines this with an emotion engine. This allows for efficient data analysis and report generation.

[0897] The system uses the following hardware and software:

[0898] Web browser or dedicated application: Software on a device used by the user to answer the survey.

[0899] Server: The central computer responsible for data collection, database storage, data preprocessing, analysis using generative AI models, emotion engine management, aggregation, analysis, and report generation.

[0900] Database: A system for storing collected survey data and analysis results.

[0901] Emotion Engine: Software that processes facial and voice data obtained from cameras and microphones in order to analyze the user's emotions in real time.

[0902] Generative artificial intelligence: Natural language processing (NLP) models used to classify emotions from survey responses. Examples: AI models such as GPT-4.

[0903] Python's Pandas library: Used for data cleaning (imputing missing values ​​and correcting outliers).

[0904] Matplotlib and ReportLab are used to generate reports that include graphs and charts.

[0905] Specific examples of implementation

[0906] For example, let's consider the case of conducting an employee satisfaction survey.

[0907] 1. Users access the survey form via a web browser or dedicated application and answer the survey. At this time, the device's camera and microphone are turned on to capture facial expressions and voice data.

[0908] 2. The terminal transmits the user's input and acquired facial expression / voice data to the server. The HTTPS protocol is used in this process.

[0909] 3. The server saves the received survey data to a database and preprocesses the data using the Python Pandas library. Preprocessing includes imputing missing values ​​and correcting outliers.

[0910] 4. The pre-processed data is input into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. For example, responses are classified as positive, negative, or neutral.

[0911] 5. The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server.

[0912] 6. The server aggregates the filtering results from the generative artificial intelligence and the data obtained from the emotion engine, and performs a detailed analysis. This analysis includes the percentage of positive responses, the percentage of negative responses, etc.

[0913] 7. The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, etc. These are generated using Python's Matplotlib or ReportLab.

[0914] 8. The server notifies the administrator of the generated report. Email or a dedicated notification system may be used for notification.

[0915] Examples of prompts for generative AI models

[0916] "Classify the emotions from the submitted survey responses into positive, negative, and neutral, and calculate the ratio of each. Furthermore, perform a comprehensive emotional analysis considering facial expression and voice data, and output the results as a report."

[0917] The above describes the embodiments for carrying out the present invention. This system achieves more detailed and reliable data analysis by incorporating real-time sentiment analysis of the user.

[0918] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0919] Step 1:

[0920] Data collection

[0921] Users access the survey form via a web browser or a dedicated application and fill it out. This includes answering questions and providing free-form text.

[0922] The terminal collects survey data entered by the user and sends it to the server. The data is structured in JSON format and sent using an HTTP POST request.

[0923] Input: User survey response data (text).

[0924] Output: Structured data (JSON format) sent to the server.

[0925] Step 2:

[0926] Real-time sentiment analysis

[0927] The device uses its camera and microphone to capture facial expressions and voice while the user is filling out a survey, and transmits this information to the emotion engine in real time.

[0928] The emotion engine analyzes the user's voice and facial expression data to recognize their emotional state (e.g., joy, sadness, anger). This analysis uses voice emotion recognition algorithms and facial expression recognition algorithms.

[0929] Input: User's facial expression data, voice data.

[0930] Output: Analyzed emotion data (e.g., joy, sadness, anger, etc.).

[0931] Step 3:

[0932] Receiving and cleaning data

[0933] The server receives survey data sent from the terminal and stores it in the database. Afterward, it performs data preprocessing. This preprocessing includes imputing missing values ​​and correcting outliers. The data is cleaned using the Python Pandas library.

[0934] Input: Survey data sent from the device.

[0935] Output: Clean data after preprocessing.

[0936] Step 4:

[0937] Filtering by generative artificial intelligence

[0938] The server inputs pre-processed data into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. The model analyzes the text data and classifies each response as positive, negative, neutral, etc.

[0939] Input: Clean text data.

[0940] Output: Classified sentiment data (positive, negative, neutral).

[0941] Step 5:

[0942] Summary and analysis of results

[0943] The server aggregates data obtained from generative artificial intelligence and emotion engines and performs detailed analysis. Specifically, it calculates the percentage of positive and negative responses. This analysis uses SQL queries and Python libraries (e.g., Pandas, Matplotlib).

[0944] Input: Classified sentiment data, real-time analyzed sentiment data.

[0945] Output: Aggregated results and detailed analysis results (e.g., positive response rate).

[0946] Step 6:

[0947] Report generation and notification

[0948] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements. Python's ReportLab and Matplotlib are used to generate the report. The generated report is notified to the administrator via email or a dedicated notification system.

[0949] Input: Detailed analysis results.

[0950] Output: Generated report (PDF or HTML format), notification email.

[0951] (Application Example 2)

[0952] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0953] Traditional survey systems often fail to adequately consider emotional factors when collecting feedback from customers and employees, resulting in superficial analysis of the results. Furthermore, data cleaning and filtering are frequently performed manually, making efficient data analysis difficult. Consequently, it becomes challenging to quickly and accurately obtain information that is useful to managers and stakeholders. In store feedback systems, there is a need for methods that analyze customer emotions in real time and classify and analyze emotional data using generative AI.

[0954] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0955] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial and voice data collected during pre-processing to recognize real-time emotions, means for aggregating the filtering results and real-time emotion data and performing analysis based on them, and means for generating a report based on the analysis results and notifying administrators or relevant parties. This enables detailed and reliable feedback analysis that takes customer emotions into account, allowing administrators and relevant parties to quickly obtain useful information.

[0956] "Means of collecting survey results" refers to methods or equipment used to collect responses from customers or employees.

[0957] "Methods for cleaning and preprocessing data" refer to methods and techniques for preparing collected data for analysis by imputing or correcting missing or outlier values.

[0958] "Methods for passing data to a generative artificial intelligence to perform filtering" refer to methods or systems for inputting pre-processed data into a generative artificial intelligence and having it classify it into appropriate emotions or categories.

[0959] "Means for analyzing facial and voice data to recognize emotions in real time" refers to technologies and devices that use collected facial and voice data to analyze and recognize a user's emotional state in real time.

[0960] "Means for aggregating data and performing analysis based on it" refers to methods and systems that integrate and aggregate filtered data and real-time sentiment data to perform statistical and sentimental analysis.

[0961] "Means for generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating them to administrators or relevant parties.

[0962] This invention is an advanced system for efficiently collecting and analyzing feedback from customers or employees. This system operates using a combination of hardware and software and includes the following means:

[0963] hardware

[0964] 1. Smart devices

[0965] Smartphones and tablets are used for collecting survey responses. These devices have built-in cameras and microphones, allowing them to collect user facial expressions and audio data.

[0966] 2. Server

[0967] A server equipped with a database and analysis engine. This enables data preprocessing, filtering by generative artificial intelligence, real-time sentiment analysis, data aggregation and analysis, and report generation and notification.

[0968] software

[0969] 1. Emotional Engine

[0970] The system analyzes the user's emotions in real time using facial expression analysis libraries (e.g., "OpenFace," "Affectiva") and voice emotion analysis software (e.g., "IBM Watson Tone Analyzer").

[0971] 2. Generative Artificial Intelligence

[0972] Generative artificial intelligence such as BERT and GPT-3 (OpenAI) will be used to classify the sentiment of text data from survey responses.

[0973] 3. Database

[0974] We use MySQL or PostgreSQL to store and manage the collected and pre-processed data.

[0975] 4. Analytics

[0976] We will use Python and its libraries (e.g., pandas, numpy, matplotlib) to analyze data and generate reports.

[0977] 5. Notification System

[0978] Use email services (e.g., AWS SES) or messaging services (e.g., Twilio) to notify administrators and relevant parties of the report.

[0979] System operation

[0980] Data collection and cleaning

[0981] The terminal collects survey responses from customers and employees. In addition to the text data entered by the user, it also collects facial expressions and voice data using the camera and microphone. The collected data is sent to a server where missing values ​​are imputed and outliers are corrected.

[0982] Filtering by generative artificial intelligence

[0983] The pre-processed data is passed to a generative artificial intelligence (AI) for emotion classification. The AI ​​classifies the data into positive, negative, neutral, etc.

[0984] Real-time sentiment analysis

[0985] The collected facial and voice data is analyzed in real time by an emotion engine to recognize the user's emotional state.

[0986] Data aggregation and analysis

[0987] The filtering results and real-time sentiment data are integrated by the server for detailed analysis. For example, by statistically analyzing the satisfaction and dissatisfaction levels of each product, areas for improvement in the store can be identified.

[0988] Report generation and notification

[0989] Based on the analysis results, a report is generated that includes text summaries, graphs, charts, and other elements. This report is then sent to administrators and relevant parties through a notification system.

[0990] Specific example

[0991] Examples of prompt statements include the following:

[0992] Please generate an emotional analysis report for product "A". Please perform the analysis based on the following data set. We would like to know the ratio of positive, negative, and neutral responses.

[0993] In this way, this system enables detailed feedback analysis, including the actual emotions of customers, allowing managers and stakeholders to quickly identify areas for improvement.

[0994] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0995] Step 1:

[0996] Data collection

[0997] When collecting survey results entered by customers or employees, the device also uses its camera and microphone to simultaneously acquire facial expressions and voice data.

[0998] Input: Customer or employee survey text, facial expression data, audio data

[0999] Output: Collected questionnaire text, facial expression data, audio data

[1000] Step 2:

[1001] Data transmission

[1002] The device transmits the collected survey text, facial expression data, and audio data to the server.

[1003] Input: Collected questionnaire text, facial expression data, audio data

[1004] Output: Survey text, facial expression data, and audio data sent to the server.

[1005] Step 3:

[1006] Data cleaning and preprocessing

[1007] The server fills in missing values ​​and corrects outliers from the received data. For example, it calculates fill-in values ​​for questions with missing answers and corrects extremely abnormal values.

[1008] Input: Survey text, facial expression data, and audio data sent from the device.

[1009] Output: Cleaned and pre-processed questionnaire text, facial expression data, audio data

[1010] Step 4:

[1011] Filtering by generative artificial intelligence

[1012] The server inputs pre-processed data into a generative artificial intelligence (e.g., GPT-3) to perform sentiment classification. The generative AI classifies the data into positive, negative, or neutral and returns the result.

[1013] Input: Preprocessed survey text

[1014] Output: Emotionally classified survey data (positive, negative, neutral)

[1015] Step 5:

[1016] Real-time sentiment analysis

[1017] The server analyzes facial and voice data collected using facial expression analysis libraries and voice emotion analysis software in real time to recognize the user's emotional state.

[1018] Input: Preprocessed facial expression data and audio data

[1019] Output: Recognized real-time sentiment data

[1020] Step 6:

[1021] Data aggregation and analysis

[1022] The server integrates and aggregates the filtering results and real-time sentiment data, and performs statistical and emotional analysis. For example, it calculates the percentage of positive emotions, negative emotions, and neutral emotions.

[1023] Input: Sentiment-classified survey data, real-time sentiment data

[1024] Output: Aggregated and analyzed results (e.g., sentiment ratio, trend analysis, etc.)

[1025] Step 7:

[1026] Report generation

[1027] The server generates a report that includes text summaries, graphs, and charts based on the aggregated and analyzed results.

[1028] Input: Aggregation and analysis results

[1029] Output: Report (text summary, graphs, charts, etc.)

[1030] Step 8:

[1031] notification

[1032] The server uses email or messaging services to notify administrators or relevant parties of the reports it generates.

[1033] Input: Generated report

[1034] Output: Notification to administrators or relevant parties (email, message, etc.)

[1035] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1036] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1037] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1038] [Third Embodiment]

[1039] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1040] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1041] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1042] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1043] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1044] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1045] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1046] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1047] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1048] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1049] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1050] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1051] This invention is a system that efficiently collects survey results from customers or employees, cleans them, filters them using generative artificial intelligence, and analyzes them, ultimately generating a report.

[1052] System Overview

[1053] This system includes means for collecting survey results, means for pre-processing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[1054] Program processing

[1055] The program's processing can be explained in natural language as follows:

[1056] 1. Data Collection

[1057] Users enter information through a survey form. This survey form operates on a web browser or a dedicated application.

[1058] The terminal sends this input data to the server. The transmitted data is received by the server and initially stored.

[1059] 2. Data preprocessing

[1060] The server stores the collected survey data in a database. Then, it extracts the data for preprocessing.

[1061] The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[1062] 3. Filtering using generative artificial intelligence

[1063] The server passes the cleaned data to a generative artificial intelligence (AI). This AI, for example, analyzes open-ended text data and classifies the emotions expressed as "positive" or "negative."

[1064] The generative artificial intelligence returns the filtered results. These results are then saved again on the server side.

[1065] 4. Summary and Analysis of Results

[1066] The server aggregates the filtering results and performs a detailed analysis based on them. For example, it calculates the percentage of positive responses and the percentage of negative responses.

[1067] Furthermore, statistical analysis will be conducted on specific question items and by department.

[1068] 5. Report generation and notification

[1069] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts.

[1070] The server generates reports and notifies the administrator. These notifications are sent via email or a dashboard.

[1071] Specific example

[1072] For example, consider a case where a company conducts a survey to investigate employee satisfaction. In this survey, employees answer questions about their work environment and job satisfaction.

[1073] 1. Questionnaire response

[1074] Users (employees) access a survey form and answer questions about the work environment.

[1075] The device sends the response data to the server.

[1076] 2. Storing in a database

[1077] The server stores the received survey data in a database.

[1078] 3. Data Cleaning

[1079] The server checks for missing or outlier values ​​and fills in the data as needed.

[1080] 4. AI-based filtering

[1081] The server passes the text response to a generative artificial intelligence system, which then classifies its content as positive or negative.

[1082] 5. Summary and Analysis of Results

[1083] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[1084] 6. Report generation and notification

[1085] The server generates a report containing the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[1086] In this way, this system can efficiently collect, preprocess, and analyze survey results from customers and employees, and provide the results as reports.

[1087] The following describes the processing flow.

[1088] Step 1:

[1089] The user fills out the survey form.

[1090] Users (customers or employees) respond to surveys via online forms or applications.

[1091] Enter your answers for each question, and then click the "Submit" button.

[1092] Step 2:

[1093] The device sends data to the server.

[1094] The device (PC or smartphone) collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[1095] The data sent is often in formats such as JSON or XML.

[1096] Step 3:

[1097] The server receives the data.

[1098] The server temporarily stores the received survey data in a buffer.

[1099] Next, the data is converted into a format suitable for storage in the database.

[1100] Step 4:

[1101] The server saves data to the database.

[1102] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[1103] The inserted data is recorded in the table.

[1104] Step 5:

[1105] The server cleans up the data.

[1106] The server reads the stored data and cleans it.

[1107] Specifically, missing values ​​are detected and either imputed with the mean or removed as invalid data. Outliers are checked and corrected in the same way.

[1108] Step 6:

[1109] The server passes data to the generative artificial intelligence.

[1110] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[1111] The data to be submitted will include, in particular, open-ended text data.

[1112] Step 7:

[1113] Generative artificial intelligence filters the data.

[1114] Generative artificial intelligence analyzes received data and filters it according to specific criteria.

[1115] For example, classify users' emotions as either "positive" or "negative."

[1116] Step 8:

[1117] The server receives the results from the generative artificial intelligence.

[1118] The server receives the filtering results returned by the generative artificial intelligence.

[1119] The received data is stored back into the buffer.

[1120] Step 9:

[1121] The server aggregates the filtered data.

[1122] The server extracts the filtering results stored in the buffer and performs statistical aggregation.

[1123] For example, count the number of positive and negative responses and calculate the percentage of each.

[1124] Step 10:

[1125] The server analyzes the data.

[1126] The server analyzes the aggregated data and performs detailed statistical analysis.

[1127] For example, you could calculate the average satisfaction score for each department and identify which departments receive high ratings.

[1128] Step 11:

[1129] The server generates the report.

[1130] The server generates a report based on the analysis results.

[1131] The report should be presented in a visually clear format, including text summaries, graphs, and charts.

[1132] Step 12:

[1133] The server notifies the administrator of the report.

[1134] The server sends a notification to the administrator's terminal to send the generated report.

[1135] The notification should include a link to the report, making it easily accessible to administrators.

[1136] The above explains in detail the program processing of this system, which includes a series of processes from collecting survey data from users, filtering and analyzing it using generative artificial intelligence, and notifying administrators of reports.

[1137] (Example 1)

[1138] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1139] Traditional survey analysis systems faced challenges such as the time and effort required for data preprocessing, sentiment classification, result analysis, and report generation. Furthermore, they suffered from accuracy issues in sentiment classification and insufficient visualization of analysis results. This made it difficult for administrators to quickly and accurately obtain useful information for making appropriate decisions.

[1140] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1141] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for aggregating the filtering results and analyzing the proportion of positive and negative emotions, and means for generating a report including a visual chart based on the analysis results and notifying the administrator or relevant parties. This automates everything from data pre-processing to emotion classification, result analysis, and visual report generation, enabling administrators to obtain information quickly and accurately.

[1142] "Customer or employee survey results" refers to data from surveys conducted by a company or organization among its customers or employees.

[1143] "Means of collection" refers to the technical means of receiving survey results and storing them in a database or storage.

[1144] "Methods for cleaning and preprocessing" refer to processes and technical means for removing missing or outlier values ​​from collected data and converting it into an analyzable format.

[1145] "Generative artificial intelligence" refers to AI models used to perform tasks such as natural language processing and sentiment analysis.

[1146] "Means of filtering" refers to technical means of inputting pre-processed data into a generative artificial intelligence system and classifying or selecting the data according to specific criteria.

[1147] "Means of aggregation and analysis" refer to technical means of statistically aggregating filtered data and analyzing patterns and trends.

[1148] A "report including visual charts" is a report that presents analysis results in a visual format such as bar graphs or pie charts.

[1149] "Means of notifying administrators or stakeholders" refers to technical means of notifying stakeholders of generated reports via email or dashboards.

[1150] "Methods for imputing missing values" refer to technical means of filling in missing values ​​in a dataset based on statistical methods or known data.

[1151] "Methods for correcting outliers" refer to technical means that detect values ​​in a dataset that deviate significantly from the normal range and correct them to bring them back within an appropriate range.

[1152] This invention relates to a system that efficiently collects survey results from customers or employees, performs preprocessing, filtering using generative artificial intelligence, analysis, and report generation. In this system, the entire process, from data collection to report notification, is automated.

[1153] System configuration and hardware / software used

[1154] 1. Data Collection

[1155] Users access a dedicated survey form and enter their answers to the questions. This survey form operates on a web browser (e.g., Google Chrome, Mozilla Firefox) or a dedicated application.

[1156] The terminal transmits the survey data entered by the user to the server in real time. The HTTPS protocol is used for this communication to ensure security.

[1157] 2. Data preprocessing

[1158] The server stores the received survey data in a temporary storage area and then formally saves it to a database (e.g., MySQL, PostgreSQL).

[1159] The server checks the data extracted from the database for missing or outlier values ​​and cleans it as needed. Missing values ​​are imputed using the median or mean, and outliers are detected and a warning is issued.

[1160] 3. Filtering using generative artificial intelligence

[1161] The server passes the cleaned data to a generative artificial intelligence (e.g., OpenAI GP T-3) for sentiment classification. Specifically, it classifies the data as either "positive" or "negative" through text analysis.

[1162] The classification results returned by the generative artificial intelligence are saved to the server, and metadata necessary for analysis (analysis date and time, model version, etc.) is also added.

[1163] 4. Summary and Analysis of Results

[1164] The server aggregates and analyzes the filtered data. This includes calculating the percentage of positive and negative responses, and performing detailed statistical analysis on specific questions.

[1165] The analytical methods used include regression analysis and clustering.

[1166] 5. Report generation and notification

[1167] The server generates a report that includes visual charts (e.g., bar graphs, pie charts) based on the analysis results.

[1168] The server notifies administrators or relevant parties of the generated reports. These notifications are sent via email or a dedicated dashboard.

[1169] Examples of specific cases and prompt statements

[1170] For example, if a company conducts a survey to investigate employee satisfaction, this system will operate as follows:

[1171] Users (employees) access a survey form and answer questions about the work environment.

[1172] The device sends the response data to the server.

[1173] The server stores the received survey data in a database and performs checks and imputations for missing or outlier values.

[1174] The server passes the text responses to a generative artificial intelligence system, which then classifies their content as positive or negative.

[1175] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[1176] The server generates a report containing the analysis results and notifies the administrator.

[1177] Examples of prompts to input into a generative AI model:

[1178] Please classify the following open-ended text data as positive or negative.

[1179] Answer 1: "The workplace atmosphere is very good, but the workload is too much."

[1180] Answer 2: "My boss is kind and helpful. I am very satisfied."

[1181] Answer 3: "The work is boring and unstimulating. Improvement is needed."

[1182] In this way, the system efficiently collects, preprocesses, and analyzes survey results, and provides the results as a report.

[1183] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1184] Step 1:

[1185] User survey input

[1186] Users access a dedicated survey form and answer the questions. This survey form operates on a web browser (e.g., Google Chrome, Mozilla Firefox) or a dedicated application.

[1187] Input: User survey response data

[1188] Output: Survey response data sent to the server via the terminal.

[1189] Step 2:

[1190] Sending data from the terminal to the server

[1191] The terminal sends the survey data entered by the user to the server using the HTTPS protocol.

[1192] Input: User survey response data

[1193] Output: Survey response data received by the server

[1194] Step 3:

[1195] Data reception and temporary storage by the server

[1196] The server receives the survey data sent from the terminal and saves it to a temporary storage area. A database system (e.g., MySQL, PostgreSQL) is used for this storage.

[1197] Input: Survey response data sent from the device.

[1198] Output: Data stored in the temporary storage area

[1199] Step 4:

[1200] Data preprocessing by the server

[1201] The server extracts survey data from the database and checks for missing or outlier values. Missing values ​​are imputed using the median or mean, and outliers are detected and corrected.

[1202] Input: Data from temporary storage area

[1203] Output: Cleaned and pre-processed data

[1204] Step 5:

[1205] Data transmission and filtering by the server to the generative artificial intelligence.

[1206] The server passes the pre-processed data to the generative artificial intelligence. The generative AI uses the prompt "Classify the following text as positive or negative" to categorize the emotions.

[1207] Input: Cleaned and pre-processed data

[1208] Output: Filtering results (positive, negative) returned by the generative artificial intelligence.

[1209] Step 6:

[1210] Server saving of filtering results

[1211] The server stores the filtering results received from the generative artificial intelligence in a database. At this time, metadata such as the analysis date and time and model version are also recorded.

[1212] Input: Filtered results from generative artificial intelligence

[1213] Output: Filtering results stored in the database

[1214] Step 7:

[1215] Server-based aggregation and analysis of filtering results

[1216] The server aggregates the filtering results and calculates the percentage of positive and negative responses. It also performs detailed statistical analysis for specific questions and departments.

[1217] Input: Database containing filtered results

[1218] Output: Aggregated and analyzed data

[1219] Step 8:

[1220] Server-based report generation and notification

[1221] The server generates a report containing visual charts from the aggregated and analyzed results. The generated report is notified to administrators or relevant parties. Notifications are made via email or a dedicated dashboard.

[1222] Input: Aggregated and analyzed data

[1223] Output: Report notified to administrator or relevant party

[1224] Through the steps described above, this system can efficiently automate and perform a series of processes, from data collection to report notification.

[1225] (Application Example 1)

[1226] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1227] Traditional survey systems struggle not only to collect feedback from customers and employees, but also to efficiently clean, analyze, and quickly implement countermeasures based on that feedback. Furthermore, the time-consuming process of data aggregation and analysis makes it difficult to immediately reflect the insights gained from the feedback. Therefore, there is a need for real-time service improvement and rapid problem resolution.

[1228] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1229] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, and means for sending the generated report to an administrator using electronic messages. This makes it possible to efficiently carry out everything from collecting and analyzing survey results to generating and notifying reports.

[1230] "Customer or employee" refers to a user who provides survey responses to this system.

[1231] "Survey results" refers to the content of responses given by customers or employees to a survey.

[1232] "Means of collection" refers to the methods and equipment used to obtain survey results and transmit them to a server.

[1233] "Methods for cleaning and preprocessing" refer to methods and equipment that fill in missing or outlier values ​​in collected survey data, making the data analyzable.

[1234] "Generative artificial intelligence" refers to intelligent systems that analyze text data using natural language processing and machine learning techniques.

[1235] "Means of filtering" refers to methods and devices that use generative artificial intelligence to analyze survey data, classify information, or extract information based on specific conditions.

[1236] "Means for aggregating filtering results and performing analysis based on them" refers to methods and equipment for compiling the results analyzed by generative artificial intelligence and performing statistical analysis and evaluation.

[1237] "Means of generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for organizing analysis results, creating reports in visual or text format, and informing administrators or relevant parties via email or other means.

[1238] "Means of sending to an administrator using electronic messages" refers to methods or devices for sending generated reports to an administrator via email or other digital communication means.

[1239] The present invention is a system for efficiently collecting survey results from customers or employees, cleaning, filtering using generative artificial intelligence, and analyzing the results, and finally generating a report. This system includes means for collecting survey results, means for preprocessing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[1240] System Overview

[1241] 1. Data Collection

[1242] Users (customers or employees) answer the survey via a smartphone application. This survey form operates on a web browser or a dedicated application. The device sends this input data to the server, where it is received and initially stored.

[1243] 2. Data preprocessing

[1244] The server stores the collected survey data in a database. It then extracts the data for preprocessing. The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[1245] 3. Filtering using generative artificial intelligence

[1246] The server passes the cleaned data to a generative artificial intelligence (AI). This AI analyzes, for example, open-ended text data and classifies the emotions expressed as "positive" or "negative." The generative AI returns the filtered results, which are then stored again on the server.

[1247] 4. Summary and Analysis of Results

[1248] The server aggregates the filtering results and performs detailed analysis based on them. For example, it calculates the percentage of positive and negative responses. Furthermore, it also performs statistical analysis for specific questions or by department.

[1249] 5. Report generation and notification

[1250] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts. The server sends the generated report to the administrator via electronic message.

[1251] Specific examples

[1252] For example, consider a case where a customer satisfaction survey is conducted at a physical store. In this survey, customers answer questions about things like "the atmosphere of the store" and "the service provided by the staff."

[1253] 1. Questionnaire response

[1254] Users (customers) access a smartphone app and answer questions about the store's atmosphere and staff service.

[1255] 2. Storing in a database

[1256] The terminal sends the response data to the server. The server stores the received survey data in a database.

[1257] 3. Data Cleaning

[1258] The server checks for missing or outlier values ​​and fills in the data as needed.

[1259] 4. AI-based filtering

[1260] The server passes the text responses to a generative artificial intelligence system, which then classifies the content as either "positive" or "negative." An example of a prompt is: "We'd love to hear your feedback: How did you feel about the service provided by our staff?"

[1261] 5. Summary and Analysis of Results

[1262] The server aggregates the classification results and analyzes trends in customer satisfaction with the store's atmosphere, for example.

[1263] 6. Report generation and notification

[1264] The server generates a report containing the analysis results and notifies the administrator via email. This report is used to consider ways to improve store services.

[1265] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1266] Step 1:

[1267] Users access the survey form and answer the questions. Specifically, customers or employees use a smartphone application to answer questions entered into the survey form using text or multiple-choice options. The entered data (e.g., free-text responses, multiple-choice answers, etc.) is reflected in the survey form.

[1268] Step 2:

[1269] The terminal sends the survey results entered by the user to the server. The transmitted data is received by the server for subsequent processing and initially stored in the database. The input is the survey response data, and the output is the raw data stored in the database.

[1270] Step 3:

[1271] The server preprocesses the survey data stored in the database. Specifically, it imputes missing values ​​and corrects outliers. This results in cleaned data. The input is the stored raw data, and the output is the cleaned data.

[1272] Step 4:

[1273] The server passes the cleaned data to a generative artificial intelligence (AI) for filtering. The AI ​​uses a natural language processing model to analyze the text data and classify the emotion of each response as either "positive" or "negative." In this process, the generative AI model receives text data as input and outputs the result of the emotion classification. The input is the cleaned data, and the output is the result of the emotion classification.

[1274] Step 5:

[1275] The server aggregates data classified by generative artificial intelligence and performs overall trend analysis and statistical analysis. Specifically, it calculates the percentage of positive and negative responses. The input is the result data of sentiment classification, and the output is the aggregated and statistically analyzed results.

[1276] Step 6:

[1277] The server generates a report based on the aggregated and analyzed results and sends it to the administrator via electronic message. The report includes visual information such as graphs and charts. Specifically, the process involves organizing the analysis results, converting them into a visual format, and sending them to the administrator via email. The input is the aggregated and analyzed results, and the output is the report sent via email.

[1278] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1279] This invention is a system that collects survey results from customers or employees, cleans the data, filters it using generative artificial intelligence, and combines it with an emotion engine to efficiently analyze the data and generate reports.

[1280] System Overview

[1281] This system utilizes an emotion engine, in particular, to recognize user emotions, streamlining the entire process from survey data collection and analysis to report generation. The system includes the following:

[1282] 1. Data collection methods:

[1283] A method for collecting survey results from users. This is done via online forms or dedicated apps.

[1284] 2. Pre-treatment means:

[1285] A method for cleaning and preprocessing collected data. This includes imputing missing values ​​and correcting outliers.

[1286] 3. Filtering methods using generative artificial intelligence:

[1287] A method of passing pre-processed data to a generative artificial intelligence system to perform emotion classification.

[1288] 4. Emotional Engine:

[1289] A method for analyzing user voice, facial expressions, and text data from survey responses to recognize emotions in real time.

[1290] 5. Analysis methods:

[1291] A method for aggregating filtered data and performing analysis based on that data.

[1292] 6. Report generation and notification methods:

[1293] A means of generating a report based on the analysis results and notifying administrators or relevant parties.

[1294] Program processing

[1295] The program's processing can be explained in natural language as follows:

[1296] 1. Data Collection

[1297] Users access the survey form and answer each question. This form operates within a web browser or application.

[1298] The terminal collects data entered by the user and sends it to the server.

[1299] 2. Emotional analysis using an emotion engine

[1300] The device uses its camera and microphone to capture facial expressions and voice while the user is responding, and transmits this information to the emotion engine in real time.

[1301] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state.

[1302] 3. Receiving and cleaning data

[1303] The server stores the received survey data in a database and performs preprocessing. During this process, missing values ​​are imputed and outliers are corrected.

[1304] 4. Filtering using generative artificial intelligence

[1305] The server passes the pre-processed data to a generative artificial intelligence system, which performs sentiment classification of the text data. The system then receives the results, categorized as positive, negative, neutral, etc.

[1306] 5. Summary and Analysis of Results

[1307] The server aggregates the filtering results and performs a detailed analysis. For example, it calculates the percentage of positive and negative responses. Real-time sentiment data from the sentiment engine is also used in the analysis.

[1308] 6. Report generation and notification

[1309] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements.

[1310] The server notifies the administrator of the reports it generates, making them easily accessible to the administrator.

[1311] Specific example

[1312] For example, when conducting an employee satisfaction survey:

[1313] 1. Questionnaire response

[1314] The user answers a series of questions. Simultaneously, the user's facial expressions and voice data are captured using the camera and microphone.

[1315] 2. Real-time sentiment analysis

[1316] The emotion engine analyzes facial expressions and voice in real time to recognize the user's emotional state.

[1317] 3. Storing in the database and cleaning

[1318] The server stores the received survey data in a database and cleans up any missing or outlier values.

[1319] 4. Filtering using generative artificial intelligence

[1320] The server passes text data to a generative artificial intelligence system, which then performs emotion classification. For example, it might classify the data as "satisfied" or "dissatisfied."

[1321] 5. Summary and Analysis of Results

[1322] The server aggregates the filtering results and the analysis results of the emotion engine to analyze trends for each department and overall satisfaction levels.

[1323] 6. Report generation and notification

[1324] The server generates a report based on the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[1325] As described above, this system can recognize users' emotions in real time and utilize this information in the analysis of survey results, thereby providing more detailed and reliable insights.

[1326] The following describes the processing flow.

[1327] Step 1:

[1328] The user fills out the survey form.

[1329] Users (customers or employees) answer survey questions via online forms or applications.

[1330] Enter detailed answers for each question and click the "Submit" button.

[1331] Step 2:

[1332] The device sends data to the server.

[1333] The device collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[1334] The data is sent to the server in formats such as JSON or XML.

[1335] Step 3:

[1336] The device acquires the user's voice and facial expression data.

[1337] The device uses its camera and microphone to capture the user's real-time voice and facial expression data.

[1338] This data is sent to the emotion engine.

[1339] Step 4:

[1340] The emotion engine analyzes voice and facial expression data.

[1341] The emotion engine analyzes acquired voice and facial expression data to recognize the user's emotional state (e.g., positive, negative, neutral).

[1342] Send the recognition results to the server.

[1343] Step 5:

[1344] The server receives and stores the survey data.

[1345] The server stores the survey data sent from the terminal in a buffer for temporary storage.

[1346] Convert the data to a format suitable for storage in a database.

[1347] Step 6:

[1348] The server saves data to the database.

[1349] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[1350] The inserted data is recorded in the survey data table.

[1351] Step 7:

[1352] The server cleans the survey data.

[1353] The server reads the stored data and cleans it.

[1354] Missing values ​​are detected and either imputed with the mean or excluded as invalid data. Outliers are checked and corrected in the same way.

[1355] Step 8:

[1356] The server passes data to the generative artificial intelligence.

[1357] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[1358] In particular, submit the text data of your open-ended responses.

[1359] Step 9:

[1360] Generative artificial intelligence filters the data.

[1361] Generative artificial intelligence analyzes received data and classifies emotions according to specific criteria.

[1362] For example, text responses are classified as either "positive" or "negative."

[1363] Step 10:

[1364] The server receives the results from the generative artificial intelligence.

[1365] The server receives the filtering results returned by the generative artificial intelligence.

[1366] The received data is stored back into the buffer.

[1367] Step 11:

[1368] The server aggregates the filtering results.

[1369] The server extracts filtered data and performs statistical aggregation.

[1370] Count the number of positive and negative responses, and calculate the percentage of each.

[1371] Step 12:

[1372] The server integrates the results from the emotion engine.

[1373] The server integrates the real-time emotion recognition results received from the emotion engine with the analysis results of the survey data.

[1374] This means that not only text responses but also the user's emotional state during the response process will be used for analysis.

[1375] Step 13:

[1376] The server analyzes the data.

[1377] The server performs detailed statistical analysis based on the aggregated results.

[1378] For example, calculate the average satisfaction score for each department and analyze specific demographic factors.

[1379] Step 14:

[1380] The server generates the report.

[1381] The server generates a report based on the analysis results.

[1382] The report should include visually clear formats such as text summaries, graphs, and charts.

[1383] Step 15:

[1384] The server notifies the administrator of the report.

[1385] The server will notify the administrator of the generated report via email or a dashboard.

[1386] Include a link that allows administrators to access the report.

[1387] In summary, the system's program processing is executed as a series of processes, from collecting survey data, to real-time analysis by the emotion engine, filtering by generative artificial intelligence, data analysis, and finally, sending reports to administrators.

[1388] (Example 2)

[1389] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1390] Traditional survey systems struggled to accurately grasp users' emotional states, making detailed emotion-based analysis difficult. Furthermore, missing or outlier data in the collected responses reduced the accuracy of the analysis. While generative artificial intelligence-based emotion classification was employed, the lack of real-time emotion analysis undermined the reliability of the results.

[1391] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1392] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial expressions and voice using an emotion engine that recognizes the emotional state of the user at the time of response, means for aggregating the filtering results and performing analysis based on them, and means for generating a report based on the analysis results and notifying the administrator or relevant parties. This enables detailed and reliable data analysis that incorporates real-time emotion analysis of the user.

[1393] "Customer or employee" refers to an individual who uses the system to respond to a survey.

[1394] "Survey results" refer to the response data entered by customers or employees through a survey form.

[1395] "Means of collection" refers to methods and devices for obtaining survey results from users via online forms, dedicated applications, etc.

[1396] "Methods for cleaning and preprocessing" refer to data preprocessing techniques and methods for imputing missing values ​​and correcting outliers in collected survey data.

[1397] "Generative artificial intelligence" refers to artificial intelligence models that analyze collected text data and perform sentiment classification and other classifications.

[1398] "Means for performing filtering" refers to methods or devices for passing pre-processed data to a generative artificial intelligence system to perform filtering, such as classifying the sentiment of the data.

[1399] An "emotion engine" is a system or software that analyzes the user's facial expressions and voice data in real time when they respond to questions, and recognizes the user's emotional state.

[1400] "Means of aggregating and performing analysis based on that data" refers to technologies and methods for aggregating results obtained from emotion engines and generative artificial intelligence, and then performing detailed analysis based on that data.

[1401] "Means for generating reports and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating those reports to administrators or relevant parties via email or notification systems.

[1402] This invention is a system that collects survey results from customers or employees, performs data cleaning, filters using generative artificial intelligence (AI models), and combines this with an emotion engine. This allows for efficient data analysis and report generation.

[1403] The system uses the following hardware and software:

[1404] Web browser or dedicated application: Software on a device used by the user to answer the survey.

[1405] Server: The central computer responsible for data collection, database storage, data preprocessing, analysis using generative AI models, emotion engine management, aggregation, analysis, and report generation.

[1406] Database: A system for storing collected survey data and analysis results.

[1407] Emotion Engine: Software that processes facial and voice data obtained from cameras and microphones in order to analyze the user's emotions in real time.

[1408] Generative artificial intelligence: Natural language processing (NLP) models used to classify emotions from survey responses. Examples: AI models such as GPT-4.

[1409] Python's Pandas library: Used for data cleaning (imputing missing values ​​and correcting outliers).

[1410] Matplotlib and ReportLab are used to generate reports that include graphs and charts.

[1411] Specific examples of implementation

[1412] For example, let's consider the case of conducting an employee satisfaction survey.

[1413] 1. Users access the survey form via a web browser or dedicated application and answer the survey. At this time, the device's camera and microphone are turned on to capture facial expressions and voice data.

[1414] 2. The terminal transmits the user's input and acquired facial expression / voice data to the server. The HTTPS protocol is used in this process.

[1415] 3. The server saves the received survey data to a database and preprocesses the data using the Python Pandas library. Preprocessing includes imputing missing values ​​and correcting outliers.

[1416] 4. The pre-processed data is input into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. For example, responses are classified as positive, negative, or neutral.

[1417] 5. The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server.

[1418] 6. The server aggregates the filtering results from the generative artificial intelligence and the data obtained from the emotion engine, and performs a detailed analysis. This analysis includes the percentage of positive responses, the percentage of negative responses, etc.

[1419] 7. The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, etc. These are generated using Python's Matplotlib or ReportLab.

[1420] 8. The server notifies the administrator of the generated report. Email or a dedicated notification system may be used for notification.

[1421] Examples of prompts for generative AI models

[1422] "Classify the emotions from the submitted survey responses into positive, negative, and neutral, and calculate the ratio of each. Furthermore, perform a comprehensive emotional analysis considering facial expression and voice data, and output the results as a report."

[1423] The above describes the embodiments for carrying out the present invention. This system achieves more detailed and reliable data analysis by incorporating real-time sentiment analysis of the user.

[1424] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1425] Step 1:

[1426] Data collection

[1427] Users access the survey form via a web browser or a dedicated application and fill it out. This includes answering questions and providing free-form text.

[1428] The terminal collects survey data entered by the user and sends it to the server. The data is structured in JSON format and sent using an HTTP POST request.

[1429] Input: User survey response data (text).

[1430] Output: Structured data (JSON format) sent to the server.

[1431] Step 2:

[1432] Real-time sentiment analysis

[1433] The device uses its camera and microphone to capture facial expressions and voice while the user is filling out a survey, and transmits this information to the emotion engine in real time.

[1434] The emotion engine analyzes the user's voice and facial expression data to recognize their emotional state (e.g., joy, sadness, anger). This analysis uses voice emotion recognition algorithms and facial expression recognition algorithms.

[1435] Input: User's facial expression data, voice data.

[1436] Output: Analyzed emotion data (e.g., joy, sadness, anger, etc.).

[1437] Step 3:

[1438] Receiving and cleaning data

[1439] The server receives survey data sent from the terminal and stores it in the database. Afterward, it performs data preprocessing. This preprocessing includes imputing missing values ​​and correcting outliers. The data is cleaned using the Python Pandas library.

[1440] Input: Survey data sent from the device.

[1441] Output: Clean data after preprocessing.

[1442] Step 4:

[1443] Filtering by generative artificial intelligence

[1444] The server inputs pre-processed data into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. The model analyzes the text data and classifies each response as positive, negative, neutral, etc.

[1445] Input: Clean text data.

[1446] Output: Classified sentiment data (positive, negative, neutral).

[1447] Step 5:

[1448] Summary and analysis of results

[1449] The server aggregates data obtained from generative artificial intelligence and emotion engines and performs detailed analysis. Specifically, it calculates the percentage of positive and negative responses. This analysis uses SQL queries and Python libraries (e.g., Pandas, Matplotlib).

[1450] Input: Classified sentiment data, real-time analyzed sentiment data.

[1451] Output: Aggregated results and detailed analysis results (e.g., positive response rate).

[1452] Step 6:

[1453] Report generation and notification

[1454] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements. Python's ReportLab and Matplotlib are used to generate the report. The generated report is notified to the administrator via email or a dedicated notification system.

[1455] Input: Detailed analysis results.

[1456] Output: Generated report (PDF or HTML format), notification email.

[1457] (Application Example 2)

[1458] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1459] Traditional survey systems often fail to adequately consider emotional factors when collecting feedback from customers and employees, resulting in superficial analysis of the results. Furthermore, data cleaning and filtering are frequently performed manually, making efficient data analysis difficult. Consequently, it becomes challenging to quickly and accurately obtain information that is useful to managers and stakeholders. In store feedback systems, there is a need for methods that analyze customer emotions in real time and classify and analyze emotional data using generative AI.

[1460] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1461] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial and voice data collected during pre-processing to recognize real-time emotions, means for aggregating the filtering results and real-time emotion data and performing analysis based on them, and means for generating a report based on the analysis results and notifying administrators or relevant parties. This enables detailed and reliable feedback analysis that takes customer emotions into account, allowing administrators and relevant parties to quickly obtain useful information.

[1462] "Means of collecting survey results" refers to methods or equipment used to collect responses from customers or employees.

[1463] "Methods for cleaning and preprocessing data" refer to methods and techniques for preparing collected data for analysis by imputing or correcting missing or outlier values.

[1464] "Methods for passing data to a generative artificial intelligence to perform filtering" refer to methods or systems for inputting pre-processed data into a generative artificial intelligence and having it classify it into appropriate emotions or categories.

[1465] "Means for analyzing facial and voice data to recognize emotions in real time" refers to technologies and devices that use collected facial and voice data to analyze and recognize a user's emotional state in real time.

[1466] "Means for aggregating data and performing analysis based on it" refers to methods and systems that integrate and aggregate filtered data and real-time sentiment data to perform statistical and sentimental analysis.

[1467] "Means for generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating them to administrators or relevant parties.

[1468] This invention is an advanced system for efficiently collecting and analyzing feedback from customers or employees. This system operates using a combination of hardware and software and includes the following means:

[1469] hardware

[1470] 1. Smart devices

[1471] Smartphones and tablets are used for collecting survey responses. These devices have built-in cameras and microphones, allowing them to collect user facial expressions and audio data.

[1472] 2. Server

[1473] A server equipped with a database and analysis engine. This enables data preprocessing, filtering by generative artificial intelligence, real-time sentiment analysis, data aggregation and analysis, and report generation and notification.

[1474] software

[1475] 1. Emotional Engine

[1476] The system analyzes the user's emotions in real time using facial expression analysis libraries (e.g., "OpenFace," "Affectiva") and voice emotion analysis software (e.g., "IBM Watson Tone Analyzer").

[1477] 2. Generative Artificial Intelligence

[1478] Generative artificial intelligence such as BERT and GPT-3 (OpenAI) will be used to classify the sentiment of text data from survey responses.

[1479] 3. Database

[1480] We use MySQL or PostgreSQL to store and manage the collected and pre-processed data.

[1481] 4. Analytics

[1482] We will use Python and its libraries (e.g., pandas, numpy, matplotlib) to analyze data and generate reports.

[1483] 5. Notification System

[1484] Use email services (e.g., AWS SES) or messaging services (e.g., Twilio) to notify administrators and relevant parties of the report.

[1485] System operation

[1486] Data collection and cleaning

[1487] The terminal collects survey responses from customers and employees. In addition to the text data entered by the user, it also collects facial expressions and voice data using the camera and microphone. The collected data is sent to a server where missing values ​​are imputed and outliers are corrected.

[1488] Filtering by generative artificial intelligence

[1489] The pre-processed data is passed to a generative artificial intelligence (AI) for emotion classification. The AI ​​classifies the data into positive, negative, neutral, etc.

[1490] Real-time sentiment analysis

[1491] The collected facial and voice data is analyzed in real time by an emotion engine to recognize the user's emotional state.

[1492] Data aggregation and analysis

[1493] The filtering results and real-time sentiment data are integrated by the server for detailed analysis. For example, by statistically analyzing the satisfaction and dissatisfaction levels of each product, areas for improvement in the store can be identified.

[1494] Report generation and notification

[1495] Based on the analysis results, a report is generated that includes text summaries, graphs, charts, and other elements. This report is then sent to administrators and relevant parties through a notification system.

[1496] Specific example

[1497] Examples of prompt statements include the following:

[1498] Please generate an emotional analysis report for product "A". Please perform the analysis based on the following data set. We would like to know the ratio of positive, negative, and neutral responses.

[1499] In this way, this system enables detailed feedback analysis, including the actual emotions of customers, allowing managers and stakeholders to quickly identify areas for improvement.

[1500] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1501] Step 1:

[1502] Data collection

[1503] When collecting survey results entered by customers or employees, the device also uses its camera and microphone to simultaneously acquire facial expressions and voice data.

[1504] Input: Customer or employee survey text, facial expression data, audio data

[1505] Output: Collected questionnaire text, facial expression data, audio data

[1506] Step 2:

[1507] Data transmission

[1508] The device transmits the collected survey text, facial expression data, and audio data to the server.

[1509] Input: Collected questionnaire text, facial expression data, audio data

[1510] Output: Survey text, facial expression data, and audio data sent to the server.

[1511] Step 3:

[1512] Data cleaning and preprocessing

[1513] The server fills in missing values ​​and corrects outliers from the received data. For example, it calculates fill-in values ​​for questions with missing answers and corrects extremely abnormal values.

[1514] Input: Survey text, facial expression data, and audio data sent from the device.

[1515] Output: Cleaned and pre-processed questionnaire text, facial expression data, audio data

[1516] Step 4:

[1517] Filtering by generative artificial intelligence

[1518] The server inputs pre-processed data into a generative artificial intelligence (e.g., GPT-3) to perform sentiment classification. The generative AI classifies the data into positive, negative, or neutral and returns the result.

[1519] Input: Preprocessed survey text

[1520] Output: Emotionally classified survey data (positive, negative, neutral)

[1521] Step 5:

[1522] Real-time sentiment analysis

[1523] The server analyzes facial and voice data collected using facial expression analysis libraries and voice emotion analysis software in real time to recognize the user's emotional state.

[1524] Input: Preprocessed facial expression data and audio data

[1525] Output: Recognized real-time sentiment data

[1526] Step 6:

[1527] Data aggregation and analysis

[1528] The server integrates and aggregates the filtering results and real-time sentiment data, and performs statistical and emotional analysis. For example, it calculates the percentage of positive emotions, negative emotions, and neutral emotions.

[1529] Input: Sentiment-classified survey data, real-time sentiment data

[1530] Output: Aggregated and analyzed results (e.g., sentiment ratio, trend analysis, etc.)

[1531] Step 7:

[1532] Report generation

[1533] The server generates a report that includes text summaries, graphs, and charts based on the aggregated and analyzed results.

[1534] Input: Aggregation and analysis results

[1535] Output: Report (text summary, graphs, charts, etc.)

[1536] Step 8:

[1537] notification

[1538] The server uses email or messaging services to notify administrators or relevant parties of the reports it generates.

[1539] Input: Generated report

[1540] Output: Notification to administrators or relevant parties (email, message, etc.)

[1541] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1542] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1543] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1544] [Fourth Embodiment]

[1545] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1546] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1547] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1548] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1549] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1550] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1551] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1552] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1553] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1554] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1555] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1556] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1557] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1558] This invention is a system that efficiently collects survey results from customers or employees, cleans them, filters them using generative artificial intelligence, and analyzes them, ultimately generating a report.

[1559] System Overview

[1560] This system includes means for collecting survey results, means for pre-processing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[1561] Program processing

[1562] The program's processing can be explained in natural language as follows:

[1563] 1. Data Collection

[1564] Users enter information through a survey form. This survey form operates on a web browser or a dedicated application.

[1565] The terminal sends this input data to the server. The transmitted data is received by the server and initially stored.

[1566] 2. Data preprocessing

[1567] The server stores the collected survey data in a database. Then, it extracts the data for preprocessing.

[1568] The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[1569] 3. Filtering using generative artificial intelligence

[1570] The server passes the cleaned data to a generative artificial intelligence (AI). This AI, for example, analyzes open-ended text data and classifies the emotions expressed as "positive" or "negative."

[1571] The generative artificial intelligence returns the filtered results. These results are then saved again on the server side.

[1572] 4. Summary and Analysis of Results

[1573] The server aggregates the filtering results and performs a detailed analysis based on them. For example, it calculates the percentage of positive responses and the percentage of negative responses.

[1574] Furthermore, statistical analysis will be conducted on specific question items and by department.

[1575] 5. Report generation and notification

[1576] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts.

[1577] The server generates reports and notifies the administrator. These notifications are sent via email or a dashboard.

[1578] Specific example

[1579] For example, consider a case where a company conducts a survey to investigate employee satisfaction. In this survey, employees answer questions about their work environment and job satisfaction.

[1580] 1. Questionnaire response

[1581] Users (employees) access a survey form and answer questions about the work environment.

[1582] The device sends the response data to the server.

[1583] 2. Storing in a database

[1584] The server stores the received survey data in a database.

[1585] 3. Data Cleaning

[1586] The server checks for missing or outlier values ​​and fills in the data as needed.

[1587] 4. AI-based filtering

[1588] The server passes the text response to a generative artificial intelligence system, which then classifies its content as positive or negative.

[1589] 5. Summary and Analysis of Results

[1590] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[1591] 6. Report generation and notification

[1592] The server generates a report containing the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[1593] In this way, this system can efficiently collect, preprocess, and analyze survey results from customers and employees, and provide the results as reports.

[1594] The following describes the processing flow.

[1595] Step 1:

[1596] The user fills out the survey form.

[1597] Users (customers or employees) respond to surveys via online forms or applications.

[1598] Enter your answers for each question, and then click the "Submit" button.

[1599] Step 2:

[1600] The device sends data to the server.

[1601] The device (PC or smartphone) collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[1602] The data sent is often in formats such as JSON or XML.

[1603] Step 3:

[1604] The server receives the data.

[1605] The server temporarily stores the received survey data in a buffer.

[1606] Next, the data is converted into a format suitable for storage in the database.

[1607] Step 4:

[1608] The server saves data to the database.

[1609] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[1610] The inserted data is recorded in the table.

[1611] Step 5:

[1612] The server cleans up the data.

[1613] The server reads the stored data and cleans it.

[1614] Specifically, missing values ​​are detected and either imputed with the mean or removed as invalid data. Outliers are checked and corrected in the same way.

[1615] Step 6:

[1616] The server passes data to the generative artificial intelligence.

[1617] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[1618] The data to be submitted will include, in particular, open-ended text data.

[1619] Step 7:

[1620] Generative artificial intelligence filters the data.

[1621] Generative artificial intelligence analyzes received data and filters it according to specific criteria.

[1622] For example, classify users' emotions as either "positive" or "negative."

[1623] Step 8:

[1624] The server receives the results from the generative artificial intelligence.

[1625] The server receives the filtering results returned by the generative artificial intelligence.

[1626] The received data is stored back into the buffer.

[1627] Step 9:

[1628] The server aggregates the filtered data.

[1629] The server extracts the filtering results stored in the buffer and performs statistical aggregation.

[1630] For example, count the number of positive and negative responses and calculate the percentage of each.

[1631] Step 10:

[1632] The server analyzes the data.

[1633] The server analyzes the aggregated data and performs detailed statistical analysis.

[1634] For example, you could calculate the average satisfaction score for each department and identify which departments receive high ratings.

[1635] Step 11:

[1636] The server generates the report.

[1637] The server generates a report based on the analysis results.

[1638] The report should be presented in a visually clear format, including text summaries, graphs, and charts.

[1639] Step 12:

[1640] The server notifies the administrator of the report.

[1641] The server sends a notification to the administrator's terminal to send the generated report.

[1642] The notification should include a link to the report, making it easily accessible to administrators.

[1643] The above explains in detail the program processing of this system, which includes a series of processes from collecting survey data from users, filtering and analyzing it using generative artificial intelligence, and notifying administrators of reports.

[1644] (Example 1)

[1645] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1646] Traditional survey analysis systems faced challenges such as the time and effort required for data preprocessing, sentiment classification, result analysis, and report generation. Furthermore, they suffered from accuracy issues in sentiment classification and insufficient visualization of analysis results. This made it difficult for administrators to quickly and accurately obtain useful information for making appropriate decisions.

[1647] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1648] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for aggregating the filtering results and analyzing the proportion of positive and negative emotions, and means for generating a report including a visual chart based on the analysis results and notifying the administrator or relevant parties. This automates everything from data pre-processing to emotion classification, result analysis, and visual report generation, enabling administrators to obtain information quickly and accurately.

[1649] "Customer or employee survey results" refers to data from surveys conducted by a company or organization among its customers or employees.

[1650] "Means of collection" refers to the technical means of receiving survey results and storing them in a database or storage.

[1651] "Methods for cleaning and preprocessing" refer to processes and technical means for removing missing or outlier values ​​from collected data and converting it into an analyzable format.

[1652] "Generative artificial intelligence" refers to AI models used to perform tasks such as natural language processing and sentiment analysis.

[1653] "Means of filtering" refers to technical means of inputting pre-processed data into a generative artificial intelligence system and classifying or selecting the data according to specific criteria.

[1654] "Means of aggregation and analysis" refer to technical means of statistically aggregating filtered data and analyzing patterns and trends.

[1655] A "report including visual charts" is a report that presents analysis results in a visual format such as bar graphs or pie charts.

[1656] "Means of notifying administrators or stakeholders" refers to technical means of notifying stakeholders of generated reports via email or dashboards.

[1657] "Methods for imputing missing values" refer to technical means of filling in missing values ​​in a dataset based on statistical methods or known data.

[1658] "Methods for correcting outliers" refer to technical means that detect values ​​in a dataset that deviate significantly from the normal range and correct them to bring them back within an appropriate range.

[1659] This invention relates to a system that efficiently collects survey results from customers or employees, performs preprocessing, filtering using generative artificial intelligence, analysis, and report generation. In this system, the entire process, from data collection to report notification, is automated.

[1660] System configuration and hardware / software used

[1661] 1. Data Collection

[1662] Users access a dedicated survey form and enter their answers to the questions. This survey form operates on a web browser (e.g., Google Chrome, Mozilla Firefox) or a dedicated application.

[1663] The terminal transmits the survey data entered by the user to the server in real time. The HTTPS protocol is used for this communication to ensure security.

[1664] 2. Data preprocessing

[1665] The server stores the received survey data in a temporary storage area and then formally saves it to a database (e.g., MySQL, PostgreSQL).

[1666] The server checks the data extracted from the database for missing or outlier values ​​and cleans it as needed. Missing values ​​are imputed using the median or mean, and outliers are detected and a warning is issued.

[1667] 3. Filtering using generative artificial intelligence

[1668] The server passes the cleaned data to a generative artificial intelligence (e.g., OpenAI GP T-3) for sentiment classification. Specifically, it classifies the data as either "positive" or "negative" through text analysis.

[1669] The classification results returned by the generative artificial intelligence are saved to the server, and metadata necessary for analysis (analysis date and time, model version, etc.) is also added.

[1670] 4. Summary and Analysis of Results

[1671] The server aggregates and analyzes the filtered data. This includes calculating the percentage of positive and negative responses, and performing detailed statistical analysis on specific questions.

[1672] The analytical methods used include regression analysis and clustering.

[1673] 5. Report generation and notification

[1674] The server generates a report that includes visual charts (e.g., bar graphs, pie charts) based on the analysis results.

[1675] The server notifies administrators or relevant parties of the generated reports. These notifications are sent via email or a dedicated dashboard.

[1676] Examples of specific cases and prompt statements

[1677] For example, if a company conducts a survey to investigate employee satisfaction, this system will operate as follows:

[1678] Users (employees) access a survey form and answer questions about the work environment.

[1679] The device sends the response data to the server.

[1680] The server stores the received survey data in a database and performs checks and imputations for missing or outlier values.

[1681] The server passes the text responses to a generative artificial intelligence system, which then classifies their content as positive or negative.

[1682] The server aggregates the classification results and analyzes the satisfaction trends for each department.

[1683] The server generates a report containing the analysis results and notifies the administrator.

[1684] Examples of prompts to input into a generative AI model:

[1685] Please classify the following open-ended text data as positive or negative.

[1686] Answer 1: "The workplace atmosphere is very good, but the workload is too much."

[1687] Answer 2: "My boss is kind and helpful. I am very satisfied."

[1688] Answer 3: "The work is boring and unstimulating. Improvement is needed."

[1689] In this way, the system efficiently collects, preprocesses, and analyzes survey results, and provides the results as a report.

[1690] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1691] Step 1:

[1692] User survey input

[1693] Users access a dedicated survey form and answer the questions. This survey form operates on a web browser (e.g., Google Chrome, Mozilla Firefox) or a dedicated application.

[1694] Input: User survey response data

[1695] Output: Survey response data sent to the server via the terminal.

[1696] Step 2:

[1697] Sending data from the terminal to the server

[1698] The terminal sends the survey data entered by the user to the server using the HTTPS protocol.

[1699] Input: User survey response data

[1700] Output: Survey response data received by the server

[1701] Step 3:

[1702] Data reception and temporary storage by the server

[1703] The server receives the survey data sent from the terminal and saves it to a temporary storage area. A database system (e.g., MySQL, PostgreSQL) is used for this storage.

[1704] Input: Survey response data sent from the device.

[1705] Output: Data stored in the temporary storage area

[1706] Step 4:

[1707] Data preprocessing by the server

[1708] The server extracts survey data from the database and checks for missing or outlier values. Missing values ​​are imputed using the median or mean, and outliers are detected and corrected.

[1709] Input: Data from temporary storage area

[1710] Output: Cleaned and pre-processed data

[1711] Step 5:

[1712] Data transmission and filtering by the server to the generative artificial intelligence.

[1713] The server passes the pre-processed data to the generative artificial intelligence. The generative AI uses the prompt "Classify the following text as positive or negative" to categorize the emotions.

[1714] Input: Cleaned and pre-processed data

[1715] Output: Filtering results (positive, negative) returned by the generative artificial intelligence.

[1716] Step 6:

[1717] Server saving of filtering results

[1718] The server stores the filtering results received from the generative artificial intelligence in a database. At this time, metadata such as the analysis date and time and model version are also recorded.

[1719] Input: Filtered results from generative artificial intelligence

[1720] Output: Filtering results stored in the database

[1721] Step 7:

[1722] Server-based aggregation and analysis of filtering results

[1723] The server aggregates the filtering results and calculates the percentage of positive and negative responses. It also performs detailed statistical analysis for specific questions and departments.

[1724] Input: Database containing filtered results

[1725] Output: Aggregated and analyzed data

[1726] Step 8:

[1727] Server-based report generation and notification

[1728] The server generates a report containing visual charts from the aggregated and analyzed results. The generated report is notified to administrators or relevant parties. Notifications are made via email or a dedicated dashboard.

[1729] Input: Aggregated and analyzed data

[1730] Output: Report notified to administrator or relevant party

[1731] Through the steps described above, this system can efficiently automate and perform a series of processes, from data collection to report notification.

[1732] (Application Example 1)

[1733] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1734] Traditional survey systems struggle not only to collect feedback from customers and employees, but also to efficiently clean, analyze, and quickly implement countermeasures based on that feedback. Furthermore, the time-consuming process of data aggregation and analysis makes it difficult to immediately reflect the insights gained from the feedback. Therefore, there is a need for real-time service improvement and rapid problem resolution.

[1735] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1736] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, and means for sending the generated report to an administrator using electronic messages. This makes it possible to efficiently carry out everything from collecting and analyzing survey results to generating and notifying reports.

[1737] "Customer or employee" refers to a user who provides survey responses to this system.

[1738] "Survey results" refers to the content of responses given by customers or employees to a survey.

[1739] "Means of collection" refers to the methods and equipment used to obtain survey results and transmit them to a server.

[1740] "Methods for cleaning and preprocessing" refer to methods and equipment that fill in missing or outlier values ​​in collected survey data, making the data analyzable.

[1741] "Generative artificial intelligence" refers to intelligent systems that analyze text data using natural language processing and machine learning techniques.

[1742] "Means of filtering" refers to methods and devices that use generative artificial intelligence to analyze survey data, classify information, or extract information based on specific conditions.

[1743] "Means for aggregating filtering results and performing analysis based on them" refers to methods and equipment for compiling the results analyzed by generative artificial intelligence and performing statistical analysis and evaluation.

[1744] "Means of generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for organizing analysis results, creating reports in visual or text format, and informing administrators or relevant parties via email or other means.

[1745] "Means of sending to an administrator using electronic messages" refers to methods or devices for sending generated reports to an administrator via email or other digital communication means.

[1746] The present invention is a system for efficiently collecting survey results from customers or employees, cleaning, filtering using generative artificial intelligence, and analyzing the results, and finally generating a report. This system includes means for collecting survey results, means for preprocessing the collected data, means for filtering the data using generative artificial intelligence, means for aggregating and analyzing the filtering results, and means for generating the analysis results as a report and notifying the administrator.

[1747] System Overview

[1748] 1. Data Collection

[1749] Users (customers or employees) answer the survey via a smartphone application. This survey form operates on a web browser or a dedicated application. The device sends this input data to the server, where it is received and initially stored.

[1750] 2. Data preprocessing

[1751] The server stores the collected survey data in a database. It then extracts the data for preprocessing. The server checks for missing values ​​and cleans the data as needed. This cleaning includes imputing missing values ​​and correcting outliers.

[1752] 3. Filtering using generative artificial intelligence

[1753] The server passes the cleaned data to a generative artificial intelligence (AI). This AI analyzes, for example, open-ended text data and classifies the emotions expressed as "positive" or "negative." The generative AI returns the filtered results, which are then stored again on the server.

[1754] 4. Summary and Analysis of Results

[1755] The server aggregates the filtering results and performs detailed analysis based on them. For example, it calculates the percentage of positive and negative responses. Furthermore, it also performs statistical analysis for specific questions or by department.

[1756] 5. Report generation and notification

[1757] The server generates a report based on the aggregated and analyzed results. This report includes visual information such as graphs and charts. The server sends the generated report to the administrator via electronic message.

[1758] Specific examples

[1759] For example, consider a case where a customer satisfaction survey is conducted at a physical store. In this survey, customers answer questions about things like "the atmosphere of the store" and "the service provided by the staff."

[1760] 1. Questionnaire response

[1761] Users (customers) access a smartphone app and answer questions about the store's atmosphere and staff service.

[1762] 2. Storing in a database

[1763] The terminal sends the response data to the server. The server stores the received survey data in a database.

[1764] 3. Data Cleaning

[1765] The server checks for missing or outlier values ​​and fills in the data as needed.

[1766] 4. AI-based filtering

[1767] The server passes the text responses to a generative artificial intelligence system, which then classifies the content as either "positive" or "negative." An example of a prompt is: "We'd love to hear your feedback: How did you feel about the service provided by our staff?"

[1768] 5. Summary and Analysis of Results

[1769] The server aggregates the classification results and analyzes trends in customer satisfaction with the store's atmosphere, for example.

[1770] 6. Report generation and notification

[1771] The server generates a report containing the analysis results and notifies the administrator via email. This report is used to consider ways to improve store services.

[1772] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1773] Step 1:

[1774] Users access the survey form and answer the questions. Specifically, customers or employees use a smartphone application to answer questions entered into the survey form using text or multiple-choice options. The entered data (e.g., free-text responses, multiple-choice answers, etc.) is reflected in the survey form.

[1775] Step 2:

[1776] The terminal sends the survey results entered by the user to the server. The transmitted data is received by the server for subsequent processing and initially stored in the database. The input is the survey response data, and the output is the raw data stored in the database.

[1777] Step 3:

[1778] The server preprocesses the survey data stored in the database. Specifically, it imputes missing values ​​and corrects outliers. This results in cleaned data. The input is the stored raw data, and the output is the cleaned data.

[1779] Step 4:

[1780] The server passes the cleaned data to a generative artificial intelligence (AI) for filtering. The AI ​​uses a natural language processing model to analyze the text data and classify the emotion of each response as either "positive" or "negative." In this process, the generative AI model receives text data as input and outputs the result of the emotion classification. The input is the cleaned data, and the output is the result of the emotion classification.

[1781] Step 5:

[1782] The server aggregates data classified by generative artificial intelligence and performs overall trend analysis and statistical analysis. Specifically, it calculates the percentage of positive and negative responses. The input is the result data of sentiment classification, and the output is the aggregated and statistically analyzed results.

[1783] Step 6:

[1784] The server generates a report based on the aggregated and analyzed results and sends it to the administrator via electronic message. The report includes visual information such as graphs and charts. Specifically, the process involves organizing the analysis results, converting them into a visual format, and sending them to the administrator via email. The input is the aggregated and analyzed results, and the output is the report sent via email.

[1785] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1786] This invention is a system that collects survey results from customers or employees, cleans the data, filters it using generative artificial intelligence, and combines it with an emotion engine to efficiently analyze the data and generate reports.

[1787] System Overview

[1788] This system utilizes an emotion engine, in particular, to recognize user emotions, streamlining the entire process from survey data collection and analysis to report generation. The system includes the following:

[1789] 1. Data collection methods:

[1790] A method for collecting survey results from users. This is done via online forms or dedicated apps.

[1791] 2. Pre-treatment means:

[1792] A method for cleaning and preprocessing collected data. This includes imputing missing values ​​and correcting outliers.

[1793] 3. Filtering methods using generative artificial intelligence:

[1794] A method of passing pre-processed data to a generative artificial intelligence system to perform emotion classification.

[1795] 4. Emotional Engine:

[1796] A method for analyzing user voice, facial expressions, and text data from survey responses to recognize emotions in real time.

[1797] 5. Analysis methods:

[1798] A method for aggregating filtered data and performing analysis based on that data.

[1799] 6. Report generation and notification methods:

[1800] A means of generating a report based on the analysis results and notifying administrators or relevant parties.

[1801] Program processing

[1802] The program's processing can be explained in natural language as follows:

[1803] 1. Data Collection

[1804] Users access the survey form and answer each question. This form operates within a web browser or application.

[1805] The terminal collects data entered by the user and sends it to the server.

[1806] 2. Emotional analysis using an emotion engine

[1807] The device uses its camera and microphone to capture facial expressions and voice while the user is responding, and transmits this information to the emotion engine in real time.

[1808] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state.

[1809] 3. Receiving and cleaning data

[1810] The server stores the received survey data in a database and performs preprocessing. During this process, missing values ​​are imputed and outliers are corrected.

[1811] 4. Filtering using generative artificial intelligence

[1812] The server passes the pre-processed data to a generative artificial intelligence system, which performs sentiment classification of the text data. The system then receives the results, categorized as positive, negative, neutral, etc.

[1813] 5. Summary and Analysis of Results

[1814] The server aggregates the filtering results and performs a detailed analysis. For example, it calculates the percentage of positive and negative responses. Real-time sentiment data from the sentiment engine is also used in the analysis.

[1815] 6. Report generation and notification

[1816] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements.

[1817] The server notifies the administrator of the reports it generates, making them easily accessible to the administrator.

[1818] Specific example

[1819] For example, when conducting an employee satisfaction survey:

[1820] 1. Questionnaire response

[1821] The user answers a series of questions. Simultaneously, the user's facial expressions and voice data are captured using the camera and microphone.

[1822] 2. Real-time sentiment analysis

[1823] The emotion engine analyzes facial expressions and voice in real time to recognize the user's emotional state.

[1824] 3. Storing in the database and cleaning

[1825] The server stores the received survey data in a database and cleans up any missing or outlier values.

[1826] 4. Filtering using generative artificial intelligence

[1827] The server passes text data to a generative artificial intelligence system, which then performs emotion classification. For example, it might classify the data as "satisfied" or "dissatisfied."

[1828] 5. Summary and Analysis of Results

[1829] The server aggregates the filtering results and the analysis results of the emotion engine to analyze trends for each department and overall satisfaction levels.

[1830] 6. Report generation and notification

[1831] The server generates a report based on the analysis results and notifies the administrator. This report is used to consider measures to improve employee satisfaction.

[1832] As described above, this system can recognize users' emotions in real time and utilize this information in the analysis of survey results, thereby providing more detailed and reliable insights.

[1833] The following describes the processing flow.

[1834] Step 1:

[1835] The user fills out the survey form.

[1836] Users (customers or employees) answer survey questions via online forms or applications.

[1837] Enter detailed answers for each question and click the "Submit" button.

[1838] Step 2:

[1839] The device sends data to the server.

[1840] The device collects survey data entered by the user and securely transmits it to the server using the HTTPS protocol.

[1841] The data is sent to the server in formats such as JSON or XML.

[1842] Step 3:

[1843] The device acquires the user's voice and facial expression data.

[1844] The device uses its camera and microphone to capture the user's real-time voice and facial expression data.

[1845] This data is sent to the emotion engine.

[1846] Step 4:

[1847] The emotion engine analyzes voice and facial expression data.

[1848] The emotion engine analyzes acquired voice and facial expression data to recognize the user's emotional state (e.g., positive, negative, neutral).

[1849] Send the recognition results to the server.

[1850] Step 5:

[1851] The server receives and stores the survey data.

[1852] The server stores the survey data sent from the terminal in a buffer for temporary storage.

[1853] Convert the data to a format suitable for storage in a database.

[1854] Step 6:

[1855] The server saves data to the database.

[1856] The server retrieves data from the buffer and inserts it into the database using an SQL query.

[1857] The inserted data is recorded in the survey data table.

[1858] Step 7:

[1859] The server cleans the survey data.

[1860] The server reads the stored data and cleans it.

[1861] Missing values ​​are detected and either imputed with the mean or excluded as invalid data. Outliers are checked and corrected in the same way.

[1862] Step 8:

[1863] The server passes data to the generative artificial intelligence.

[1864] The server formats the cleaned data and sends a POST request to the generative artificial intelligence API endpoint.

[1865] In particular, submit the text data of your open-ended responses.

[1866] Step 9:

[1867] Generative artificial intelligence filters the data.

[1868] Generative artificial intelligence analyzes received data and classifies emotions according to specific criteria.

[1869] For example, text responses are classified as either "positive" or "negative."

[1870] Step 10:

[1871] The server receives the results from the generative artificial intelligence.

[1872] The server receives the filtering results returned by the generative artificial intelligence.

[1873] The received data is stored back into the buffer.

[1874] Step 11:

[1875] The server aggregates the filtering results.

[1876] The server extracts filtered data and performs statistical aggregation.

[1877] Count the number of positive and negative responses, and calculate the percentage of each.

[1878] Step 12:

[1879] The server integrates the results from the emotion engine.

[1880] The server integrates the real-time emotion recognition results received from the emotion engine with the analysis results of the survey data.

[1881] This means that not only text responses but also the user's emotional state during the response process will be used for analysis.

[1882] Step 13:

[1883] The server analyzes the data.

[1884] The server performs detailed statistical analysis based on the aggregated results.

[1885] For example, calculate the average satisfaction score for each department and analyze specific demographic factors.

[1886] Step 14:

[1887] The server generates the report.

[1888] The server generates a report based on the analysis results.

[1889] The report should include visually clear formats such as text summaries, graphs, and charts.

[1890] Step 15:

[1891] The server notifies the administrator of the report.

[1892] The server will notify the administrator of the generated report via email or a dashboard.

[1893] Include a link that allows administrators to access the report.

[1894] In summary, the system's program processing is executed as a series of processes, from collecting survey data, to real-time analysis by the emotion engine, filtering by generative artificial intelligence, data analysis, and finally, sending reports to administrators.

[1895] (Example 2)

[1896] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1897] Traditional survey systems struggled to accurately grasp users' emotional states, making detailed emotion-based analysis difficult. Furthermore, missing or outlier data in the collected responses reduced the accuracy of the analysis. While generative artificial intelligence-based emotion classification was employed, the lack of real-time emotion analysis undermined the reliability of the results.

[1898] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1899] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial expressions and voice using an emotion engine that recognizes the emotional state of the user at the time of response, means for aggregating the filtering results and performing analysis based on them, and means for generating a report based on the analysis results and notifying the administrator or relevant parties. This enables detailed and reliable data analysis that incorporates real-time emotion analysis of the user.

[1900] "Customer or employee" refers to an individual who uses the system to respond to a survey.

[1901] "Survey results" refer to the response data entered by customers or employees through a survey form.

[1902] "Means of collection" refers to methods and devices for obtaining survey results from users via online forms, dedicated applications, etc.

[1903] "Methods for cleaning and preprocessing" refer to data preprocessing techniques and methods for imputing missing values ​​and correcting outliers in collected survey data.

[1904] "Generative artificial intelligence" refers to artificial intelligence models that analyze collected text data and perform sentiment classification and other classifications.

[1905] "Means for performing filtering" refers to methods or devices for passing pre-processed data to a generative artificial intelligence system to perform filtering, such as classifying the sentiment of the data.

[1906] An "emotion engine" is a system or software that analyzes the user's facial expressions and voice data in real time when they respond to questions, and recognizes the user's emotional state.

[1907] "Means of aggregating and performing analysis based on that data" refers to technologies and methods for aggregating results obtained from emotion engines and generative artificial intelligence, and then performing detailed analysis based on that data.

[1908] "Means for generating reports and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating those reports to administrators or relevant parties via email or notification systems.

[1909] This invention is a system that collects survey results from customers or employees, performs data cleaning, filters using generative artificial intelligence (AI models), and combines this with an emotion engine. This allows for efficient data analysis and report generation.

[1910] The system uses the following hardware and software:

[1911] Web browser or dedicated application: Software on a device used by the user to answer the survey.

[1912] Server: The central computer responsible for data collection, database storage, data preprocessing, analysis using generative AI models, emotion engine management, aggregation, analysis, and report generation.

[1913] Database: A system for storing collected survey data and analysis results.

[1914] Emotion Engine: Software that processes facial and voice data obtained from cameras and microphones in order to analyze the user's emotions in real time.

[1915] Generative artificial intelligence: Natural language processing (NLP) models used to classify emotions from survey responses. Examples: AI models such as GPT-4.

[1916] Python's Pandas library: Used for data cleaning (imputing missing values ​​and correcting outliers).

[1917] Matplotlib and ReportLab are used to generate reports that include graphs and charts.

[1918] Specific examples of implementation

[1919] For example, let's consider the case of conducting an employee satisfaction survey.

[1920] 1. Users access the survey form via a web browser or dedicated application and answer the survey. At this time, the device's camera and microphone are turned on to capture facial expressions and voice data.

[1921] 2. The terminal transmits the user's input and acquired facial expression / voice data to the server. The HTTPS protocol is used in this process.

[1922] 3. The server saves the received survey data to a database and preprocesses the data using the Python Pandas library. Preprocessing includes imputing missing values ​​and correcting outliers.

[1923] 4. The pre-processed data is input into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. For example, responses are classified as positive, negative, or neutral.

[1924] 5. The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server.

[1925] 6. The server aggregates the filtering results from the generative artificial intelligence and the data obtained from the emotion engine, and performs a detailed analysis. This analysis includes the percentage of positive responses, the percentage of negative responses, etc.

[1926] 7. The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, etc. These are generated using Python's Matplotlib or ReportLab.

[1927] 8. The server notifies the administrator of the generated report. Email or a dedicated notification system may be used for notification.

[1928] Examples of prompts for generative AI models

[1929] "Classify the emotions from the submitted survey responses into positive, negative, and neutral, and calculate the ratio of each. Furthermore, perform a comprehensive emotional analysis considering facial expression and voice data, and output the results as a report."

[1930] The above describes the embodiments for carrying out the present invention. This system achieves more detailed and reliable data analysis by incorporating real-time sentiment analysis of the user.

[1931] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1932] Step 1:

[1933] Data collection

[1934] Users access the survey form via a web browser or a dedicated application and fill it out. This includes answering questions and providing free-form text.

[1935] The terminal collects survey data entered by the user and sends it to the server. The data is structured in JSON format and sent using an HTTP POST request.

[1936] Input: User survey response data (text).

[1937] Output: Structured data (JSON format) sent to the server.

[1938] Step 2:

[1939] Real-time sentiment analysis

[1940] The device uses its camera and microphone to capture facial expressions and voice while the user is filling out a survey, and transmits this information to the emotion engine in real time.

[1941] The emotion engine analyzes the user's voice and facial expression data to recognize their emotional state (e.g., joy, sadness, anger). This analysis uses voice emotion recognition algorithms and facial expression recognition algorithms.

[1942] Input: User's facial expression data, voice data.

[1943] Output: Analyzed emotion data (e.g., joy, sadness, anger, etc.).

[1944] Step 3:

[1945] Receiving and cleaning data

[1946] The server receives survey data sent from the terminal and stores it in the database. Afterward, it performs data preprocessing. This preprocessing includes imputing missing values ​​and correcting outliers. The data is cleaned using the Python Pandas library.

[1947] Input: Survey data sent from the device.

[1948] Output: Clean data after preprocessing.

[1949] Step 4:

[1950] Filtering by generative artificial intelligence

[1951] The server inputs pre-processed data into a generative artificial intelligence model (e.g., GPT-4) to classify the sentiment of the data. The model analyzes the text data and classifies each response as positive, negative, neutral, etc.

[1952] Input: Clean text data.

[1953] Output: Classified sentiment data (positive, negative, neutral).

[1954] Step 5:

[1955] Summary and analysis of results

[1956] The server aggregates data obtained from generative artificial intelligence and emotion engines and performs detailed analysis. Specifically, it calculates the percentage of positive and negative responses. This analysis uses SQL queries and Python libraries (e.g., Pandas, Matplotlib).

[1957] Input: Classified sentiment data, real-time analyzed sentiment data.

[1958] Output: Aggregated results and detailed analysis results (e.g., positive response rate).

[1959] Step 6:

[1960] Report generation and notification

[1961] The server generates a report based on the analysis results. The report includes text summaries, graphs, charts, and other elements. Python's ReportLab and Matplotlib are used to generate the report. The generated report is notified to the administrator via email or a dedicated notification system.

[1962] Input: Detailed analysis results.

[1963] Output: Generated report (PDF or HTML format), notification email.

[1964] (Application Example 2)

[1965] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1966] Traditional survey systems often fail to adequately consider emotional factors when collecting feedback from customers and employees, resulting in superficial analysis of the results. Furthermore, data cleaning and filtering are frequently performed manually, making efficient data analysis difficult. Consequently, it becomes challenging to quickly and accurately obtain information that is useful to managers and stakeholders. In store feedback systems, there is a need for methods that analyze customer emotions in real time and classify and analyze emotional data using generative AI.

[1967] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1968] In this invention, the server includes means for collecting survey results from customers or employees, means for cleaning and pre-processing the collected survey data, means for passing the pre-processed data to a generative artificial intelligence system for filtering, means for analyzing facial and voice data collected during pre-processing to recognize real-time emotions, means for aggregating the filtering results and real-time emotion data and performing analysis based on them, and means for generating a report based on the analysis results and notifying administrators or relevant parties. This enables detailed and reliable feedback analysis that takes customer emotions into account, allowing administrators and relevant parties to quickly obtain useful information.

[1969] "Means of collecting survey results" refers to methods or equipment used to collect responses from customers or employees.

[1970] "Methods for cleaning and preprocessing data" refer to methods and techniques for preparing collected data for analysis by imputing or correcting missing or outlier values.

[1971] "Methods for passing data to a generative artificial intelligence to perform filtering" refer to methods or systems for inputting pre-processed data into a generative artificial intelligence and having it classify it into appropriate emotions or categories.

[1972] "Means for analyzing facial and voice data to recognize emotions in real time" refers to technologies and devices that use collected facial and voice data to analyze and recognize a user's emotional state in real time.

[1973] "Means for aggregating data and performing analysis based on it" refers to methods and systems that integrate and aggregate filtered data and real-time sentiment data to perform statistical and sentimental analysis.

[1974] "Means for generating reports based on analysis results and notifying administrators or relevant parties" refers to methods or devices for creating reports based on analysis results and communicating them to administrators or relevant parties.

[1975] This invention is an advanced system for efficiently collecting and analyzing feedback from customers or employees. This system operates using a combination of hardware and software and includes the following means:

[1976] hardware

[1977] 1. Smart devices

[1978] Smartphones and tablets are used for collecting survey responses. These devices have built-in cameras and microphones, allowing them to collect user facial expressions and audio data.

[1979] 2. Server

[1980] A server equipped with a database and analysis engine. This enables data preprocessing, filtering by generative artificial intelligence, real-time sentiment analysis, data aggregation and analysis, and report generation and notification.

[1981] software

[1982] 1. Emotional Engine

[1983] The system analyzes the user's emotions in real time using facial expression analysis libraries (e.g., "OpenFace," "Affectiva") and voice emotion analysis software (e.g., "IBM Watson Tone Analyzer").

[1984] 2. Generative Artificial Intelligence

[1985] Generative artificial intelligence such as BERT and GPT-3 (OpenAI) will be used to classify the sentiment of text data from survey responses.

[1986] 3. Database

[1987] We use MySQL or PostgreSQL to store and manage the collected and pre-processed data.

[1988] 4. Analytics

[1989] We will use Python and its libraries (e.g., pandas, numpy, matplotlib) to analyze data and generate reports.

[1990] 5. Notification System

[1991] Use email services (e.g., AWS SES) or messaging services (e.g., Twilio) to notify administrators and relevant parties of the report.

[1992] System operation

[1993] Data collection and cleaning

[1994] The terminal collects survey responses from customers and employees. In addition to the text data entered by the user, it also collects facial expressions and voice data using the camera and microphone. The collected data is sent to a server where missing values ​​are imputed and outliers are corrected.

[1995] Filtering by generative artificial intelligence

[1996] The pre-processed data is passed to a generative artificial intelligence (AI) for emotion classification. The AI ​​classifies the data into positive, negative, neutral, etc.

[1997] Real-time sentiment analysis

[1998] The collected facial and voice data is analyzed in real time by an emotion engine to recognize the user's emotional state.

[1999] Data aggregation and analysis

[2000] The filtering results and real-time sentiment data are integrated by the server for detailed analysis. For example, by statistically analyzing the satisfaction and dissatisfaction levels of each product, areas for improvement in the store can be identified.

[2001] Report generation and notification

[2002] Based on the analysis results, a report is generated that includes text summaries, graphs, charts, and other elements. This report is then sent to administrators and relevant parties through a notification system.

[2003] Specific example

[2004] Examples of prompt statements include the following:

[2005] Please generate an emotional analysis report for product "A". Please perform the analysis based on the following data set. We would like to know the ratio of positive, negative, and neutral responses.

[2006] In this way, this system enables detailed feedback analysis, including the actual emotions of customers, allowing managers and stakeholders to quickly identify areas for improvement.

[2007] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2008] Step 1:

[2009] Data collection

[2010] When collecting survey results entered by customers or employees, the device also uses its camera and microphone to simultaneously acquire facial expressions and voice data.

[2011] Input: Customer or employee survey text, facial expression data, audio data

[2012] Output: Collected questionnaire text, facial expression data, audio data

[2013] Step 2:

[2014] Data transmission

[2015] The device transmits the collected survey text, facial expression data, and audio data to the server.

[2016] Input: Collected questionnaire text, facial expression data, audio data

[2017] Output: Survey text, facial expression data, and audio data sent to the server.

[2018] Step 3:

[2019] Data cleaning and preprocessing

[2020] The server fills in missing values ​​and corrects outliers from the received data. For example, it calculates fill-in values ​​for questions with missing answers and corrects extremely abnormal values.

[2021] Input: Survey text, facial expression data, and audio data sent from the device.

[2022] Output: Cleaned and pre-processed questionnaire text, facial expression data, audio data

[2023] Step 4:

[2024] Filtering by generative artificial intelligence

[2025] The server inputs pre-processed data into a generative artificial intelligence (e.g., GPT-3) to perform sentiment classification. The generative AI classifies the data into positive, negative, or neutral and returns the result.

[2026] Input: Preprocessed survey text

[2027] Output: Emotionally classified survey data (positive, negative, neutral)

[2028] Step 5:

[2029] Real-time sentiment analysis

[2030] The server analyzes facial and voice data collected using facial expression analysis libraries and voice emotion analysis software in real time to recognize the user's emotional state.

[2031] Input: Preprocessed facial expression data and audio data

[2032] Output: Recognized real-time sentiment data

[2033] Step 6:

[2034] Data aggregation and analysis

[2035] The server integrates and aggregates the filtering results and real-time sentiment data, and performs statistical and emotional analysis. For example, it calculates the percentage of positive emotions, negative emotions, and neutral emotions.

[2036] Input: Sentiment-classified survey data, real-time sentiment data

[2037] Output: Aggregated and analyzed results (e.g., sentiment ratio, trend analysis, etc.)

[2038] Step 7:

[2039] Report generation

[2040] The server generates a report that includes text summaries, graphs, and charts based on the aggregated and analyzed results.

[2041] Input: Aggregation and analysis results

[2042] Output: Report (text summary, graphs, charts, etc.)

[2043] Step 8:

[2044] notification

[2045] The server uses email or messaging services to notify administrators or relevant parties of the reports it generates.

[2046] Input: Generated report

[2047] Output: Notification to administrators or relevant parties (email, message, etc.)

[2048] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2049] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2050] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2051] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2052] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2053] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2054] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2055] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2056] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2057] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2058] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2059] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2060] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2061] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2062] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2063] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2064] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2065] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2066] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2067] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2068] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[2069] The following is further disclosed regarding the embodiments described above.

[2070] (Claim 1)

[2071] Means for collecting survey results from customers or employees,

[2072] A means of cleaning and preprocessing the collected questionnaire data,

[2073] A method for passing pre-processed data to a generative artificial intelligence system to perform filtering,

[2074] A means for aggregating filtering results and performing analysis based on them,

[2075] A means of generating a report based on the analysis results and notifying administrators or relevant parties,

[2076] A system that includes this.

[2077] (Claim 2)

[2078] Generative artificial intelligence provides a means to classify the emotions expressed in survey responses,

[2079] The system according to claim 1, comprising means for transmitting classified emotion data.

[2080] (Claim 3)

[2081] The system according to claim 1, wherein the preprocessing means includes means for imputing missing values ​​in the questionnaire data.

[2082] "Example 1"

[2083] (Claim 1)

[2084] Means for collecting survey results from customers or employees,

[2085] A means of cleaning and preprocessing the collected questionnaire data,

[2086] A method for passing pre-processed data to a generative artificial intelligence system to perform filtering,

[2087] A means for aggregating the filtering results and analyzing the proportion of positive and negative emotions,

[2088] A means of generating a report including visual charts based on the analysis results and notifying administrators or relevant parties,

[2089] A system that includes this.

[2090] (Claim 2)

[2091] Generative artificial intelligence has a means of classifying the emotions in survey responses as "positive" or "negative,"

[2092] The system according to claim 1, comprising means for transmitting classified emotion data.

[2093] (Claim 3)

[2094] The system according to claim 1, wherein the preprocessing means includes means for imputing missing values ​​and correcting outliers in the survey data.

[2095] "Application Example 1"

[2096] (Claim 1)

[2097] Means for collecting survey results from customers or employees,

[2098] A means of cleaning and preprocessing the collected questionnaire data,

[2099] A method for passing pre-processed data to a generative artificial intelligence system to perform filtering,

[2100] A means for aggregating filtering results and performing analysis based on them,

[2101] A means of generating a report based on the analysis results and notifying administrators or relevant parties,

[2102] A means of sending the generated report to the administrator using electronic message,

[2103] A system that includes this.

[2104] (Claim 2)

[2105] Generative artificial intelligence provides a means to classify the emotions expressed in survey responses,

[2106] The system according to claim 1, comprising means for transmitting classified emotion data.

[2107] (Claim 3)

[2108] The system according to claim 1, wherein the preprocessing means includes means for imputing missing values ​​in the questionnaire data.

[2109] "Example 2 of combining an emotion engine"

[2110] (Claim 1)

[2111] Means for collecting survey results from customers or employees,

[2112] A means of cleaning and preprocessing the collected questionnaire data,

[2113] A method for passing pre-processed data to a generative artificial intelligence system to perform filtering,

[2114] A method for analyzing facial expressions and voice using an emotion engine that recognizes the emotional state of the user when they respond,

[2115] A means for aggregating filtering results and performing analysis based on them,

[2116] A means of generating a report based on the analysis results and notifying administrators or relevant parties,

[2117] A system that includes this.

[2118] (Claim 2)

[2119] The system according to claim 1, wherein the generative artificial intelligence includes means for classifying the emotions of survey responses and real-time emotion analysis means including facial expression and voice data obtained from an emotion engine.

[2120] (Claim 3)

[2121] The system according to claim 1, wherein the preprocessing means includes means for imputing missing values ​​in the survey data and means for correcting outliers using a data cleaning algorithm.

[2122] "Application example 2 when combining with an emotional engine"

[2123] (Claim 1)

[2124] Means for collecting survey results from customers or employees,

[2125] A means of cleaning and preprocessing the collected questionnaire data,

[2126] A method for passing pre-processed data to a generative artificial intelligence system to perform filtering,

[2127] A means for recognizing emotions in real time by analyzing facial and audio data collected during preprocessing,

[2128] A means for aggregating filtering results and real-time sentiment data and performing analysis based on them,

[2129] A means of generating a report based on the analysis results and notifying administrators or relevant parties,

[2130] A system that includes this.

[2131] (Claim 2)

[2132] Generative artificial intelligence provides a means to classify the emotions expressed in survey responses,

[2133] The system according to claim 1, comprising means for transmitting classified emotion data.

[2134] (Claim 3)

[2135] The system according to claim 1, wherein the preprocessing means includes means for imputing missing values ​​in the questionnaire data and means for correcting outliers. [Explanation of Symbols]

[2136] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting survey results from customers or employees, A means of cleaning and preprocessing the collected questionnaire data, A method for passing pre-processed data to a generative artificial intelligence system to perform filtering, A means for aggregating filtering results and performing analysis based on them, A means of generating a report based on the analysis results and notifying administrators or relevant parties, A system that includes this.

2. Generative artificial intelligence provides a means to classify the emotions expressed in survey responses, The system according to claim 1, comprising means for transmitting classified emotion data.

3. The system according to claim 1, wherein the preprocessing means includes means for imputing missing values ​​in the questionnaire data.

Images