System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting user data; means for organizing the collected user data into a consistent format; means for analyzing the organized data to extract user needs and challenges; means for generating an output that visualizes a user image and a user experience based on the analysis; and means for updating the output based on new data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Understanding users is extremely important in modern service and product development, but the process requires specialized skills and a significant amount of time and cost. This makes it difficult to fully understand users. Furthermore, the process of collecting, organizing, analyzing, and generating output from diverse user data is time-consuming, potentially delaying rapid improvements and the development cycle. [Means for solving the problem]

[0005] This invention provides a system that automatically collects user data, organizes it into a consistent format, and analyzes it using generative AI. Specifically, the system includes a means for collecting user data, a means for organizing the collected user data, a means for analyzing the organized data to extract user needs and issues, a means for generating output that visualizes user profiles and user experiences based on the analysis results, and a means for updating the output based on new data. This system enables consistent handling of user data, significantly streamlining the user understanding process, accelerating development cycles, and reducing costs.

[0006] "User Data" refers to information about users, including data based on user behavior and opinions, such as voice data, log data, feedback data, and social media data.

[0007] "Means of collection" refers to the technical methods and devices used to receive data from users, and have the ability to obtain the necessary information from various data sources.

[0008] "Organization tools" refers to the techniques and processes used to convert collected data into a consistent format, consolidate it, and make it easier to manage.

[0009] "Means of analysis" refers to technologies and algorithms used to analyze collected and organized data to extract user needs and issues, particularly those that use natural language processing and generative AI.

[0010] "User needs" refer to the demands and desires that users have for a service or product, and are an important factor in improving a service or adding new features.

[0011] A "user profile" refers to a virtual persona constructed based on specific attributes and behavioral patterns of a user, and is used to understand the target user.

[0012] "User experience" refers to the overall experience and emotions that users have when using a service or product, and is a factor that significantly affects user satisfaction.

[0013] "Output" refers to reports, visualized information, dashboards, etc. generated as a result of data analysis, and is a concrete expression of the results of user understanding.

[0014] "Updating methods" refers to the techniques and processes used to update existing outputs with the latest information based on newly collected data.

[0015] "Generative AI" refers to artificial intelligence technology that automatically learns from large amounts of data and analyzes and understands user needs and challenges. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system for promoting user understanding, and more specifically, to a system for consistently collecting, organizing, and analyzing user data, and generating and updating output to streamline the process of understanding users. The following describes in detail an embodiment of this system.

[0038] Program processing overview

[0039] Data collection

[0040] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[0041] Examples:

[0042] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[0043] Data organization

[0044] The server organizes the received user data into a consistent format, for example by converting audio data into text data and tagging the organized data.

[0045] Examples:

[0046] The server converts the audio data from user interviews into text and adds tags such as "dissatisfaction points," "requests," and "frequently used words."

[0047] Data analysis

[0048] The server analyzes the organized data using generative AI to extract user needs and challenges, particularly by using natural language processing technology to identify positive and negative feedback.

[0049] Examples:

[0050] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems.

[0051] Output Generation

[0052] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results.

[0053] Examples:

[0054] The generated report shows the different needs and challenges of each user segment, and uses graphs and charts to visually display trends in user behavior and feedback.

[0055] update

[0056] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[0057] Examples:

[0058] The server will take in new feedback data after the update and reflect changes in user satisfaction in the report.

[0059] This system allows for efficient collection, organization, and analysis of user data, and the creation of specific action plans based on the results, enabling the development team to accurately understand user needs and make rapid and effective product improvements.

[0060] The processing flow will be explained below.

[0061] Step 1: Data collection

[0062] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[0063] (Specific actions)

[0064] The server collects data in real time through designated API endpoints.

[0065] The server checks the format of the received data and filters out any invalid or missing data.

[0066] Step 2: Data organization

[0067] The server organizes the received user data into a consistent format.

[0068] (Specific actions)

[0069] The server uses voice recognition technology to convert the audio data of the user interview into text.

[0070] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[0071] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[0072] Step 3: Data analysis

[0073] The server analyzes the organized data using generative AI.

[0074] (Specific actions)

[0075] The server performs sentiment analysis to classify user feedback as positive, negative, or neutral.

[0076] The server clusters the user data and identifies different user segments.

[0077] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0078] Step 4: Generate output

[0079] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0080] (Specific actions)

[0081] The server generates reports that show the different needs and challenges of each user segment.

[0082] The server creates a dashboard based on user data, displaying visual trend graphs and charts that are updated in real time.

[0083] The server provides these outputs in PDF format and as a web interface.

[0084] Step 5: Update

[0085] The server updates the generated output based on the new data.

[0086] (Specific actions)

[0087] The server analyzes the newly collected data and incorporates it into existing reports and dashboards.

[0088] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[0089] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0090] Through these steps, the present invention efficiently and automatically collects, organizes, and analyzes user data, significantly streamlining the process of understanding users, enabling development teams to accurately grasp user needs and make rapid and effective product improvements.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] In today's world, better understanding users is essential for improving products and services. However, user data exists in a variety of formats, making it complex and time-consuming to collect, organize, analyze, and then generate output from that data and update it based on the latest information. Therefore, there is a need for a method to process user data efficiently and in a consistent format to obtain fast and accurate feedback.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables efficient processing and updating of complex user data, and promotes rapid and accurate user understanding.

[0096] "User Data" refers to all information related to users, including voice data, log data, feedback data, and SNS data.

[0097] "Means of collection" refers to the methods and technologies required to store user data on a server.

[0098] "Organization means" refers to the methods and techniques used to convert collected user data into a consistent format and to tag and categorize the data.

[0099] A "generative AI model" refers to an artificial intelligence model that analyzes and generates information based on given data, such as one that uses natural language processing technology.

[0100] "Means of analysis" refers to methods and techniques for extracting user needs and issues using organized data.

[0101] "Means for generating output" refers to the process for visualizing the user profile and user experience based on the analysis results.

[0102] "Updating means" refers to the methods and techniques used to bring existing outputs up to date based on new data.

[0103] "Audio data" refers to audio recordings of a user's speech, which may be converted into text data for analysis.

[0104] "Log data" refers to detailed historical information about users' application operations and system usage.

[0105] "Feedback Data" refers to information including user ratings, opinions, comments, etc.

[0106] "SNS Data" refers to information, including comments and reactions, posted by users on social networking services.

[0107] "Tagging" refers to the act of adding specific labels or metadata to data, making it easier to search and classify the data.

[0108] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To implement this system, the following hardware and software are used.

[0109] Hardware and Software

[0110] The server is the central device for collecting, organizing, analyzing, generating output, and updating user data. It must be equipped with a high-performance processor, large storage capacity, and have a stable network connection.

[0111] A device is a device used to collect user data, such as a smartphone, tablet, or PC. The device is equipped with a microphone for recording audio data from user interviews and software for recording application usage logs.

[0112] A user is an end user who operates a terminal and interacts with the system, such as participating in an interview, answering a survey, or using an application.

[0113] Specific example of system operation

[0114] 1. Data Collection:

[0115] When a user operates the app, click logs and form input data are generated. This data is sent from the device to the server in real time. In addition, when a user interview is conducted, audio data is recorded through the device's microphone and sent to the server.

[0116] 2. Data reduction:

[0117] The server organizes the collected user data into a consistent format. For example, voice data is converted into text data using the Google Speech-to-Text API. The data is tagged with tags such as "dissatisfaction points," "requests," and "frequently used words."

[0118] 3. Data Analysis:

[0119] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). It uses natural language processing technology to extract user needs and issues. For example, feedback such as "the app's response is slow" or "the new features are difficult to use" is identified and extracted as common issues.

[0120] 4. Output generation:

[0121] The server generates outputs (e.g., reports and dashboards) based on the analysis results that visualize user profiles and user experiences. These outputs are presented in an easy-to-understand format, highlighting the different needs and challenges of each user segment.

[0122] 5. Update:

[0123] As new user data is added, the server automatically updates existing outputs, ensuring a consistent, up-to-date understanding of users. For example, it incorporates the latest feedback data and reflects changes in user satisfaction in its reports.

[0124] Prompt Sentence Examples

[0125] "Convert the audio data to text."

[0126] "Identify user needs and common problems."

[0127] This system allows the development team to quickly and accurately understand user needs and use them to improve the product, by consistently and efficiently collecting, organizing, analyzing, generating output, and updating data on user behavior and feedback.

[0128] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0129] Step 1: Data collection

[0130] The server begins collecting user data. As input, it receives click logs, form input data, audio data from user interviews, and social media posting data sent from the device. This data is stored in a database on the server. Specifically, the device records the user's actions and sends the data to the server in real time. For example, when a user clicks a button in an app, that information is transferred to the server within seconds and stored in the database.

[0131] Step 2: Data organization

[0132] The server organizes the collected user data into a consistent format. Voice data is converted into text data using the Google Speech-to-Text API, and log data and feedback data are also converted into a consistent format. This allows data of different formats to be stored in a unified format. Specifically, the server uses voice recognition technology to convert the interview audio into text. This converted data is organized with tags such as "dissatisfaction points," "requests," and "frequently used words."

[0133] Step 3: Data analysis

[0134] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). The organized text data is provided as input to the generative AI model, which then automatically extracts user needs and challenges. Specifically, the server analyzes user feedback such as "the app's response is slow" or "the new features are difficult to use" and identifies common issues. The output is a list of common user needs and challenges.

[0135] Step 4: Generate output

[0136] The server generates output (reports, dashboards, etc.) that visualizes user profiles and user experiences based on the results of data analysis. The input uses user needs and issues derived from data analysis. Specifically, the server uses data visualization technology to generate graphs and charts to visually display trends in user data and analytical results. As output, reports and dashboards are provided to end users in an easy-to-understand format.

[0137] Step 5: Update

[0138] The server automatically updates the output based on newly collected user data. The newly added data is included as input. Specifically, the server re-analyzes the latest voice and log data and reflects it in reports and dashboards. The output always provides reports and output based on the latest user data.

[0139] Through these steps, the system can efficiently process user data and promote fast and accurate user understanding.

[0140] (Application example 1)

[0141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] Current online shopping sites lack systems that can effectively collect and analyze users' diverse needs and feedback and recommend products suited to each individual user in real time. This results in a poor user experience and a decline in purchasing intent. Furthermore, delays in product improvements based on collected feedback can potentially undermine a company's competitiveness.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0144] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, means for updating the output based on new data, means for collecting user operation logs, purchase histories, and posts on social networking services, and means for analyzing the collected data based on prompts and recommending products that meet user needs. This makes it possible to analyze user behavior and feedback in real time and propose optimal products, thereby improving user satisfaction and strengthening corporate competitiveness.

[0145] definition statement

[0146] "User data" refers to data collected in a variety of forms, such as user operation logs, purchase history, posts on social networking services, voice data, log data, and feedback data.

[0147] A "consistent format" refers to a format in which collected user data is organized in a unified format to facilitate subsequent analysis.

[0148] A "generative AI model" is a model that uses machine learning technology to process data and extract user needs and issues.

[0149] "User needs" refers to what users need or want in a particular situation.

[0150] A "user profile" is a collection of user characteristics and behavioral patterns derived from collected and analyzed data.

[0151] "Output" refers to output such as reports and dashboards generated by the server to visualize the user profile and user experience.

[0152] A "prompt sentence" is a text input given to a generative AI model to enable it to analyze and generate.

[0153] "Recommendation" means proposing products or services that meet the user's needs based on the results of analysis.

[0154] An "operation log" is recorded data generated when a user operates an application or website.

[0155] "Purchase history" is a record of past purchases made by a user.

[0156] "Social networking service posts" refers to tweets, comments, images, and videos posted by users on social networking sites.

[0157] MODE FOR CARRYING OUT THE INVENTION

[0158] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To realize this system, the following configuration and processing are performed.

[0159] System configuration

[0160] The system includes the following main features:

[0161] User Data Collection

[0162] Data organization

[0163] Data analysis

[0164] Output Generation

[0165] update

[0166] 1. Collection of User Data

[0167] The server collects user data such as user operation logs, purchase history, and posts on social networking services, as well as click logs and form input data sent from user devices (smartphones, computers, etc.).

[0168] For example, click logs and purchase histories generated when a user uses an online shopping site are sent to the server in real time. This data is collected via API and used for subsequent processing.

[0169] 2. Data organization

[0170] The server then organizes the collected user data into a consistent format. For example, voice data is converted to text using a speech recognition API, and social media data is analyzed and tagged appropriately.

[0171] As a specific example, when reviews posted by users on social networking services are collected as audio data, they are converted into text data and given tags such as "positive" or "negative."

[0172] 3. Data analysis

[0173] The server analyzes the organized data using a generative AI model, such as OpenAI's GPT-3, to extract user needs and issues.

[0174] As a specific example, the subject of data analysis is a social media post stating, "The user is interested in new outfits." In this case, the prompt would be, "Analyze the following user data and identify the user's needs and challenges. The user has posted on social media that they are interested in new outfits."

[0175] 4. Output Generation

[0176] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results, making it easy to understand user needs and behavioral patterns.

[0177] For example, the generated report may include product recommendations based on user needs, visually displayed using graphs and charts.

[0178] 5. Updates

[0179] The server automatically updates the generated output based on new data, ensuring that the latest user information is always reflected.

[0180] For example, as new feedback data is acquired, changes in user satisfaction are reflected in the report.

[0181] In this way, the system of the present invention can efficiently collect, organize, and analyze user data and generate specific action plans based on the results, enabling development teams to accurately understand user needs and make rapid and effective product improvements.

[0182] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0183] Program processing flow

[0184] Step 1:

[0185] The device collects user data such as user operation logs, purchase history, and posts on social networking services. This also includes click logs and form input data generated when users use online shopping sites. The device sends the collected data to a server in real time. The input is various user data, and the output is data sent to the server.

[0186] Step 2:

[0187] The server organizes the collected user data into a consistent format. For example, voice data is converted into text using a speech recognition API (e.g., Google Cloud Speech-to-Text), and social media data is analyzed using natural language processing technology and appropriately tagged. The input is the collected user data, and the output is organized text data and tagged data.

[0188] Step 3:

[0189] The server analyzes the organized data using a generative AI model. During this process, the organized text data is input as prompts into the generative AI model (e.g., OpenAI GPT-3) to extract user needs and issues. The inputs are the organized data and prompts, and the output is the analysis results.

[0190] Step 4:

[0191] The server generates output (e.g., reports, dashboards) that visualize the user profile and user experience based on the analysis results. This output visually shows user needs and problems. The input is the analysis results, and the output is the visualized output.

[0192] Step 5:

[0193] The server updates the generated output based on the new data, ensuring that it always reflects the latest user information. For example, when the latest user feedback data is obtained, a report or dashboard is updated based on it. The input is the new user data, and the output is the updated output.

[0194] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0195] The present invention relates to a user understanding promotion system that combines an emotion engine, which streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Furthermore, it utilizes an emotion engine that recognizes user emotions to provide detailed analysis including user emotion information.

[0196] Program processing overview

[0197] Data collection

[0198] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[0199] Examples:

[0200] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[0201] Data organization

[0202] The server organizes the received user data into a consistent format.

[0203] Specific behavior:

[0204] The server uses voice recognition technology to convert the audio data of the user interview into text.

[0205] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[0206] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[0207] Data analysis

[0208] The server analyzes the organized data using generative AI and an emotion engine.

[0209] Specific behavior:

[0210] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[0211] The server clusters the user data and identifies different user segments.

[0212] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0213] Examples:

[0214] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems. It then uses an emotion engine to identify user sentiment regarding these problems and calculates the percentage of negative feedback.

[0215] Output Generation

[0216] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0217] Specific behavior:

[0218] The server generates reports that show the different needs and challenges of each user segment.

[0219] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[0220] The server provides these outputs in PDF format and as a web interface.

[0221] Examples:

[0222] The generated report shows the different needs and challenges of each user segment, highlighting items with particularly negative feedback, and includes sentiment information for each piece of feedback, along with visually displaying trends in user behavior and feedback content using graphs and charts.

[0223] update

[0224] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[0225] Specific behavior:

[0226] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[0227] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[0228] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0229] Examples:

[0230] When feedback data is updated, the server not only reflects changes in user satisfaction in the report, but also highlights areas where sentiment fluctuations are particularly noticeable.

[0231] In this way, by combining an emotion engine, the present invention provides detailed user understanding, including user emotional information, enabling development teams to make product improvements quickly and accurately.

[0232] The processing flow will be explained below.

[0233] Step 1: Data collection

[0234] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[0235] (Specific actions)

[0236] The server collects various data in real time through API endpoints.

[0237] The server checks the format of the received data and filters out any invalid or missing data.

[0238] Step 2: Data organization

[0239] The server organizes the received user data into a consistent format.

[0240] (Specific actions)

[0241] The server uses voice recognition technology to convert the audio data of the user interview into text.

[0242] The server converts the log data and feedback data into a standard format (JSON, CSV, etc.) and stores it in a database.

[0243] The server uses natural language processing (NLP) technology on the text data to extract and tag important keywords and themes.

[0244] Step 3: Emotion Recognition

[0245] The server analyzes the organized data using an emotion engine to identify the user's emotional state.

[0246] (Specific actions)

[0247] The server inputs text and voice data into an emotion engine and classifies the user's emotions as positive, negative, or neutral.

[0248] The server stores the emotion recognition results in a database, making them available for the next analysis step.

[0249] Step 4: Data analysis

[0250] The server uses generative AI to comprehensively analyze the organized data, including the results of the emotion engine, to extract user needs and issues.

[0251] (Specific actions)

[0252] The server analyzes the trends in user feedback based on the emotional state output by the emotion engine.

[0253] The server clusters user data to identify different segments and extracts common needs and challenges for each segment.

[0254] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0255] Examples:

[0256] The server uses an emotion engine to identify that there is a lot of feedback about the app's slow response, and identifies that there is a particularly large amount of negative emotion among the feedback.

[0257] Step 5: Generate output

[0258] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0259] (Specific actions)

[0260] The server generates reports that show the different needs and challenges of each user segment.

[0261] The server creates a dashboard that reflects sentiment information and data trends, displaying graphs and charts that are updated in real time.

[0262] The server provides the generated reports and dashboards in PDF format or as a web interface.

[0263] Examples:

[0264] The generated report shows the different needs and challenges of each user segment, highlights items with a high percentage of negative feedback, and includes a graph showing the emotional state of users based on the results of the emotion engine.

[0265] Step 6: Update

[0266] The server automatically updates the output based on the new data.

[0267] (Specific actions)

[0268] The server performs additional analysis on the newly collected data and re-evaluates the results of the emotion engine.

[0269] The server retrains the generative AI to continuously improve its analytical accuracy.

[0270] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0271] Examples:

[0272] The server analyzes the new feedback data and reports fluctuations in user satisfaction, as well as highlighting areas where sentiment fluctuations are particularly pronounced.

[0273] In this way, the present invention, by combining an emotion engine, achieves a deeper understanding of users, enabling development teams to make quick and accurate product improvements.

[0274] Example 2

[0275] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0276] In many modern systems, the collection, organization, and analysis of user data are carried out separately, resulting in a lack of a consistent process. This makes it difficult to accurately understand users, and in particular, detailed analysis including emotional information is not performed. Furthermore, the inability to update collected data in real time or automatically update output hinders continuous improvement of the user experience.

[0277] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model and an emotion recognition engine, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables consistent collection, organization, analysis, and real-time updating of user data. In addition, the use of the emotion recognition engine enables detailed analysis including user emotion information, enabling continuous improvement of the user experience.

[0278] "User data" refers to all information related to users, such as user behavior, feedback, usage logs, survey responses, and social media posts.

[0279] "Means of collection" refers to the processes and technologies used to receive user data from devices and store it on servers.

[0280] "Organization methods" refers to the processes and techniques used to convert collected user data into a consistent format and to cleanse, format, and store the data.

[0281] A "generative AI model" refers to an algorithm or system that uses machine learning or deep learning techniques to generate, analyze, and predict data.

[0282] An "emotion recognition engine" refers to a system that uses natural language processing and machine learning techniques to identify emotions (positive, negative, neutral, etc.) from user feedback and data.

[0283] "Means of analysis" refers to the processes and techniques used to analyze collected and organized user data using various analytical tools and techniques to extract insights and patterns.

[0284] "Means of generating output" refers to the processes and techniques used to create visual reports and dashboards based on the analysis results, and visualize user profiles and user experiences.

[0285] "Updating methods" refers to the processes and techniques used to reanalyze existing outputs based on newly collected data, ensuring that they always reflect the latest information.

[0286] The present invention relates to a user understanding promotion system that combines an emotion engine, and streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Specific embodiments of the system are described below.

[0287] Data collection

[0288] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[0289] Hardware / software used: Internet connection, database, cloud storage

[0290] Examples:

[0291] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[0292] Data organization

[0293] The server organizes the received user data into a consistent format.

[0294] Hardware / software used: Data cleansing tools, data format conversion tools

[0295] Examples:

[0296] The server converts the user interview audio data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text API, IBM Watson Speech to Text).

[0297] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[0298] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[0299] Data analysis

[0300] The server analyzes the organized data using generative AI models and emotion engines.

[0301] Hardware / software used: Machine learning platforms (e.g., TensorFlow, PyTorch), sentiment analysis tools (e.g., IBM Watson Natural Language Understanding)

[0302] Examples:

[0303] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[0304] The server clusters the user data and identifies different user segments (e.g., K-means clustering).

[0305] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0306] Example prompt: Identify user feedback such as "The app is slow to respond" or "The new features are difficult to use" and extract them as common problems. Use an emotion engine to identify user sentiment regarding these problems and calculate the percentage of negative feedback.

[0307] Output Generation

[0308] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0309] Hardware / software used: Report generation tools (e.g., Tableau, Microsoft Power BI), interactive dashboard tools (e.g., D3.js, Chart.js)

[0310] Examples:

[0311] The server generates reports that show the different needs and challenges of each user segment.

[0312] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[0313] The server provides these outputs in PDF format and via a web interface. The generated reports show the different needs and challenges of each user segment, highlighting areas that generate the most negative feedback.

[0314] update

[0315] The server updates its output based on new data, always providing the latest user profile and experience.

[0316] Hardware / software used: Cron jobs, scheduling tools, retraining platform

[0317] Examples:

[0318] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[0319] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[0320] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0321] In this way, by combining emotion engines, the present invention provides detailed user understanding including user emotion information, enabling rapid and accurate product improvements.

[0322] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0323] Step 1:

[0324] Data collection

[0325] Input: User data such as user utterances, app usage logs, survey responses, and social media posts

[0326] Output: raw data collected

[0327] Specific behavior:

[0328] 1. The user uses the app to perform various operations.

[0329] 2. The terminal records the user's operation log and input data in real time and sends them to the server.

[0330] 3. The server receives the interview audio data, survey results, and social media posts and stores them in a buffer.

[0331] Step 2:

[0332] Data organization

[0333] Input: Raw data collected

[0334] Output: A clean, uniformly formatted dataset

[0335] Specific behavior:

[0336] 1. The server converts the voice data into text data using speech recognition technology (e.g., Google Cloud Speech-to-Text API).

[0337] 2. The server uses a data cleansing tool to remove duplicates and errors from the raw data and organize the necessary information.

[0338] 3. The server converts the log data and feedback data into a unified format (JSON, CSV, etc.) and stores it in the database.

[0339] 4. The server uses natural language processing (NLP) technology to extract important keywords from the text data and tag them.

[0340] Step 3:

[0341] Data analysis

[0342] Input: A clean, uniformly formatted dataset

[0343] Output: Analysis results and insights

[0344] Specific behavior:

[0345] 1. The server analyzes the emotional state (positive, negative, neutral) of the text data using an emotion recognition engine (e.g., IBM Watson Natural Language Understanding).

[0346] 2. The server uses a generative AI model to analyze patterns in the data and segment users.

[0347] 3. The server performs trend analysis to identify major issues and user requests within a specific period of time.

[0348] 4. Example: Extract common issues from feedback such as "The app's response is slow" or "The new features are difficult to use," and use sentiment analysis to calculate the percentage of negative feedback.

[0349] Step 4:

[0350] Output Generation

[0351] Input: Analysis results and insights

[0352] Output: Visualized reports and dashboards

[0353] Specific behavior:

[0354] 1. The server generates a report based on the analysis results that shows the needs and challenges of each user segment (e.g., Tableau, Microsoft Power BI).

[0355] 2. The server creates a dashboard with trend graphs and charts that update in real time based on NLP and sentiment information (e.g. D3.js, Chart.js).

[0356] 3. The server provides the generated output in PDF format and / or via a web interface.

[0357] 4. Example: The generated report will highlight the different needs of different user segments and highlight areas with particularly negative feedback.

[0358] Step 5:

[0359] Output Updates

[0360] Input: Freshly collected raw data

[0361] Output: Updated reports and dashboards

[0362] Specific behavior:

[0363] 1. The server analyzes the newly collected data and applies it to existing outputs.

[0364] 2. The server retrains the generative AI model to continuously improve the accuracy of the analysis (e.g., TensorFlow, PyTorch).

[0365] 3. If the server detects any important changes or abnormalities, it will automatically notify relevant parties via alerts.

[0366] 4. Example: When new user feedback is collected, the server analyzes it using an emotion recognition engine and updates existing reports and dashboards accordingly.

[0367] (Application example 2)

[0368] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0369] There is a need for a means to efficiently collect, organize, and analyze diverse user behavior and feedback data in order to deepen user understanding in real time. However, to achieve this, technology is needed that can properly identify user emotions and provide fast, highly accurate analysis results. Furthermore, there is a need to improve the user experience by providing analysis results in a visually easy-to-understand format. A system that can solve these issues in an integrated manner is needed.

[0370] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0371] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data to extract user needs and issues, means for identifying the emotional state of user feedback using a sentiment analysis engine, means for generating an output that visualizes user profiles and user experiences based on the analysis results, and means for updating and visualizing the output in real time based on new data, thereby enabling rapid and accurate analysis of various user data and providing detailed analysis results, including user sentiment, in real time.

[0372] "User Data" refers to information generated when a user uses an application or system, including voice data, log data, feedback data, and SNS data.

[0373] "Means of collection" refers to the hardware and software functions required to efficiently obtain a variety of user data.

[0374] "Organization" refers to the ability to convert collected user data into a consistent format and make it analyzable.

[0375] "Means of analyzing and extracting user needs and issues" refers to algorithms and engines that use organized user data to determine the problems and requests users are facing.

[0376] "Means for generating output" refers to a function for visually expressing the user profile and user experience based on the analysis results.

[0377] "Means of updating" refers to the ability to automatically update existing outputs based on new data, ensuring that information is always up-to-date.

[0378] "Sentiment analysis engine" refers to technology used to identify a user's emotional state (e.g., positive, negative, neutral) from user feedback and other data.

[0379] "Visualization means" refers to the ability to display analytical results in a visual format such as graphs or charts.

[0380] As an embodiment of the present invention, a customer sentiment analysis system for a physical store is configured as follows.

[0381] Data collection

[0382] The server collects the following user data from users (customers) using their smartphones. Voice data is collected using the smartphone's microphone to gather customer feedback. Log data includes smartphone operation history, feedback data includes in-app ratings and comments, and social media data includes mentions posted by customers on social media.

[0383] Data organization

[0384] The server organizes the collected user data into a consistent format. It converts the voice data into text using speech recognition technology (e.g., the speech_recognition library). It converts the generated log and feedback data into JSON or CSV format and stores it in a database. It uses natural language processing (NLP) technology (e.g., the TextBlob library) to extract and tag important keywords and themes from the text data.

[0385] Data analysis

[0386] The server analyzes the organized data using generative AI and a sentiment engine, which identifies the emotional state of user feedback (positive, negative, neutral, etc.), clusters user data to identify different user segments, and performs trend analysis to extract key issues and requests within a specific period.

[0387] Output Generation

[0388] The server generates outputs based on the analysis results that visualize user profiles and user experiences. It generates reports showing the different needs and challenges of each user segment, creates dashboards based on user data and sentiment information, and displays visual trend graphs and charts that are updated in real time. These outputs are provided in PDF format or as a web interface.

[0389] update

[0390] The server automatically updates the output based on new data, always providing the latest user profile and user experience. It analyzes newly collected data, applies it to existing reports and dashboards using an emotion engine, and retrains the generative AI to continuously improve the accuracy of the analysis. If there are important changes or abnormalities, it automatically notifies relevant parties with alerts.

[0391] Specific examples

[0392] For example, if a customer gives voice feedback at a store, such as "The cash register is too slow," this is collected using the smartphone's microphone, converted into text data, and then judged as "negative" using the emotion engine. Social media data and other feedback data are also analyzed in real time and reflected in reports. In this way, items with a high rate of negative feedback can be identified and specific improvement measures can be implemented.

[0393] Prompt Sentence Examples

[0394] "Based on user behavioral data and feedback, use a sentiment analysis engine and generative AI model to answer the following questions:

[0395] Summarize the customer's feedback.

[0396] Determine your feelings about each piece of feedback (positive, negative, neutral).

[0397] List the most frequently cited issues over a specific period of time.

[0398] Analyze how your customers feel about each issue.

[0399] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0400] Step 1: Data collection

[0401] The server collects user data from users (customers) through their devices (smartphones). Voice feedback provided by customers is recorded using the smartphone's microphone, and other feedback data and social media data are also collected. These data are sent to the server and stored.

[0402] Input: Voice data, feedback data, social media data

[0403] Output: Raw data stored on the server

[0404] Step 2: Data organization

[0405] The server organizes the collected user data into a consistent format: voice data is converted into text using speech recognition technology, and log and feedback data is converted into JSON or CSV format.

[0406] Input: Raw data (audio data, feedback data, SNS data)

[0407] Output: Text data, JSON format data, CSV format data

[0408] Step 3: Natural Language Processing (NLP)

[0409] The server then performs natural language processing (NLP) on the organized data to extract and tag important keywords and themes, for example by using the TextBlob library to parse the text data.

[0410] Input: Organized data (text, JSON, CSV)

[0411] Output: Text data with keywords

[0412] Step 4: Sentiment analysis

[0413] The server runs the keyworded text data through a sentiment analysis engine to identify the emotional state of the user feedback (positive, negative, neutral, etc.). For example, the TextBlob library performs sentiment analysis.

[0414] Input: Text data with keywords

[0415] Output: Text data with emotion information

[0416] Step 5: Data Clustering

[0417] The server clusters user data based on the text data with emotional information, identifies different user segments, and uses a clustering algorithm to classify users by their attributes.

[0418] Input: Text data with emotion information

[0419] Output: Clustered user data

[0420] Step 6: Trend analysis

[0421] The server performs trend analysis based on the clustered user data, extracts the main issues and requests within a specific period, and identifies user trends and requests based on the analysis results.

[0422] Input: Clustered user data

[0423] Output: Trend analysis results

[0424] Step 7: Generate Output

[0425] The server generates output based on the trend analysis results, visualizing user profiles and user experiences. It creates visual reports (graphs and charts) that show the different needs and challenges of each user segment.

[0426] Input: Trend analysis results

[0427] Output: Visualized reports, dashboards

[0428] Step 8: Update the output

[0429] The server automatically updates the output based on new data, always providing the latest user profile and experience. Newly collected data is analyzed and the generative AI is retrained.

[0430] Input: New user data, existing analysis results

[0431] Output: Modernized reports, dashboards

[0432] The above steps enable the efficient collection, organization, analysis, output generation, and updating of diverse user data, making it possible to understand users in real time and provide improvement measures.

[0433] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0434] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0435] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0436] [Second embodiment]

[0437] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0438] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0439] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0440] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0441] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0442] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0443] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0444] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0445] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0446] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0447] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0448] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0449] The present invention relates to a system for promoting user understanding, and more specifically, to a system for consistently collecting, organizing, and analyzing user data, and generating and updating output to streamline the process of understanding users. The following describes in detail an embodiment of this system.

[0450] Program processing overview

[0451] Data collection

[0452] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[0453] Examples:

[0454] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[0455] Data organization

[0456] The server organizes the received user data into a consistent format, for example by converting audio data into text data and tagging the organized data.

[0457] Examples:

[0458] The server converts the audio data from user interviews into text and adds tags such as "dissatisfaction points," "requests," and "frequently used words."

[0459] Data analysis

[0460] The server analyzes the organized data using generative AI to extract user needs and challenges, particularly by using natural language processing technology to identify positive and negative feedback.

[0461] Examples:

[0462] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems.

[0463] Output Generation

[0464] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results.

[0465] Examples:

[0466] The generated report shows the different needs and challenges of each user segment, and uses graphs and charts to visually display trends in user behavior and feedback.

[0467] update

[0468] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[0469] Examples:

[0470] The server will take in new feedback data after the update and reflect changes in user satisfaction in the report.

[0471] This system allows for efficient collection, organization, and analysis of user data, and the creation of specific action plans based on the results, enabling the development team to accurately understand user needs and make rapid and effective product improvements.

[0472] The processing flow will be explained below.

[0473] Step 1: Data collection

[0474] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[0475] (Specific actions)

[0476] The server collects data in real time through designated API endpoints.

[0477] The server checks the format of the received data and filters out any invalid or missing data.

[0478] Step 2: Data organization

[0479] The server organizes the received user data into a consistent format.

[0480] (Specific actions)

[0481] The server uses voice recognition technology to convert the audio data of the user interview into text.

[0482] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[0483] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[0484] Step 3: Data analysis

[0485] The server analyzes the organized data using generative AI.

[0486] (Specific actions)

[0487] The server performs sentiment analysis to classify user feedback as positive, negative, or neutral.

[0488] The server clusters the user data and identifies different user segments.

[0489] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0490] Step 4: Generate output

[0491] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0492] (Specific actions)

[0493] The server generates reports that show the different needs and challenges of each user segment.

[0494] The server creates a dashboard based on user data, displaying visual trend graphs and charts that are updated in real time.

[0495] The server provides these outputs in PDF format and as a web interface.

[0496] Step 5: Update

[0497] The server updates the generated output based on the new data.

[0498] (Specific actions)

[0499] The server analyzes the newly collected data and incorporates it into existing reports and dashboards.

[0500] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[0501] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0502] Through these steps, the present invention efficiently and automatically collects, organizes, and analyzes user data, significantly streamlining the process of understanding users, enabling development teams to accurately grasp user needs and make rapid and effective product improvements.

[0503] Example 1

[0504] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0505] In today's world, better understanding users is essential for improving products and services. However, user data exists in a variety of formats, making it complex and time-consuming to collect, organize, analyze, and then generate output from that data and update it based on the latest information. Therefore, there is a need for a method to process user data efficiently and in a consistent format to obtain fast and accurate feedback.

[0506] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0507] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables efficient processing and updating of complex user data, and promotes rapid and accurate user understanding.

[0508] "User Data" refers to all information related to users, including voice data, log data, feedback data, and SNS data.

[0509] "Means of collection" refers to the methods and technologies required to store user data on a server.

[0510] "Organization means" refers to the methods and techniques used to convert collected user data into a consistent format and to tag and categorize the data.

[0511] A "generative AI model" refers to an artificial intelligence model that analyzes and generates information based on given data, such as one that uses natural language processing technology.

[0512] "Means of analysis" refers to methods and techniques for extracting user needs and issues using organized data.

[0513] "Means for generating output" refers to the process for visualizing the user profile and user experience based on the analysis results.

[0514] "Updating means" refers to the methods and techniques used to bring existing outputs up to date based on new data.

[0515] "Audio data" refers to audio recordings of a user's speech, which may be converted into text data for analysis.

[0516] "Log data" refers to detailed historical information about users' application operations and system usage.

[0517] "Feedback Data" refers to information including user ratings, opinions, comments, etc.

[0518] "SNS Data" refers to information, including comments and reactions, posted by users on social networking services.

[0519] "Tagging" refers to the act of adding specific labels or metadata to data, making it easier to search and classify the data.

[0520] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To implement this system, the following hardware and software are used.

[0521] Hardware and Software

[0522] The server is the central device for collecting, organizing, analyzing, generating output, and updating user data. It must be equipped with a high-performance processor, large storage capacity, and have a stable network connection.

[0523] A device is a device used to collect user data, such as a smartphone, tablet, or PC. The device is equipped with a microphone for recording audio data from user interviews and software for recording application usage logs.

[0524] A user is an end user who operates a terminal and interacts with the system, such as participating in an interview, answering a survey, or using an application.

[0525] Specific example of system operation

[0526] 1. Data Collection:

[0527] When a user operates the app, click logs and form input data are generated. This data is sent from the device to the server in real time. In addition, when a user interview is conducted, audio data is recorded through the device's microphone and sent to the server.

[0528] 2. Data reduction:

[0529] The server organizes the collected user data into a consistent format. For example, voice data is converted into text data using the Google Speech-to-Text API. The data is tagged with tags such as "dissatisfaction points," "requests," and "frequently used words."

[0530] 3. Data Analysis:

[0531] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). It uses natural language processing technology to extract user needs and issues. For example, feedback such as "the app's response is slow" or "the new features are difficult to use" is identified and extracted as common issues.

[0532] 4. Output generation:

[0533] The server generates outputs (e.g., reports and dashboards) based on the analysis results that visualize user profiles and user experiences. These outputs are presented in an easy-to-understand format, highlighting the different needs and challenges of each user segment.

[0534] 5. Update:

[0535] As new user data is added, the server automatically updates existing outputs, ensuring a consistent, up-to-date understanding of users. For example, it incorporates the latest feedback data and reflects changes in user satisfaction in its reports.

[0536] Prompt Sentence Examples

[0537] "Convert the audio data to text."

[0538] "Identify user needs and common problems."

[0539] This system allows the development team to quickly and accurately understand user needs and use them to improve the product, by consistently and efficiently collecting, organizing, analyzing, generating output, and updating data on user behavior and feedback.

[0540] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0541] Step 1: Data collection

[0542] The server begins collecting user data. As input, it receives click logs, form input data, audio data from user interviews, and social media posting data sent from the device. This data is stored in a database on the server. Specifically, the device records the user's actions and sends the data to the server in real time. For example, when a user clicks a button in an app, that information is transferred to the server within seconds and stored in the database.

[0543] Step 2: Data organization

[0544] The server organizes the collected user data into a consistent format. Voice data is converted into text data using the Google Speech-to-Text API, and log data and feedback data are also converted into a consistent format. This allows data of different formats to be stored in a unified format. Specifically, the server uses voice recognition technology to convert the interview audio into text. This converted data is organized with tags such as "dissatisfaction points," "requests," and "frequently used words."

[0545] Step 3: Data analysis

[0546] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). The organized text data is provided as input to the generative AI model, which then automatically extracts user needs and challenges. Specifically, the server analyzes user feedback such as "the app's response is slow" or "the new features are difficult to use" and identifies common issues. The output is a list of common user needs and challenges.

[0547] Step 4: Generate output

[0548] The server generates output (reports, dashboards, etc.) that visualizes user profiles and user experiences based on the results of data analysis. The input uses user needs and issues derived from data analysis. Specifically, the server uses data visualization technology to generate graphs and charts to visually display trends in user data and analytical results. As output, reports and dashboards are provided to end users in an easy-to-understand format.

[0549] Step 5: Update

[0550] The server automatically updates the output based on newly collected user data. The newly added data is included as input. Specifically, the server re-analyzes the latest voice and log data and reflects it in reports and dashboards. The output always provides reports and output based on the latest user data.

[0551] Through these steps, the system can efficiently process user data and promote fast and accurate user understanding.

[0552] (Application example 1)

[0553] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0554] Current online shopping sites lack systems that can effectively collect and analyze users' diverse needs and feedback and recommend products suited to each individual user in real time. This results in a poor user experience and a decline in purchasing intent. Furthermore, delays in product improvements based on collected feedback can potentially undermine a company's competitiveness.

[0555] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0556] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, means for updating the output based on new data, means for collecting user operation logs, purchase histories, and posts on social networking services, and means for analyzing the collected data based on prompts and recommending products that meet user needs. This makes it possible to analyze user behavior and feedback in real time and propose optimal products, thereby improving user satisfaction and strengthening corporate competitiveness.

[0557] definition statement

[0558] "User data" refers to data collected in a variety of forms, such as user operation logs, purchase history, posts on social networking services, voice data, log data, and feedback data.

[0559] A "consistent format" refers to a format in which collected user data is organized in a unified format to facilitate subsequent analysis.

[0560] A "generative AI model" is a model that uses machine learning technology to process data and extract user needs and issues.

[0561] "User needs" refers to what users need or want in a particular situation.

[0562] A "user profile" is a collection of user characteristics and behavioral patterns derived from collected and analyzed data.

[0563] "Output" refers to output such as reports and dashboards generated by the server to visualize the user profile and user experience.

[0564] A "prompt sentence" is a text input given to a generative AI model to enable it to analyze and generate.

[0565] "Recommendation" means proposing products or services that meet the user's needs based on the results of analysis.

[0566] An "operation log" is recorded data generated when a user operates an application or website.

[0567] "Purchase history" is a record of past purchases made by a user.

[0568] "Social networking service posts" refers to tweets, comments, images, and videos posted by users on social networking sites.

[0569] MODE FOR CARRYING OUT THE INVENTION

[0570] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To realize this system, the following configuration and processing are performed.

[0571] System configuration

[0572] The system includes the following main features:

[0573] User Data Collection

[0574] Data organization

[0575] Data analysis

[0576] Output Generation

[0577] update

[0578] 1. Collection of User Data

[0579] The server collects user data such as user operation logs, purchase history, and posts on social networking services, as well as click logs and form input data sent from user devices (smartphones, computers, etc.).

[0580] For example, click logs and purchase histories generated when a user uses an online shopping site are sent to the server in real time. This data is collected via API and used for subsequent processing.

[0581] 2. Data organization

[0582] The server then organizes the collected user data into a consistent format. For example, voice data is converted to text using a speech recognition API, and social media data is analyzed and tagged appropriately.

[0583] As a specific example, when reviews posted by users on social networking services are collected as audio data, they are converted into text data and given tags such as "positive" or "negative."

[0584] 3. Data analysis

[0585] The server analyzes the organized data using a generative AI model, such as OpenAI's GPT-3, to extract user needs and issues.

[0586] As a specific example, the subject of data analysis is a social media post stating, "The user is interested in new outfits." In this case, the prompt would be, "Analyze the following user data and identify the user's needs and challenges. The user has posted on social media that they are interested in new outfits."

[0587] 4. Output Generation

[0588] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results, making it easy to understand user needs and behavioral patterns.

[0589] For example, the generated report may include product recommendations based on user needs, visually displayed using graphs and charts.

[0590] 5. Updates

[0591] The server automatically updates the generated output based on new data, ensuring that the latest user information is always reflected.

[0592] For example, as new feedback data is acquired, changes in user satisfaction are reflected in the report.

[0593] In this way, the system of the present invention can efficiently collect, organize, and analyze user data and generate specific action plans based on the results, enabling development teams to accurately understand user needs and make rapid and effective product improvements.

[0594] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0595] Program processing flow

[0596] Step 1:

[0597] The device collects user data such as user operation logs, purchase history, and posts on social networking services. This also includes click logs and form input data generated when users use online shopping sites. The device sends the collected data to a server in real time. The input is various user data, and the output is data sent to the server.

[0598] Step 2:

[0599] The server organizes the collected user data into a consistent format. For example, voice data is converted into text using a speech recognition API (e.g., Google Cloud Speech-to-Text), and social media data is analyzed using natural language processing technology and appropriately tagged. The input is the collected user data, and the output is organized text data and tagged data.

[0600] Step 3:

[0601] The server analyzes the organized data using a generative AI model. During this process, the organized text data is input as prompts into the generative AI model (e.g., OpenAI GPT-3) to extract user needs and issues. The inputs are the organized data and prompts, and the output is the analysis results.

[0602] Step 4:

[0603] The server generates output (e.g., reports, dashboards) that visualize the user profile and user experience based on the analysis results. This output visually shows user needs and problems. The input is the analysis results, and the output is the visualized output.

[0604] Step 5:

[0605] The server updates the generated output based on the new data, ensuring that it always reflects the latest user information. For example, when the latest user feedback data is obtained, a report or dashboard is updated based on it. The input is the new user data, and the output is the updated output.

[0606] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0607] The present invention relates to a user understanding promotion system that combines an emotion engine, which streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Furthermore, it utilizes an emotion engine that recognizes user emotions to provide detailed analysis including user emotion information.

[0608] Program processing overview

[0609] Data collection

[0610] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[0611] Examples:

[0612] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[0613] Data organization

[0614] The server organizes the received user data into a consistent format.

[0615] Specific behavior:

[0616] The server uses voice recognition technology to convert the audio data of the user interview into text.

[0617] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[0618] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[0619] Data analysis

[0620] The server analyzes the organized data using generative AI and an emotion engine.

[0621] Specific behavior:

[0622] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[0623] The server clusters the user data and identifies different user segments.

[0624] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0625] Examples:

[0626] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems. It then uses an emotion engine to identify user sentiment regarding these problems and calculates the percentage of negative feedback.

[0627] Output Generation

[0628] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0629] Specific behavior:

[0630] The server generates reports that show the different needs and challenges of each user segment.

[0631] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[0632] The server provides these outputs in PDF format and as a web interface.

[0633] Examples:

[0634] The generated report shows the different needs and challenges of each user segment, highlighting items with particularly negative feedback, and includes sentiment information for each piece of feedback, along with visually displaying trends in user behavior and feedback content using graphs and charts.

[0635] update

[0636] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[0637] Specific behavior:

[0638] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[0639] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[0640] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0641] Examples:

[0642] When feedback data is updated, the server not only reflects changes in user satisfaction in the report, but also highlights areas where sentiment fluctuations are particularly noticeable.

[0643] In this way, by combining an emotion engine, the present invention provides detailed user understanding, including user emotional information, enabling development teams to make product improvements quickly and accurately.

[0644] The processing flow will be explained below.

[0645] Step 1: Data collection

[0646] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[0647] (Specific actions)

[0648] The server collects various data in real time through API endpoints.

[0649] The server checks the format of the received data and filters out any invalid or missing data.

[0650] Step 2: Data organization

[0651] The server organizes the received user data into a consistent format.

[0652] (Specific actions)

[0653] The server uses voice recognition technology to convert the audio data of the user interview into text.

[0654] The server converts the log data and feedback data into a standard format (JSON, CSV, etc.) and stores it in a database.

[0655] The server uses natural language processing (NLP) technology on the text data to extract and tag important keywords and themes.

[0656] Step 3: Emotion Recognition

[0657] The server analyzes the organized data using an emotion engine to identify the user's emotional state.

[0658] (Specific actions)

[0659] The server inputs text and voice data into an emotion engine and classifies the user's emotions as positive, negative, or neutral.

[0660] The server stores the emotion recognition results in a database, making them available for the next analysis step.

[0661] Step 4: Data analysis

[0662] The server uses generative AI to comprehensively analyze the organized data, including the results of the emotion engine, to extract user needs and issues.

[0663] (Specific actions)

[0664] The server analyzes the trends in user feedback based on the emotional state output by the emotion engine.

[0665] The server clusters user data to identify different segments and extracts common needs and challenges for each segment.

[0666] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0667] Examples:

[0668] The server uses an emotion engine to identify that there is a lot of feedback about the app's slow response, and identifies that there is a particularly large amount of negative emotion among the feedback.

[0669] Step 5: Generate output

[0670] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0671] (Specific actions)

[0672] The server generates reports that show the different needs and challenges of each user segment.

[0673] The server creates a dashboard that reflects sentiment information and data trends, displaying graphs and charts that are updated in real time.

[0674] The server provides the generated reports and dashboards in PDF format or as a web interface.

[0675] Examples:

[0676] The generated report shows the different needs and challenges of each user segment, highlights items with a high percentage of negative feedback, and includes a graph showing the emotional state of users based on the results of the emotion engine.

[0677] Step 6: Update

[0678] The server automatically updates the output based on the new data.

[0679] (Specific actions)

[0680] The server performs additional analysis on the newly collected data and re-evaluates the results of the emotion engine.

[0681] The server retrains the generative AI to continuously improve its analytical accuracy.

[0682] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0683] Examples:

[0684] The server analyzes the new feedback data and reports fluctuations in user satisfaction, as well as highlighting areas where sentiment fluctuations are particularly pronounced.

[0685] In this way, the present invention, by combining an emotion engine, achieves a deeper understanding of users, enabling development teams to make quick and accurate product improvements.

[0686] Example 2

[0687] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0688] In many modern systems, the collection, organization, and analysis of user data are carried out separately, resulting in a lack of a consistent process. This makes it difficult to accurately understand users, and in particular, detailed analysis including emotional information is not performed. Furthermore, the inability to update collected data in real time or automatically update output hinders continuous improvement of the user experience.

[0689] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model and an emotion recognition engine, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables consistent collection, organization, analysis, and real-time updating of user data. In addition, the use of the emotion recognition engine enables detailed analysis including user emotion information, enabling continuous improvement of the user experience.

[0690] "User data" refers to all information related to users, such as user behavior, feedback, usage logs, survey responses, and social media posts.

[0691] "Means of collection" refers to the processes and technologies used to receive user data from devices and store it on servers.

[0692] "Organization methods" refers to the processes and techniques used to convert collected user data into a consistent format and to cleanse, format, and store the data.

[0693] A "generative AI model" refers to an algorithm or system that uses machine learning or deep learning techniques to generate, analyze, and predict data.

[0694] An "emotion recognition engine" refers to a system that uses natural language processing and machine learning techniques to identify emotions (positive, negative, neutral, etc.) from user feedback and data.

[0695] "Means of analysis" refers to the processes and techniques used to analyze collected and organized user data using various analytical tools and techniques to extract insights and patterns.

[0696] "Means of generating output" refers to the processes and techniques used to create visual reports and dashboards based on the analysis results, and visualize user profiles and user experiences.

[0697] "Updating methods" refers to the processes and techniques used to reanalyze existing outputs based on newly collected data, ensuring that they always reflect the latest information.

[0698] The present invention relates to a user understanding promotion system that combines an emotion engine, and streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Specific embodiments of the system are described below.

[0699] Data collection

[0700] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[0701] Hardware / software used: Internet connection, database, cloud storage

[0702] Examples:

[0703] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[0704] Data organization

[0705] The server organizes the received user data into a consistent format.

[0706] Hardware / software used: Data cleansing tools, data format conversion tools

[0707] Examples:

[0708] The server converts the user interview audio data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text API, IBM Watson Speech to Text).

[0709] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[0710] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[0711] Data analysis

[0712] The server analyzes the organized data using generative AI models and emotion engines.

[0713] Hardware / software used: Machine learning platforms (e.g., TensorFlow, PyTorch), sentiment analysis tools (e.g., IBM Watson Natural Language Understanding)

[0714] Examples:

[0715] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[0716] The server clusters the user data and identifies different user segments (e.g., K-means clustering).

[0717] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0718] Example prompt: Identify user feedback such as "The app is slow to respond" or "The new features are difficult to use" and extract them as common problems. Use an emotion engine to identify user sentiment regarding these problems and calculate the percentage of negative feedback.

[0719] Output Generation

[0720] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0721] Hardware / software used: Report generation tools (e.g., Tableau, Microsoft Power BI), interactive dashboard tools (e.g., D3.js, Chart.js)

[0722] Examples:

[0723] The server generates reports that show the different needs and challenges of each user segment.

[0724] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[0725] The server provides these outputs in PDF format and via a web interface. The generated reports show the different needs and challenges of each user segment, highlighting areas that generate the most negative feedback.

[0726] update

[0727] The server updates its output based on new data, always providing the latest user profile and experience.

[0728] Hardware / software used: Cron jobs, scheduling tools, retraining platform

[0729] Examples:

[0730] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[0731] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[0732] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0733] In this way, by combining emotion engines, the present invention provides detailed user understanding including user emotion information, enabling rapid and accurate product improvements.

[0734] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0735] Step 1:

[0736] Data collection

[0737] Input: User data such as user utterances, app usage logs, survey responses, and social media posts

[0738] Output: raw data collected

[0739] Specific behavior:

[0740] 1. The user uses the app to perform various operations.

[0741] 2. The terminal records the user's operation log and input data in real time and sends them to the server.

[0742] 3. The server receives the interview audio data, survey results, and social media posts and stores them in a buffer.

[0743] Step 2:

[0744] Data organization

[0745] Input: Raw data collected

[0746] Output: A clean, uniformly formatted dataset

[0747] Specific behavior:

[0748] 1. The server converts the voice data into text data using speech recognition technology (e.g., Google Cloud Speech-to-Text API).

[0749] 2. The server uses a data cleansing tool to remove duplicates and errors from the raw data and organize the necessary information.

[0750] 3. The server converts the log data and feedback data into a unified format (JSON, CSV, etc.) and stores it in the database.

[0751] 4. The server uses natural language processing (NLP) technology to extract important keywords from the text data and tag them.

[0752] Step 3:

[0753] Data analysis

[0754] Input: A clean, uniformly formatted dataset

[0755] Output: Analysis results and insights

[0756] Specific behavior:

[0757] 1. The server analyzes the emotional state (positive, negative, neutral) of the text data using an emotion recognition engine (e.g., IBM Watson Natural Language Understanding).

[0758] 2. The server uses a generative AI model to analyze patterns in the data and segment users.

[0759] 3. The server performs trend analysis to identify major issues and user requests within a specific period of time.

[0760] 4. Example: Extract common issues from feedback such as "The app's response is slow" or "The new features are difficult to use," and use sentiment analysis to calculate the percentage of negative feedback.

[0761] Step 4:

[0762] Output Generation

[0763] Input: Analysis results and insights

[0764] Output: Visualized reports and dashboards

[0765] Specific behavior:

[0766] 1. The server generates a report based on the analysis results that shows the needs and challenges of each user segment (e.g., Tableau, Microsoft Power BI).

[0767] 2. The server creates a dashboard with trend graphs and charts that update in real time based on NLP and sentiment information (e.g. D3.js, Chart.js).

[0768] 3. The server provides the generated output in PDF format and / or via a web interface.

[0769] 4. Example: The generated report will highlight the different needs of different user segments and highlight areas with particularly negative feedback.

[0770] Step 5:

[0771] Output Updates

[0772] Input: Freshly collected raw data

[0773] Output: Updated reports and dashboards

[0774] Specific behavior:

[0775] 1. The server analyzes the newly collected data and applies it to existing outputs.

[0776] 2. The server retrains the generative AI model to continuously improve the accuracy of the analysis (e.g., TensorFlow, PyTorch).

[0777] 3. If the server detects any important changes or abnormalities, it will automatically notify relevant parties via alerts.

[0778] 4. Example: When new user feedback is collected, the server analyzes it using an emotion recognition engine and updates existing reports and dashboards accordingly.

[0779] (Application example 2)

[0780] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0781] There is a need for a means to efficiently collect, organize, and analyze diverse user behavior and feedback data in order to deepen user understanding in real time. However, to achieve this, technology is needed that can properly identify user emotions and provide fast, highly accurate analysis results. Furthermore, there is a need to improve the user experience by providing analysis results in a visually easy-to-understand format. A system that can solve these issues in an integrated manner is needed.

[0782] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0783] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data to extract user needs and issues, means for identifying the emotional state of user feedback using a sentiment analysis engine, means for generating an output that visualizes user profiles and user experiences based on the analysis results, and means for updating and visualizing the output in real time based on new data, thereby enabling rapid and accurate analysis of various user data and providing detailed analysis results, including user sentiment, in real time.

[0784] "User Data" refers to information generated when a user uses an application or system, including voice data, log data, feedback data, and SNS data.

[0785] "Means of collection" refers to the hardware and software functions required to efficiently obtain a variety of user data.

[0786] "Organization" refers to the ability to convert collected user data into a consistent format and make it analyzable.

[0787] "Means of analyzing and extracting user needs and issues" refers to algorithms and engines that use organized user data to determine the problems and requests users are facing.

[0788] "Means for generating output" refers to a function for visually expressing the user profile and user experience based on the analysis results.

[0789] "Means of updating" refers to the ability to automatically update existing outputs based on new data, ensuring that information is always up-to-date.

[0790] "Sentiment analysis engine" refers to technology used to identify a user's emotional state (e.g., positive, negative, neutral) from user feedback and other data.

[0791] "Visualization means" refers to the ability to display analytical results in a visual format such as graphs or charts.

[0792] As an embodiment of the present invention, a customer sentiment analysis system for a physical store is configured as follows.

[0793] Data collection

[0794] The server collects the following user data from users (customers) using their smartphones. Voice data is collected using the smartphone's microphone to gather customer feedback. Log data includes smartphone operation history, feedback data includes in-app ratings and comments, and social media data includes mentions posted by customers on social media.

[0795] Data organization

[0796] The server organizes the collected user data into a consistent format. It converts the voice data into text using speech recognition technology (e.g., the speech_recognition library). It converts the generated log and feedback data into JSON or CSV format and stores it in a database. It uses natural language processing (NLP) technology (e.g., the TextBlob library) to extract and tag important keywords and themes from the text data.

[0797] Data analysis

[0798] The server analyzes the organized data using generative AI and a sentiment engine, which identifies the emotional state of user feedback (positive, negative, neutral, etc.), clusters user data to identify different user segments, and performs trend analysis to extract key issues and requests within a specific period.

[0799] Output Generation

[0800] The server generates outputs based on the analysis results that visualize user profiles and user experiences. It generates reports showing the different needs and challenges of each user segment, creates dashboards based on user data and sentiment information, and displays visual trend graphs and charts that are updated in real time. These outputs are provided in PDF format or as a web interface.

[0801] update

[0802] The server automatically updates the output based on new data, always providing the latest user profile and user experience. It analyzes newly collected data, applies it to existing reports and dashboards using an emotion engine, and retrains the generative AI to continuously improve the accuracy of the analysis. If there are important changes or abnormalities, it automatically notifies relevant parties with alerts.

[0803] Specific examples

[0804] For example, if a customer gives voice feedback at a store, such as "The cash register is too slow," this is collected using the smartphone's microphone, converted into text data, and then judged as "negative" using the emotion engine. Social media data and other feedback data are also analyzed in real time and reflected in reports. In this way, items with a high rate of negative feedback can be identified and specific improvement measures can be implemented.

[0805] Prompt Sentence Examples

[0806] "Based on user behavioral data and feedback, use a sentiment analysis engine and generative AI model to answer the following questions:

[0807] Summarize the customer's feedback.

[0808] Determine your feelings about each piece of feedback (positive, negative, neutral).

[0809] List the most frequently cited issues over a specific period of time.

[0810] Analyze how your customers feel about each issue.

[0811] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0812] Step 1: Data collection

[0813] The server collects user data from users (customers) through their devices (smartphones). Voice feedback provided by customers is recorded using the smartphone's microphone, and other feedback data and social media data are also collected. These data are sent to the server and stored.

[0814] Input: Voice data, feedback data, social media data

[0815] Output: Raw data stored on the server

[0816] Step 2: Data organization

[0817] The server organizes the collected user data into a consistent format: voice data is converted into text using speech recognition technology, and log and feedback data is converted into JSON or CSV format.

[0818] Input: Raw data (audio data, feedback data, SNS data)

[0819] Output: Text data, JSON format data, CSV format data

[0820] Step 3: Natural Language Processing (NLP)

[0821] The server then performs natural language processing (NLP) on the organized data to extract and tag important keywords and themes, for example by using the TextBlob library to parse the text data.

[0822] Input: Organized data (text, JSON, CSV)

[0823] Output: Text data with keywords

[0824] Step 4: Sentiment analysis

[0825] The server runs the keyworded text data through a sentiment analysis engine to identify the emotional state of the user feedback (positive, negative, neutral, etc.). For example, the TextBlob library performs sentiment analysis.

[0826] Input: Text data with keywords

[0827] Output: Text data with emotion information

[0828] Step 5: Data Clustering

[0829] The server clusters user data based on the text data with emotional information, identifies different user segments, and uses a clustering algorithm to classify users by their attributes.

[0830] Input: Text data with emotion information

[0831] Output: Clustered user data

[0832] Step 6: Trend analysis

[0833] The server performs trend analysis based on the clustered user data, extracts the main issues and requests within a specific period, and identifies user trends and requests based on the analysis results.

[0834] Input: Clustered user data

[0835] Output: Trend analysis results

[0836] Step 7: Generate Output

[0837] The server generates output based on the trend analysis results, visualizing user profiles and user experiences. It creates visual reports (graphs and charts) that show the different needs and challenges of each user segment.

[0838] Input: Trend analysis results

[0839] Output: Visualized reports, dashboards

[0840] Step 8: Update the output

[0841] The server automatically updates the output based on new data, always providing the latest user profile and experience. Newly collected data is analyzed and the generative AI is retrained.

[0842] Input: New user data, existing analysis results

[0843] Output: Modernized reports, dashboards

[0844] The above steps enable the efficient collection, organization, analysis, output generation, and updating of diverse user data, making it possible to understand users in real time and provide improvement measures.

[0845] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0846] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0847] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0848] [Third embodiment]

[0849] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0850] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0851] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0852] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0853] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0854] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0855] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0856] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0857] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0858] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0859] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0860] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0861] The present invention relates to a system for promoting user understanding, and more specifically, to a system for consistently collecting, organizing, and analyzing user data, and generating and updating output to streamline the process of understanding users. The following describes in detail an embodiment of this system.

[0862] Program processing overview

[0863] Data collection

[0864] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[0865] Examples:

[0866] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[0867] Data organization

[0868] The server organizes the received user data into a consistent format, for example by converting audio data into text data and tagging the organized data.

[0869] Examples:

[0870] The server converts the audio data from user interviews into text and adds tags such as "dissatisfaction points," "requests," and "frequently used words."

[0871] Data analysis

[0872] The server analyzes the organized data using generative AI to extract user needs and challenges, particularly by using natural language processing technology to identify positive and negative feedback.

[0873] Examples:

[0874] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems.

[0875] Output Generation

[0876] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results.

[0877] Examples:

[0878] The generated report shows the different needs and challenges of each user segment, and uses graphs and charts to visually display trends in user behavior and feedback.

[0879] update

[0880] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[0881] Examples:

[0882] The server will take in new feedback data after the update and reflect changes in user satisfaction in the report.

[0883] This system allows for efficient collection, organization, and analysis of user data, and the creation of specific action plans based on the results, enabling the development team to accurately understand user needs and make rapid and effective product improvements.

[0884] The processing flow will be explained below.

[0885] Step 1: Data collection

[0886] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[0887] (Specific actions)

[0888] The server collects data in real time through designated API endpoints.

[0889] The server checks the format of the received data and filters out any invalid or missing data.

[0890] Step 2: Data organization

[0891] The server organizes the received user data into a consistent format.

[0892] (Specific actions)

[0893] The server uses voice recognition technology to convert the audio data of the user interview into text.

[0894] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[0895] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[0896] Step 3: Data analysis

[0897] The server analyzes the organized data using generative AI.

[0898] (Specific actions)

[0899] The server performs sentiment analysis to classify user feedback as positive, negative, or neutral.

[0900] The server clusters the user data and identifies different user segments.

[0901] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[0902] Step 4: Generate output

[0903] The server generates output that visualizes the user profile and user experience based on the analysis results.

[0904] (Specific actions)

[0905] The server generates reports that show the different needs and challenges of each user segment.

[0906] The server creates a dashboard based on user data, displaying visual trend graphs and charts that are updated in real time.

[0907] The server provides these outputs in PDF format and as a web interface.

[0908] Step 5: Update

[0909] The server updates the generated output based on the new data.

[0910] (Specific actions)

[0911] The server analyzes the newly collected data and incorporates it into existing reports and dashboards.

[0912] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[0913] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[0914] Through these steps, the present invention efficiently and automatically collects, organizes, and analyzes user data, significantly streamlining the process of understanding users, enabling development teams to accurately grasp user needs and make rapid and effective product improvements.

[0915] Example 1

[0916] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0917] In today's world, better understanding users is essential for improving products and services. However, user data exists in a variety of formats, making it complex and time-consuming to collect, organize, analyze, and then generate output from that data and update it based on the latest information. Therefore, there is a need for a method to process user data efficiently and in a consistent format to obtain fast and accurate feedback.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0919] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables efficient processing and updating of complex user data, and promotes rapid and accurate user understanding.

[0920] "User Data" refers to all information related to users, including voice data, log data, feedback data, and SNS data.

[0921] "Means of collection" refers to the methods and technologies required to store user data on a server.

[0922] "Organization means" refers to the methods and techniques used to convert collected user data into a consistent format and to tag and categorize the data.

[0923] A "generative AI model" refers to an artificial intelligence model that analyzes and generates information based on given data, such as one that uses natural language processing technology.

[0924] "Means of analysis" refers to methods and techniques for extracting user needs and issues using organized data.

[0925] "Means for generating output" refers to the process for visualizing the user profile and user experience based on the analysis results.

[0926] "Updating means" refers to the methods and techniques used to bring existing outputs up to date based on new data.

[0927] "Audio data" refers to audio recordings of a user's speech, which may be converted into text data for analysis.

[0928] "Log data" refers to detailed historical information about users' application operations and system usage.

[0929] "Feedback Data" refers to information including user ratings, opinions, comments, etc.

[0930] "SNS Data" refers to information, including comments and reactions, posted by users on social networking services.

[0931] "Tagging" refers to the act of adding specific labels or metadata to data, making it easier to search and classify the data.

[0932] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To implement this system, the following hardware and software are used.

[0933] Hardware and Software

[0934] The server is the central device for collecting, organizing, analyzing, generating output, and updating user data. It must be equipped with a high-performance processor, large storage capacity, and have a stable network connection.

[0935] A device is a device used to collect user data, such as a smartphone, tablet, or PC. The device is equipped with a microphone for recording audio data from user interviews and software for recording application usage logs.

[0936] A user is an end user who operates a terminal and interacts with the system, such as participating in an interview, answering a survey, or using an application.

[0937] Specific example of system operation

[0938] 1. Data Collection:

[0939] When a user operates the app, click logs and form input data are generated. This data is sent from the device to the server in real time. In addition, when a user interview is conducted, audio data is recorded through the device's microphone and sent to the server.

[0940] 2. Data reduction:

[0941] The server organizes the collected user data into a consistent format. For example, voice data is converted into text data using the Google Speech-to-Text API. The data is tagged with tags such as "dissatisfaction points," "requests," and "frequently used words."

[0942] 3. Data Analysis:

[0943] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). It uses natural language processing technology to extract user needs and issues. For example, feedback such as "the app's response is slow" or "the new features are difficult to use" is identified and extracted as common issues.

[0944] 4. Output generation:

[0945] The server generates outputs (e.g., reports and dashboards) based on the analysis results that visualize user profiles and user experiences. These outputs are presented in an easy-to-understand format, highlighting the different needs and challenges of each user segment.

[0946] 5. Update:

[0947] As new user data is added, the server automatically updates existing outputs, ensuring a consistent, up-to-date understanding of users. For example, it incorporates the latest feedback data and reflects changes in user satisfaction in its reports.

[0948] Prompt Sentence Examples

[0949] "Convert the audio data to text."

[0950] "Identify user needs and common problems."

[0951] This system allows the development team to quickly and accurately understand user needs and use them to improve the product, by consistently and efficiently collecting, organizing, analyzing, generating output, and updating data on user behavior and feedback.

[0952] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0953] Step 1: Data collection

[0954] The server begins collecting user data. As input, it receives click logs, form input data, audio data from user interviews, and social media posting data sent from the device. This data is stored in a database on the server. Specifically, the device records the user's actions and sends the data to the server in real time. For example, when a user clicks a button in an app, that information is transferred to the server within seconds and stored in the database.

[0955] Step 2: Data organization

[0956] The server organizes the collected user data into a consistent format. Voice data is converted into text data using the Google Speech-to-Text API, and log data and feedback data are also converted into a consistent format. This allows data of different formats to be stored in a unified format. Specifically, the server uses voice recognition technology to convert the interview audio into text. This converted data is organized with tags such as "dissatisfaction points," "requests," and "frequently used words."

[0957] Step 3: Data analysis

[0958] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). The organized text data is provided as input to the generative AI model, which then automatically extracts user needs and challenges. Specifically, the server analyzes user feedback such as "the app's response is slow" or "the new features are difficult to use" and identifies common issues. The output is a list of common user needs and challenges.

[0959] Step 4: Generate output

[0960] The server generates output (reports, dashboards, etc.) that visualizes user profiles and user experiences based on the results of data analysis. The input uses user needs and issues derived from data analysis. Specifically, the server uses data visualization technology to generate graphs and charts to visually display trends in user data and analytical results. As output, reports and dashboards are provided to end users in an easy-to-understand format.

[0961] Step 5: Update

[0962] The server automatically updates the output based on newly collected user data. The newly added data is included as input. Specifically, the server re-analyzes the latest voice and log data and reflects it in reports and dashboards. The output always provides reports and output based on the latest user data.

[0963] Through these steps, the system can efficiently process user data and promote fast and accurate user understanding.

[0964] (Application example 1)

[0965] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0966] Current online shopping sites lack systems that can effectively collect and analyze users' diverse needs and feedback and recommend products suited to each individual user in real time. This results in a poor user experience and a decline in purchasing intent. Furthermore, delays in product improvements based on collected feedback can potentially undermine a company's competitiveness.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0968] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, means for updating the output based on new data, means for collecting user operation logs, purchase histories, and posts on social networking services, and means for analyzing the collected data based on prompts and recommending products that meet user needs. This makes it possible to analyze user behavior and feedback in real time and propose optimal products, thereby improving user satisfaction and strengthening corporate competitiveness.

[0969] definition statement

[0970] "User data" refers to data collected in a variety of forms, such as user operation logs, purchase history, posts on social networking services, voice data, log data, and feedback data.

[0971] A "consistent format" refers to a format in which collected user data is organized in a unified format to facilitate subsequent analysis.

[0972] A "generative AI model" is a model that uses machine learning technology to process data and extract user needs and issues.

[0973] "User needs" refers to what users need or want in a particular situation.

[0974] A "user profile" is a collection of user characteristics and behavioral patterns derived from collected and analyzed data.

[0975] "Output" refers to output such as reports and dashboards generated by the server to visualize the user profile and user experience.

[0976] A "prompt sentence" is a text input given to a generative AI model to enable it to analyze and generate.

[0977] "Recommendation" means proposing products or services that meet the user's needs based on the results of analysis.

[0978] An "operation log" is recorded data generated when a user operates an application or website.

[0979] "Purchase history" is a record of past purchases made by a user.

[0980] "Social networking service posts" refers to tweets, comments, images, and videos posted by users on social networking sites.

[0981] MODE FOR CARRYING OUT THE INVENTION

[0982] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To realize this system, the following configuration and processing are performed.

[0983] System configuration

[0984] The system includes the following main features:

[0985] User Data Collection

[0986] Data organization

[0987] Data analysis

[0988] Output Generation

[0989] update

[0990] 1. Collection of User Data

[0991] The server collects user data such as user operation logs, purchase history, and posts on social networking services, as well as click logs and form input data sent from user devices (smartphones, computers, etc.).

[0992] For example, click logs and purchase histories generated when a user uses an online shopping site are sent to the server in real time. This data is collected via API and used for subsequent processing.

[0993] 2. Data organization

[0994] The server then organizes the collected user data into a consistent format. For example, voice data is converted to text using a speech recognition API, and social media data is analyzed and tagged appropriately.

[0995] As a specific example, when reviews posted by users on social networking services are collected as audio data, they are converted into text data and given tags such as "positive" or "negative."

[0996] 3. Data analysis

[0997] The server analyzes the organized data using a generative AI model, such as OpenAI's GPT-3, to extract user needs and issues.

[0998] As a specific example, the subject of data analysis is a social media post stating, "The user is interested in new outfits." In this case, the prompt would be, "Analyze the following user data and identify the user's needs and challenges. The user has posted on social media that they are interested in new outfits."

[0999] 4. Output Generation

[1000] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results, making it easy to understand user needs and behavioral patterns.

[1001] For example, the generated report may include product recommendations based on user needs, visually displayed using graphs and charts.

[1002] 5. Updates

[1003] The server automatically updates the generated output based on new data, ensuring that the latest user information is always reflected.

[1004] For example, as new feedback data is acquired, changes in user satisfaction are reflected in the report.

[1005] In this way, the system of the present invention can efficiently collect, organize, and analyze user data and generate specific action plans based on the results, enabling development teams to accurately understand user needs and make rapid and effective product improvements.

[1006] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1007] Program processing flow

[1008] Step 1:

[1009] The device collects user data such as user operation logs, purchase history, and posts on social networking services. This also includes click logs and form input data generated when users use online shopping sites. The device sends the collected data to a server in real time. The input is various user data, and the output is data sent to the server.

[1010] Step 2:

[1011] The server organizes the collected user data into a consistent format. For example, voice data is converted into text using a speech recognition API (e.g., Google Cloud Speech-to-Text), and social media data is analyzed using natural language processing technology and appropriately tagged. The input is the collected user data, and the output is organized text data and tagged data.

[1012] Step 3:

[1013] The server analyzes the organized data using a generative AI model. During this process, the organized text data is input as prompts into the generative AI model (e.g., OpenAI GPT-3) to extract user needs and issues. The inputs are the organized data and prompts, and the output is the analysis results.

[1014] Step 4:

[1015] The server generates output (e.g., reports, dashboards) that visualize the user profile and user experience based on the analysis results. This output visually shows user needs and problems. The input is the analysis results, and the output is the visualized output.

[1016] Step 5:

[1017] The server updates the generated output based on the new data, ensuring that it always reflects the latest user information. For example, when the latest user feedback data is obtained, a report or dashboard is updated based on it. The input is the new user data, and the output is the updated output.

[1018] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1019] The present invention relates to a user understanding promotion system that combines an emotion engine, which streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Furthermore, it utilizes an emotion engine that recognizes user emotions to provide detailed analysis including user emotion information.

[1020] Program processing overview

[1021] Data collection

[1022] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[1023] Examples:

[1024] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[1025] Data organization

[1026] The server organizes the received user data into a consistent format.

[1027] Specific behavior:

[1028] The server uses voice recognition technology to convert the audio data of the user interview into text.

[1029] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[1030] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[1031] Data analysis

[1032] The server analyzes the organized data using generative AI and an emotion engine.

[1033] Specific behavior:

[1034] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[1035] The server clusters the user data and identifies different user segments.

[1036] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[1037] Examples:

[1038] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems. It then uses an emotion engine to identify user sentiment regarding these problems and calculates the percentage of negative feedback.

[1039] Output Generation

[1040] The server generates output that visualizes the user profile and user experience based on the analysis results.

[1041] Specific behavior:

[1042] The server generates reports that show the different needs and challenges of each user segment.

[1043] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[1044] The server provides these outputs in PDF format and as a web interface.

[1045] Examples:

[1046] The generated report shows the different needs and challenges of each user segment, highlighting items with particularly negative feedback, and includes sentiment information for each piece of feedback, along with visually displaying trends in user behavior and feedback content using graphs and charts.

[1047] update

[1048] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[1049] Specific behavior:

[1050] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[1051] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[1052] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[1053] Examples:

[1054] When feedback data is updated, the server not only reflects changes in user satisfaction in the report, but also highlights areas where sentiment fluctuations are particularly noticeable.

[1055] In this way, by combining an emotion engine, the present invention provides detailed user understanding, including user emotional information, enabling development teams to make product improvements quickly and accurately.

[1056] The processing flow will be explained below.

[1057] Step 1: Data collection

[1058] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[1059] (Specific actions)

[1060] The server collects various data in real time through API endpoints.

[1061] The server checks the format of the received data and filters out any invalid or missing data.

[1062] Step 2: Data organization

[1063] The server organizes the received user data into a consistent format.

[1064] (Specific actions)

[1065] The server uses voice recognition technology to convert the audio data of the user interview into text.

[1066] The server converts the log data and feedback data into a standard format (JSON, CSV, etc.) and stores it in a database.

[1067] The server uses natural language processing (NLP) technology on the text data to extract and tag important keywords and themes.

[1068] Step 3: Emotion Recognition

[1069] The server analyzes the organized data using an emotion engine to identify the user's emotional state.

[1070] (Specific actions)

[1071] The server inputs text and voice data into an emotion engine and classifies the user's emotions as positive, negative, or neutral.

[1072] The server stores the emotion recognition results in a database, making them available for the next analysis step.

[1073] Step 4: Data analysis

[1074] The server uses generative AI to comprehensively analyze the organized data, including the results of the emotion engine, to extract user needs and issues.

[1075] (Specific actions)

[1076] The server analyzes the trends in user feedback based on the emotional state output by the emotion engine.

[1077] The server clusters user data to identify different segments and extracts common needs and challenges for each segment.

[1078] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[1079] Examples:

[1080] The server uses an emotion engine to identify that there is a lot of feedback about the app's slow response, and identifies that there is a particularly large amount of negative emotion among the feedback.

[1081] Step 5: Generate output

[1082] The server generates output that visualizes the user profile and user experience based on the analysis results.

[1083] (Specific actions)

[1084] The server generates reports that show the different needs and challenges of each user segment.

[1085] The server creates a dashboard that reflects sentiment information and data trends, displaying graphs and charts that are updated in real time.

[1086] The server provides the generated reports and dashboards in PDF format or as a web interface.

[1087] Examples:

[1088] The generated report shows the different needs and challenges of each user segment, highlights items with a high percentage of negative feedback, and includes a graph showing the emotional state of users based on the results of the emotion engine.

[1089] Step 6: Update

[1090] The server automatically updates the output based on the new data.

[1091] (Specific actions)

[1092] The server performs additional analysis on the newly collected data and re-evaluates the results of the emotion engine.

[1093] The server retrains the generative AI to continuously improve its analytical accuracy.

[1094] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[1095] Examples:

[1096] The server analyzes the new feedback data and reports fluctuations in user satisfaction, as well as highlighting areas where sentiment fluctuations are particularly pronounced.

[1097] In this way, the present invention, by combining an emotion engine, achieves a deeper understanding of users, enabling development teams to make quick and accurate product improvements.

[1098] Example 2

[1099] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1100] In many modern systems, the collection, organization, and analysis of user data are carried out separately, resulting in a lack of a consistent process. This makes it difficult to accurately understand users, and in particular, detailed analysis including emotional information is not performed. Furthermore, the inability to update collected data in real time or automatically update output hinders continuous improvement of the user experience.

[1101] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model and an emotion recognition engine, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables consistent collection, organization, analysis, and real-time updating of user data. In addition, the use of the emotion recognition engine enables detailed analysis including user emotion information, enabling continuous improvement of the user experience.

[1102] "User data" refers to all information related to users, such as user behavior, feedback, usage logs, survey responses, and social media posts.

[1103] "Means of collection" refers to the processes and technologies used to receive user data from devices and store it on servers.

[1104] "Organization methods" refers to the processes and techniques used to convert collected user data into a consistent format and to cleanse, format, and store the data.

[1105] A "generative AI model" refers to an algorithm or system that uses machine learning or deep learning techniques to generate, analyze, and predict data.

[1106] An "emotion recognition engine" refers to a system that uses natural language processing and machine learning techniques to identify emotions (positive, negative, neutral, etc.) from user feedback and data.

[1107] "Means of analysis" refers to the processes and techniques used to analyze collected and organized user data using various analytical tools and techniques to extract insights and patterns.

[1108] "Means of generating output" refers to the processes and techniques used to create visual reports and dashboards based on the analysis results, and visualize user profiles and user experiences.

[1109] "Updating methods" refers to the processes and techniques used to reanalyze existing outputs based on newly collected data, ensuring that they always reflect the latest information.

[1110] The present invention relates to a user understanding promotion system that combines an emotion engine, and streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Specific embodiments of the system are described below.

[1111] Data collection

[1112] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[1113] Hardware / software used: Internet connection, database, cloud storage

[1114] Examples:

[1115] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[1116] Data organization

[1117] The server organizes the received user data into a consistent format.

[1118] Hardware / software used: Data cleansing tools, data format conversion tools

[1119] Examples:

[1120] The server converts the user interview audio data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text API, IBM Watson Speech to Text).

[1121] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[1122] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[1123] Data analysis

[1124] The server analyzes the organized data using generative AI models and emotion engines.

[1125] Hardware / software used: Machine learning platforms (e.g., TensorFlow, PyTorch), sentiment analysis tools (e.g., IBM Watson Natural Language Understanding)

[1126] Examples:

[1127] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[1128] The server clusters the user data and identifies different user segments (e.g., K-means clustering).

[1129] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[1130] Example prompt: Identify user feedback such as "The app is slow to respond" or "The new features are difficult to use" and extract them as common problems. Use an emotion engine to identify user sentiment regarding these problems and calculate the percentage of negative feedback.

[1131] Output Generation

[1132] The server generates output that visualizes the user profile and user experience based on the analysis results.

[1133] Hardware / software used: Report generation tools (e.g., Tableau, Microsoft Power BI), interactive dashboard tools (e.g., D3.js, Chart.js)

[1134] Examples:

[1135] The server generates reports that show the different needs and challenges of each user segment.

[1136] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[1137] The server provides these outputs in PDF format and via a web interface. The generated reports show the different needs and challenges of each user segment, highlighting areas that generate the most negative feedback.

[1138] update

[1139] The server updates its output based on new data, always providing the latest user profile and experience.

[1140] Hardware / software used: Cron jobs, scheduling tools, retraining platform

[1141] Examples:

[1142] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[1143] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[1144] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[1145] In this way, by combining emotion engines, the present invention provides detailed user understanding including user emotion information, enabling rapid and accurate product improvements.

[1146] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1147] Step 1:

[1148] Data collection

[1149] Input: User data such as user utterances, app usage logs, survey responses, and social media posts

[1150] Output: raw data collected

[1151] Specific behavior:

[1152] 1. The user uses the app to perform various operations.

[1153] 2. The terminal records the user's operation log and input data in real time and sends them to the server.

[1154] 3. The server receives the interview audio data, survey results, and social media posts and stores them in a buffer.

[1155] Step 2:

[1156] Data organization

[1157] Input: Raw data collected

[1158] Output: A clean, uniformly formatted dataset

[1159] Specific behavior:

[1160] 1. The server converts the voice data into text data using speech recognition technology (e.g., Google Cloud Speech-to-Text API).

[1161] 2. The server uses a data cleansing tool to remove duplicates and errors from the raw data and organize the necessary information.

[1162] 3. The server converts the log data and feedback data into a unified format (JSON, CSV, etc.) and stores it in the database.

[1163] 4. The server uses natural language processing (NLP) technology to extract important keywords from the text data and tag them.

[1164] Step 3:

[1165] Data analysis

[1166] Input: A clean, uniformly formatted dataset

[1167] Output: Analysis results and insights

[1168] Specific behavior:

[1169] 1. The server analyzes the emotional state (positive, negative, neutral) of the text data using an emotion recognition engine (e.g., IBM Watson Natural Language Understanding).

[1170] 2. The server uses a generative AI model to analyze patterns in the data and segment users.

[1171] 3. The server performs trend analysis to identify major issues and user requests within a specific period of time.

[1172] 4. Example: Extract common issues from feedback such as "The app's response is slow" or "The new features are difficult to use," and use sentiment analysis to calculate the percentage of negative feedback.

[1173] Step 4:

[1174] Output Generation

[1175] Input: Analysis results and insights

[1176] Output: Visualized reports and dashboards

[1177] Specific behavior:

[1178] 1. The server generates a report based on the analysis results that shows the needs and challenges of each user segment (e.g., Tableau, Microsoft Power BI).

[1179] 2. The server creates a dashboard with trend graphs and charts that update in real time based on NLP and sentiment information (e.g. D3.js, Chart.js).

[1180] 3. The server provides the generated output in PDF format and / or via a web interface.

[1181] 4. Example: The generated report will highlight the different needs of different user segments and highlight areas with particularly negative feedback.

[1182] Step 5:

[1183] Output Updates

[1184] Input: Freshly collected raw data

[1185] Output: Updated reports and dashboards

[1186] Specific behavior:

[1187] 1. The server analyzes the newly collected data and applies it to existing outputs.

[1188] 2. The server retrains the generative AI model to continuously improve the accuracy of the analysis (e.g., TensorFlow, PyTorch).

[1189] 3. If the server detects any important changes or abnormalities, it will automatically notify relevant parties via alerts.

[1190] 4. Example: When new user feedback is collected, the server analyzes it using an emotion recognition engine and updates existing reports and dashboards accordingly.

[1191] (Application example 2)

[1192] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1193] There is a need for a means to efficiently collect, organize, and analyze diverse user behavior and feedback data in order to deepen user understanding in real time. However, to achieve this, technology is needed that can properly identify user emotions and provide fast, highly accurate analysis results. Furthermore, there is a need to improve the user experience by providing analysis results in a visually easy-to-understand format. A system that can solve these issues in an integrated manner is needed.

[1194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1195] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data to extract user needs and issues, means for identifying the emotional state of user feedback using a sentiment analysis engine, means for generating an output that visualizes user profiles and user experiences based on the analysis results, and means for updating and visualizing the output in real time based on new data, thereby enabling rapid and accurate analysis of various user data and providing detailed analysis results, including user sentiment, in real time.

[1196] "User Data" refers to information generated when a user uses an application or system, including voice data, log data, feedback data, and SNS data.

[1197] "Means of collection" refers to the hardware and software functions required to efficiently obtain a variety of user data.

[1198] "Organization" refers to the ability to convert collected user data into a consistent format and make it analyzable.

[1199] "Means of analyzing and extracting user needs and issues" refers to algorithms and engines that use organized user data to determine the problems and requests users are facing.

[1200] "Means for generating output" refers to a function for visually expressing the user profile and user experience based on the analysis results.

[1201] "Means of updating" refers to the ability to automatically update existing outputs based on new data, ensuring that information is always up-to-date.

[1202] "Sentiment analysis engine" refers to technology used to identify a user's emotional state (e.g., positive, negative, neutral) from user feedback and other data.

[1203] "Visualization means" refers to the ability to display analytical results in a visual format such as graphs or charts.

[1204] As an embodiment of the present invention, a customer sentiment analysis system for a physical store is configured as follows.

[1205] Data collection

[1206] The server collects the following user data from users (customers) using their smartphones. Voice data is collected using the smartphone's microphone to gather customer feedback. Log data includes smartphone operation history, feedback data includes in-app ratings and comments, and social media data includes mentions posted by customers on social media.

[1207] Data organization

[1208] The server organizes the collected user data into a consistent format. It converts the voice data into text using speech recognition technology (e.g., the speech_recognition library). It converts the generated log and feedback data into JSON or CSV format and stores it in a database. It uses natural language processing (NLP) technology (e.g., the TextBlob library) to extract and tag important keywords and themes from the text data.

[1209] Data analysis

[1210] The server analyzes the organized data using generative AI and a sentiment engine, which identifies the emotional state of user feedback (positive, negative, neutral, etc.), clusters user data to identify different user segments, and performs trend analysis to extract key issues and requests within a specific period.

[1211] Output Generation

[1212] The server generates outputs based on the analysis results that visualize user profiles and user experiences. It generates reports showing the different needs and challenges of each user segment, creates dashboards based on user data and sentiment information, and displays visual trend graphs and charts that are updated in real time. These outputs are provided in PDF format or as a web interface.

[1213] update

[1214] The server automatically updates the output based on new data, always providing the latest user profile and user experience. It analyzes newly collected data, applies it to existing reports and dashboards using an emotion engine, and retrains the generative AI to continuously improve the accuracy of the analysis. If there are important changes or abnormalities, it automatically notifies relevant parties with alerts.

[1215] Specific examples

[1216] For example, if a customer gives voice feedback at a store, such as "The cash register is too slow," this is collected using the smartphone's microphone, converted into text data, and then judged as "negative" using the emotion engine. Social media data and other feedback data are also analyzed in real time and reflected in reports. In this way, items with a high rate of negative feedback can be identified and specific improvement measures can be implemented.

[1217] Prompt Sentence Examples

[1218] "Based on user behavioral data and feedback, use a sentiment analysis engine and generative AI model to answer the following questions:

[1219] Summarize the customer's feedback.

[1220] Determine your feelings about each piece of feedback (positive, negative, neutral).

[1221] List the most frequently cited issues over a specific period of time.

[1222] Analyze how your customers feel about each issue.

[1223] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1224] Step 1: Data collection

[1225] The server collects user data from users (customers) through their devices (smartphones). Voice feedback provided by customers is recorded using the smartphone's microphone, and other feedback data and social media data are also collected. These data are sent to the server and stored.

[1226] Input: Voice data, feedback data, social media data

[1227] Output: Raw data stored on the server

[1228] Step 2: Data organization

[1229] The server organizes the collected user data into a consistent format: voice data is converted into text using speech recognition technology, and log and feedback data is converted into JSON or CSV format.

[1230] Input: Raw data (audio data, feedback data, SNS data)

[1231] Output: Text data, JSON format data, CSV format data

[1232] Step 3: Natural Language Processing (NLP)

[1233] The server then performs natural language processing (NLP) on the organized data to extract and tag important keywords and themes, for example by using the TextBlob library to parse the text data.

[1234] Input: Organized data (text, JSON, CSV)

[1235] Output: Text data with keywords

[1236] Step 4: Sentiment analysis

[1237] The server runs the keyworded text data through a sentiment analysis engine to identify the emotional state of the user feedback (positive, negative, neutral, etc.). For example, the TextBlob library performs sentiment analysis.

[1238] Input: Text data with keywords

[1239] Output: Text data with emotion information

[1240] Step 5: Data Clustering

[1241] The server clusters user data based on the text data with emotional information, identifies different user segments, and uses a clustering algorithm to classify users by their attributes.

[1242] Input: Text data with emotion information

[1243] Output: Clustered user data

[1244] Step 6: Trend analysis

[1245] The server performs trend analysis based on the clustered user data, extracts the main issues and requests within a specific period, and identifies user trends and requests based on the analysis results.

[1246] Input: Clustered user data

[1247] Output: Trend analysis results

[1248] Step 7: Generate Output

[1249] The server generates output based on the trend analysis results, visualizing user profiles and user experiences. It creates visual reports (graphs and charts) that show the different needs and challenges of each user segment.

[1250] Input: Trend analysis results

[1251] Output: Visualized reports, dashboards

[1252] Step 8: Update the output

[1253] The server automatically updates the output based on new data, always providing the latest user profile and experience. Newly collected data is analyzed and the generative AI is retrained.

[1254] Input: New user data, existing analysis results

[1255] Output: Modernized reports, dashboards

[1256] The above steps enable the efficient collection, organization, analysis, output generation, and updating of diverse user data, making it possible to understand users in real time and provide improvement measures.

[1257] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1258] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1259] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1260] [Fourth embodiment]

[1261] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1262] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1263] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1264] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1265] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1266] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1267] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1268] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1269] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1270] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1271] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1272] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1273] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1274] The present invention relates to a system for promoting user understanding, and more specifically, to a system for consistently collecting, organizing, and analyzing user data, and generating and updating output to streamline the process of understanding users. The following describes in detail an embodiment of this system.

[1275] Program processing overview

[1276] Data collection

[1277] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[1278] Examples:

[1279] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[1280] Data organization

[1281] The server organizes the received user data into a consistent format, for example by converting audio data into text data and tagging the organized data.

[1282] Examples:

[1283] The server converts the audio data from user interviews into text and adds tags such as "dissatisfaction points," "requests," and "frequently used words."

[1284] Data analysis

[1285] The server analyzes the organized data using generative AI to extract user needs and challenges, particularly by using natural language processing technology to identify positive and negative feedback.

[1286] Examples:

[1287] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems.

[1288] Output Generation

[1289] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results.

[1290] Examples:

[1291] The generated report shows the different needs and challenges of each user segment, and uses graphs and charts to visually display trends in user behavior and feedback.

[1292] update

[1293] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[1294] Examples:

[1295] The server will take in new feedback data after the update and reflect changes in user satisfaction in the report.

[1296] This system allows for efficient collection, organization, and analysis of user data, and the creation of specific action plans based on the results, enabling the development team to accurately understand user needs and make rapid and effective product improvements.

[1297] The processing flow will be explained below.

[1298] Step 1: Data collection

[1299] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[1300] (Specific actions)

[1301] The server collects data in real time through designated API endpoints.

[1302] The server checks the format of the received data and filters out any invalid or missing data.

[1303] Step 2: Data organization

[1304] The server organizes the received user data into a consistent format.

[1305] (Specific actions)

[1306] The server uses voice recognition technology to convert the audio data of the user interview into text.

[1307] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[1308] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[1309] Step 3: Data analysis

[1310] The server analyzes the organized data using generative AI.

[1311] (Specific actions)

[1312] The server performs sentiment analysis to classify user feedback as positive, negative, or neutral.

[1313] The server clusters the user data and identifies different user segments.

[1314] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[1315] Step 4: Generate output

[1316] The server generates output that visualizes the user profile and user experience based on the analysis results.

[1317] (Specific actions)

[1318] The server generates reports that show the different needs and challenges of each user segment.

[1319] The server creates a dashboard based on user data, displaying visual trend graphs and charts that are updated in real time.

[1320] The server provides these outputs in PDF format and as a web interface.

[1321] Step 5: Update

[1322] The server updates the generated output based on the new data.

[1323] (Specific actions)

[1324] The server analyzes the newly collected data and incorporates it into existing reports and dashboards.

[1325] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[1326] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[1327] Through these steps, the present invention efficiently and automatically collects, organizes, and analyzes user data, significantly streamlining the process of understanding users, enabling development teams to accurately grasp user needs and make rapid and effective product improvements.

[1328] Example 1

[1329] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1330] In today's world, better understanding users is essential for improving products and services. However, user data exists in a variety of formats, making it complex and time-consuming to collect, organize, analyze, and then generate output from that data and update it based on the latest information. Therefore, there is a need for a method to process user data efficiently and in a consistent format to obtain fast and accurate feedback.

[1331] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1332] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables efficient processing and updating of complex user data, and promotes rapid and accurate user understanding.

[1333] "User Data" refers to all information related to users, including voice data, log data, feedback data, and SNS data.

[1334] "Means of collection" refers to the methods and technologies required to store user data on a server.

[1335] "Organization means" refers to the methods and techniques used to convert collected user data into a consistent format and to tag and categorize the data.

[1336] A "generative AI model" refers to an artificial intelligence model that analyzes and generates information based on given data, such as one that uses natural language processing technology.

[1337] "Means of analysis" refers to methods and techniques for extracting user needs and issues using organized data.

[1338] "Means for generating output" refers to the process for visualizing the user profile and user experience based on the analysis results.

[1339] "Updating means" refers to the methods and techniques used to bring existing outputs up to date based on new data.

[1340] "Audio data" refers to audio recordings of a user's speech, which may be converted into text data for analysis.

[1341] "Log data" refers to detailed historical information about users' application operations and system usage.

[1342] "Feedback Data" refers to information including user ratings, opinions, comments, etc.

[1343] "SNS Data" refers to information, including comments and reactions, posted by users on social networking services.

[1344] "Tagging" refers to the act of adding specific labels or metadata to data, making it easier to search and classify the data.

[1345] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To implement this system, the following hardware and software are used.

[1346] Hardware and Software

[1347] The server is the central device for collecting, organizing, analyzing, generating output, and updating user data. It must be equipped with a high-performance processor, large storage capacity, and have a stable network connection.

[1348] A device is a device used to collect user data, such as a smartphone, tablet, or PC. The device is equipped with a microphone for recording audio data from user interviews and software for recording application usage logs.

[1349] A user is an end user who operates a terminal and interacts with the system, such as participating in an interview, answering a survey, or using an application.

[1350] Specific example of system operation

[1351] 1. Data Collection:

[1352] When a user operates the app, click logs and form input data are generated. This data is sent from the device to the server in real time. In addition, when a user interview is conducted, audio data is recorded through the device's microphone and sent to the server.

[1353] 2. Data reduction:

[1354] The server organizes the collected user data into a consistent format. For example, voice data is converted into text data using the Google Speech-to-Text API. The data is tagged with tags such as "dissatisfaction points," "requests," and "frequently used words."

[1355] 3. Data Analysis:

[1356] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). It uses natural language processing technology to extract user needs and issues. For example, feedback such as "the app's response is slow" or "the new features are difficult to use" is identified and extracted as common issues.

[1357] 4. Output generation:

[1358] The server generates outputs (e.g., reports and dashboards) based on the analysis results that visualize user profiles and user experiences. These outputs are presented in an easy-to-understand format, highlighting the different needs and challenges of each user segment.

[1359] 5. Update:

[1360] As new user data is added, the server automatically updates existing outputs, ensuring a consistent, up-to-date understanding of users. For example, it incorporates the latest feedback data and reflects changes in user satisfaction in its reports.

[1361] Prompt Sentence Examples

[1362] "Convert the audio data to text."

[1363] "Identify user needs and common problems."

[1364] This system allows the development team to quickly and accurately understand user needs and use them to improve the product, by consistently and efficiently collecting, organizing, analyzing, generating output, and updating data on user behavior and feedback.

[1365] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1366] Step 1: Data collection

[1367] The server begins collecting user data. As input, it receives click logs, form input data, audio data from user interviews, and social media posting data sent from the device. This data is stored in a database on the server. Specifically, the device records the user's actions and sends the data to the server in real time. For example, when a user clicks a button in an app, that information is transferred to the server within seconds and stored in the database.

[1368] Step 2: Data organization

[1369] The server organizes the collected user data into a consistent format. Voice data is converted into text data using the Google Speech-to-Text API, and log data and feedback data are also converted into a consistent format. This allows data of different formats to be stored in a unified format. Specifically, the server uses voice recognition technology to convert the interview audio into text. This converted data is organized with tags such as "dissatisfaction points," "requests," and "frequently used words."

[1370] Step 3: Data analysis

[1371] The server analyzes the organized data using a generative AI model (e.g., OpenAI GPT-4). The organized text data is provided as input to the generative AI model, which then automatically extracts user needs and challenges. Specifically, the server analyzes user feedback such as "the app's response is slow" or "the new features are difficult to use" and identifies common issues. The output is a list of common user needs and challenges.

[1372] Step 4: Generate output

[1373] The server generates output (reports, dashboards, etc.) that visualizes user profiles and user experiences based on the results of data analysis. The input uses user needs and issues derived from data analysis. Specifically, the server uses data visualization technology to generate graphs and charts to visually display trends in user data and analytical results. As output, reports and dashboards are provided to end users in an easy-to-understand format.

[1374] Step 5: Update

[1375] The server automatically updates the output based on newly collected user data. The newly added data is included as input. Specifically, the server re-analyzes the latest voice and log data and reflects it in reports and dashboards. The output always provides reports and output based on the latest user data.

[1376] Through these steps, the system can efficiently process user data and promote fast and accurate user understanding.

[1377] (Application example 1)

[1378] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1379] Current online shopping sites lack systems that can effectively collect and analyze users' diverse needs and feedback and recommend products suited to each individual user in real time. This results in a poor user experience and a decline in purchasing intent. Furthermore, delays in product improvements based on collected feedback can potentially undermine a company's competitiveness.

[1380] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1381] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model to extract user needs and issues, means for generating output that visualizes user profiles and user experiences based on the analysis results, means for updating the output based on new data, means for collecting user operation logs, purchase histories, and posts on social networking services, and means for analyzing the collected data based on prompts and recommending products that meet user needs. This makes it possible to analyze user behavior and feedback in real time and propose optimal products, thereby improving user satisfaction and strengthening corporate competitiveness.

[1382] definition statement

[1383] "User data" refers to data collected in a variety of forms, such as user operation logs, purchase history, posts on social networking services, voice data, log data, and feedback data.

[1384] A "consistent format" refers to a format in which collected user data is organized in a unified format to facilitate subsequent analysis.

[1385] A "generative AI model" is a model that uses machine learning technology to process data and extract user needs and issues.

[1386] "User needs" refers to what users need or want in a particular situation.

[1387] A "user profile" is a collection of user characteristics and behavioral patterns derived from collected and analyzed data.

[1388] "Output" refers to output such as reports and dashboards generated by the server to visualize the user profile and user experience.

[1389] A "prompt sentence" is a text input given to a generative AI model to enable it to analyze and generate.

[1390] "Recommendation" means proposing products or services that meet the user's needs based on the results of analysis.

[1391] An "operation log" is recorded data generated when a user operates an application or website.

[1392] "Purchase history" is a record of past purchases made by a user.

[1393] "Social networking service posts" refers to tweets, comments, images, and videos posted by users on social networking sites.

[1394] MODE FOR CARRYING OUT THE INVENTION

[1395] The present invention relates to a system for promoting user understanding, specifically, a system that streamlines the process of user understanding by consistently collecting, organizing, analyzing user data, and generating and updating output. To realize this system, the following configuration and processing are performed.

[1396] System configuration

[1397] The system includes the following main features:

[1398] User Data Collection

[1399] Data organization

[1400] Data analysis

[1401] Output Generation

[1402] update

[1403] 1. Collection of User Data

[1404] The server collects user data such as user operation logs, purchase history, and posts on social networking services, as well as click logs and form input data sent from user devices (smartphones, computers, etc.).

[1405] For example, click logs and purchase histories generated when a user uses an online shopping site are sent to the server in real time. This data is collected via API and used for subsequent processing.

[1406] 2. Data organization

[1407] The server then organizes the collected user data into a consistent format. For example, voice data is converted to text using a speech recognition API, and social media data is analyzed and tagged appropriately.

[1408] As a specific example, when reviews posted by users on social networking services are collected as audio data, they are converted into text data and given tags such as "positive" or "negative."

[1409] 3. Data analysis

[1410] The server analyzes the organized data using a generative AI model, such as OpenAI's GPT-3, to extract user needs and issues.

[1411] As a specific example, the subject of data analysis is a social media post stating, "The user is interested in new outfits." In this case, the prompt would be, "Analyze the following user data and identify the user's needs and challenges. The user has posted on social media that they are interested in new outfits."

[1412] 4. Output Generation

[1413] The server generates output, such as reports and dashboards, that visualize user profiles and user experiences based on the analysis results, making it easy to understand user needs and behavioral patterns.

[1414] For example, the generated report may include product recommendations based on user needs, visually displayed using graphs and charts.

[1415] 5. Updates

[1416] The server automatically updates the generated output based on new data, ensuring that the latest user information is always reflected.

[1417] For example, as new feedback data is acquired, changes in user satisfaction are reflected in the report.

[1418] In this way, the system of the present invention can efficiently collect, organize, and analyze user data and generate specific action plans based on the results, enabling development teams to accurately understand user needs and make rapid and effective product improvements.

[1419] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1420] Program processing flow

[1421] Step 1:

[1422] The device collects user data such as user operation logs, purchase history, and posts on social networking services. This also includes click logs and form input data generated when users use online shopping sites. The device sends the collected data to a server in real time. The input is various user data, and the output is data sent to the server.

[1423] Step 2:

[1424] The server organizes the collected user data into a consistent format. For example, voice data is converted into text using a speech recognition API (e.g., Google Cloud Speech-to-Text), and social media data is analyzed using natural language processing technology and appropriately tagged. The input is the collected user data, and the output is organized text data and tagged data.

[1425] Step 3:

[1426] The server analyzes the organized data using a generative AI model. During this process, the organized text data is input as prompts into the generative AI model (e.g., OpenAI GPT-3) to extract user needs and issues. The inputs are the organized data and prompts, and the output is the analysis results.

[1427] Step 4:

[1428] The server generates output (e.g., reports, dashboards) that visualize the user profile and user experience based on the analysis results. This output visually shows user needs and problems. The input is the analysis results, and the output is the visualized output.

[1429] Step 5:

[1430] The server updates the generated output based on the new data, ensuring that it always reflects the latest user information. For example, when the latest user feedback data is obtained, a report or dashboard is updated based on it. The input is the new user data, and the output is the updated output.

[1431] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1432] The present invention relates to a user understanding promotion system that combines an emotion engine, which streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Furthermore, it utilizes an emotion engine that recognizes user emotions to provide detailed analysis including user emotion information.

[1433] Program processing overview

[1434] Data collection

[1435] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[1436] Examples:

[1437] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[1438] Data organization

[1439] The server organizes the received user data into a consistent format.

[1440] Specific behavior:

[1441] The server uses voice recognition technology to convert the audio data of the user interview into text.

[1442] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[1443] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[1444] Data analysis

[1445] The server analyzes the organized data using generative AI and an emotion engine.

[1446] Specific behavior:

[1447] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[1448] The server clusters the user data and identifies different user segments.

[1449] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[1450] Examples:

[1451] The server identifies specific feedback such as "the app's response is slow" or "new features are difficult to use" and extracts them as common problems. It then uses an emotion engine to identify user sentiment regarding these problems and calculates the percentage of negative feedback.

[1452] Output Generation

[1453] The server generates output that visualizes the user profile and user experience based on the analysis results.

[1454] Specific behavior:

[1455] The server generates reports that show the different needs and challenges of each user segment.

[1456] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[1457] The server provides these outputs in PDF format and as a web interface.

[1458] Examples:

[1459] The generated report shows the different needs and challenges of each user segment, highlighting items with particularly negative feedback, and includes sentiment information for each piece of feedback, along with visually displaying trends in user behavior and feedback content using graphs and charts.

[1460] update

[1461] The server automatically updates the output based on new data, always providing the latest user profile and experience.

[1462] Specific behavior:

[1463] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[1464] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[1465] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[1466] Examples:

[1467] When feedback data is updated, the server not only reflects changes in user satisfaction in the report, but also highlights areas where sentiment fluctuations are particularly noticeable.

[1468] In this way, by combining an emotion engine, the present invention provides detailed user understanding, including user emotional information, enabling development teams to make product improvements quickly and accurately.

[1469] The processing flow will be explained below.

[1470] Step 1: Data collection

[1471] The server receives a variety of user data, including audio data from user interviews sent from the device, application usage logs, survey responses, and social media data.

[1472] (Specific actions)

[1473] The server collects various data in real time through API endpoints.

[1474] The server checks the format of the received data and filters out any invalid or missing data.

[1475] Step 2: Data organization

[1476] The server organizes the received user data into a consistent format.

[1477] (Specific actions)

[1478] The server uses voice recognition technology to convert the audio data of the user interview into text.

[1479] The server converts the log data and feedback data into a standard format (JSON, CSV, etc.) and stores it in a database.

[1480] The server uses natural language processing (NLP) technology on the text data to extract and tag important keywords and themes.

[1481] Step 3: Emotion Recognition

[1482] The server analyzes the organized data using an emotion engine to identify the user's emotional state.

[1483] (Specific actions)

[1484] The server inputs text and voice data into an emotion engine and classifies the user's emotions as positive, negative, or neutral.

[1485] The server stores the emotion recognition results in a database, making them available for the next analysis step.

[1486] Step 4: Data analysis

[1487] The server uses generative AI to comprehensively analyze the organized data, including the results of the emotion engine, to extract user needs and issues.

[1488] (Specific actions)

[1489] The server analyzes the trends in user feedback based on the emotional state output by the emotion engine.

[1490] The server clusters user data to identify different segments and extracts common needs and challenges for each segment.

[1491] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[1492] Examples:

[1493] The server uses an emotion engine to identify that there is a lot of feedback about the app's slow response, and identifies that there is a particularly large amount of negative emotion among the feedback.

[1494] Step 5: Generate output

[1495] The server generates output that visualizes the user profile and user experience based on the analysis results.

[1496] (Specific actions)

[1497] The server generates reports that show the different needs and challenges of each user segment.

[1498] The server creates a dashboard that reflects sentiment information and data trends, displaying graphs and charts that are updated in real time.

[1499] The server provides the generated reports and dashboards in PDF format or as a web interface.

[1500] Examples:

[1501] The generated report shows the different needs and challenges of each user segment, highlights items with a high percentage of negative feedback, and includes a graph showing the emotional state of users based on the results of the emotion engine.

[1502] Step 6: Update

[1503] The server automatically updates the output based on the new data.

[1504] (Specific actions)

[1505] The server performs additional analysis on the newly collected data and re-evaluates the results of the emotion engine.

[1506] The server retrains the generative AI to continuously improve its analytical accuracy.

[1507] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[1508] Examples:

[1509] The server analyzes the new feedback data and reports fluctuations in user satisfaction, as well as highlighting areas where sentiment fluctuations are particularly pronounced.

[1510] In this way, the present invention, by combining an emotion engine, achieves a deeper understanding of users, enabling development teams to make quick and accurate product improvements.

[1511] Example 2

[1512] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1513] In many modern systems, the collection, organization, and analysis of user data are carried out separately, resulting in a lack of a consistent process. This makes it difficult to accurately understand users, and in particular, detailed analysis including emotional information is not performed. Furthermore, the inability to update collected data in real time or automatically update output hinders continuous improvement of the user experience.

[1514] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data using a generative AI model and an emotion recognition engine, means for generating output that visualizes user profiles and user experiences based on the analysis results, and means for updating the output based on new data. This enables consistent collection, organization, analysis, and real-time updating of user data. In addition, the use of the emotion recognition engine enables detailed analysis including user emotion information, enabling continuous improvement of the user experience.

[1515] "User data" refers to all information related to users, such as user behavior, feedback, usage logs, survey responses, and social media posts.

[1516] "Means of collection" refers to the processes and technologies used to receive user data from devices and store it on servers.

[1517] "Organization methods" refers to the processes and techniques used to convert collected user data into a consistent format and to cleanse, format, and store the data.

[1518] A "generative AI model" refers to an algorithm or system that uses machine learning or deep learning techniques to generate, analyze, and predict data.

[1519] An "emotion recognition engine" refers to a system that uses natural language processing and machine learning techniques to identify emotions (positive, negative, neutral, etc.) from user feedback and data.

[1520] "Means of analysis" refers to the processes and techniques used to analyze collected and organized user data using various analytical tools and techniques to extract insights and patterns.

[1521] "Means of generating output" refers to the processes and techniques used to create visual reports and dashboards based on the analysis results, and visualize user profiles and user experiences.

[1522] "Updating methods" refers to the processes and techniques used to reanalyze existing outputs based on newly collected data, ensuring that they always reflect the latest information.

[1523] The present invention relates to a user understanding promotion system that combines an emotion engine, and streamlines the user understanding process by consistently collecting, organizing, analyzing, and generating and updating output from user data. Specific embodiments of the system are described below.

[1524] Data collection

[1525] The server receives a variety of user data, such as audio data from user interviews sent from the device, application usage logs, survey responses, and social media data, and collects the necessary data.

[1526] Hardware / software used: Internet connection, database, cloud storage

[1527] Examples:

[1528] Click logs and form input data generated when a user uses the app are sent from the device to the server in real time.

[1529] Data organization

[1530] The server organizes the received user data into a consistent format.

[1531] Hardware / software used: Data cleansing tools, data format conversion tools

[1532] Examples:

[1533] The server converts the user interview audio data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text API, IBM Watson Speech to Text).

[1534] The server converts the log data and feedback data into JSON or CSV format and stores it in a database as needed.

[1535] The server uses natural language processing (NLP) technology to extract and tag important keywords and themes from the text data.

[1536] Data analysis

[1537] The server analyzes the organized data using generative AI models and emotion engines.

[1538] Hardware / software used: Machine learning platforms (e.g., TensorFlow, PyTorch), sentiment analysis tools (e.g., IBM Watson Natural Language Understanding)

[1539] Examples:

[1540] The server identifies the emotional state (positive, negative, neutral) of the user feedback through an emotion engine.

[1541] The server clusters the user data and identifies different user segments (e.g., K-means clustering).

[1542] The server performs trend analysis and extracts the major issues and requests within a specific period of time.

[1543] Example prompt: Identify user feedback such as "The app is slow to respond" or "The new features are difficult to use" and extract them as common problems. Use an emotion engine to identify user sentiment regarding these problems and calculate the percentage of negative feedback.

[1544] Output Generation

[1545] The server generates output that visualizes the user profile and user experience based on the analysis results.

[1546] Hardware / software used: Report generation tools (e.g., Tableau, Microsoft Power BI), interactive dashboard tools (e.g., D3.js, Chart.js)

[1547] Examples:

[1548] The server generates reports that show the different needs and challenges of each user segment.

[1549] The server creates a dashboard based on user data and sentiment information, displaying visual trend graphs and charts that are updated in real time.

[1550] The server provides these outputs in PDF format and via a web interface. The generated reports show the different needs and challenges of each user segment, highlighting areas that generate the most negative feedback.

[1551] update

[1552] The server updates its output based on new data, always providing the latest user profile and experience.

[1553] Hardware / software used: Cron jobs, scheduling tools, retraining platform

[1554] Examples:

[1555] The server analyzes the newly collected data and uses an emotion engine to reflect it in existing reports and dashboards.

[1556] The server continues to improve the accuracy of the analysis by retraining the generative AI.

[1557] The server automatically alerts relevant parties if there are any important changes or abnormalities.

[1558] In this way, by combining emotion engines, the present invention provides detailed user understanding including user emotion information, enabling rapid and accurate product improvements.

[1559] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1560] Step 1:

[1561] Data collection

[1562] Input: User data such as user utterances, app usage logs, survey responses, and social media posts

[1563] Output: raw data collected

[1564] Specific behavior:

[1565] 1. The user uses the app to perform various operations.

[1566] 2. The terminal records the user's operation log and input data in real time and sends them to the server.

[1567] 3. The server receives the interview audio data, survey results, and social media posts and stores them in a buffer.

[1568] Step 2:

[1569] Data organization

[1570] Input: Raw data collected

[1571] Output: A clean, uniformly formatted dataset

[1572] Specific behavior:

[1573] 1. The server converts the voice data into text data using speech recognition technology (e.g., Google Cloud Speech-to-Text API).

[1574] 2. The server uses a data cleansing tool to remove duplicates and errors from the raw data and organize the necessary information.

[1575] 3. The server converts the log data and feedback data into a unified format (JSON, CSV, etc.) and stores it in the database.

[1576] 4. The server uses natural language processing (NLP) technology to extract important keywords from the text data and tag them.

[1577] Step 3:

[1578] Data analysis

[1579] Input: A clean, uniformly formatted dataset

[1580] Output: Analysis results and insights

[1581] Specific behavior:

[1582] 1. The server analyzes the emotional state (positive, negative, neutral) of the text data using an emotion recognition engine (e.g., IBM Watson Natural Language Understanding).

[1583] 2. The server uses a generative AI model to analyze patterns in the data and segment users.

[1584] 3. The server performs trend analysis to identify major issues and user requests within a specific period of time.

[1585] 4. Example: Extract common issues from feedback such as "The app's response is slow" or "The new features are difficult to use," and use sentiment analysis to calculate the percentage of negative feedback.

[1586] Step 4:

[1587] Output Generation

[1588] Input: Analysis results and insights

[1589] Output: Visualized reports and dashboards

[1590] Specific behavior:

[1591] 1. The server generates a report based on the analysis results that shows the needs and challenges of each user segment (e.g., Tableau, Microsoft Power BI).

[1592] 2. The server creates a dashboard with trend graphs and charts that update in real time based on NLP and sentiment information (e.g. D3.js, Chart.js).

[1593] 3. The server provides the generated output in PDF format and / or via a web interface.

[1594] 4. Example: The generated report will highlight the different needs of different user segments and highlight areas with particularly negative feedback.

[1595] Step 5:

[1596] Output Updates

[1597] Input: Freshly collected raw data

[1598] Output: Updated reports and dashboards

[1599] Specific behavior:

[1600] 1. The server analyzes the newly collected data and applies it to existing outputs.

[1601] 2. The server retrains the generative AI model to continuously improve the accuracy of the analysis (e.g., TensorFlow, PyTorch).

[1602] 3. If the server detects any important changes or abnormalities, it will automatically notify relevant parties via alerts.

[1603] 4. Example: When new user feedback is collected, the server analyzes it using an emotion recognition engine and updates existing reports and dashboards accordingly.

[1604] (Application example 2)

[1605] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1606] There is a need for a means to efficiently collect, organize, and analyze diverse user behavior and feedback data in order to deepen user understanding in real time. However, to achieve this, technology is needed that can properly identify user emotions and provide fast, highly accurate analysis results. Furthermore, there is a need to improve the user experience by providing analysis results in a visually easy-to-understand format. A system that can solve these issues in an integrated manner is needed.

[1607] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1608] In this invention, the server includes means for collecting user data, means for organizing the collected user data into a consistent format, means for analyzing the organized data to extract user needs and issues, means for identifying the emotional state of user feedback using a sentiment analysis engine, means for generating an output that visualizes user profiles and user experiences based on the analysis results, and means for updating and visualizing the output in real time based on new data, thereby enabling rapid and accurate analysis of various user data and providing detailed analysis results, including user sentiment, in real time.

[1609] "User Data" refers to information generated when a user uses an application or system, including voice data, log data, feedback data, and SNS data.

[1610] "Means of collection" refers to the hardware and software functions required to efficiently obtain a variety of user data.

[1611] "Organization" refers to the ability to convert collected user data into a consistent format and make it analyzable.

[1612] "Means of analyzing and extracting user needs and issues" refers to algorithms and engines that use organized user data to determine the problems and requests users are facing.

[1613] "Means for generating output" refers to a function for visually expressing the user profile and user experience based on the analysis results.

[1614] "Means of updating" refers to the ability to automatically update existing outputs based on new data, ensuring that information is always up-to-date.

[1615] "Sentiment analysis engine" refers to technology used to identify a user's emotional state (e.g., positive, negative, neutral) from user feedback and other data.

[1616] "Visualization means" refers to the ability to display analytical results in a visual format such as graphs or charts.

[1617] As an embodiment of the present invention, a customer sentiment analysis system for a physical store is configured as follows.

[1618] Data collection

[1619] The server collects the following user data from users (customers) using their smartphones. Voice data is collected using the smartphone's microphone to gather customer feedback. Log data includes smartphone operation history, feedback data includes in-app ratings and comments, and social media data includes mentions posted by customers on social media.

[1620] Data organization

[1621] The server organizes the collected user data into a consistent format. It converts the voice data into text using speech recognition technology (e.g., the speech_recognition library). It converts the generated log and feedback data into JSON or CSV format and stores it in a database. It uses natural language processing (NLP) technology (e.g., the TextBlob library) to extract and tag important keywords and themes from the text data.

[1622] Data analysis

[1623] The server analyzes the organized data using generative AI and a sentiment engine, which identifies the emotional state of user feedback (positive, negative, neutral, etc.), clusters user data to identify different user segments, and performs trend analysis to extract key issues and requests within a specific period.

[1624] Output Generation

[1625] The server generates outputs based on the analysis results that visualize user profiles and user experiences. It generates reports showing the different needs and challenges of each user segment, creates dashboards based on user data and sentiment information, and displays visual trend graphs and charts that are updated in real time. These outputs are provided in PDF format or as a web interface.

[1626] update

[1627] The server automatically updates the output based on new data, always providing the latest user profile and user experience. It analyzes newly collected data, applies it to existing reports and dashboards using an emotion engine, and retrains the generative AI to continuously improve the accuracy of the analysis. If there are important changes or abnormalities, it automatically notifies relevant parties with alerts.

[1628] Specific examples

[1629] For example, if a customer gives voice feedback at a store, such as "The cash register is too slow," this is collected using the smartphone's microphone, converted into text data, and then judged as "negative" using the emotion engine. Social media data and other feedback data are also analyzed in real time and reflected in reports. In this way, items with a high rate of negative feedback can be identified and specific improvement measures can be implemented.

[1630] Prompt Sentence Examples

[1631] "Based on user behavioral data and feedback, use a sentiment analysis engine and generative AI model to answer the following questions:

[1632] Summarize the customer's feedback.

[1633] Determine your feelings about each piece of feedback (positive, negative, neutral).

[1634] List the most frequently cited issues over a specific period of time.

[1635] Analyze how your customers feel about each issue.

[1636] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1637] Step 1: Data collection

[1638] The server collects user data from users (customers) through their devices (smartphones). Voice feedback provided by customers is recorded using the smartphone's microphone, and other feedback data and social media data are also collected. These data are sent to the server and stored.

[1639] Input: Voice data, feedback data, social media data

[1640] Output: Raw data stored on the server

[1641] Step 2: Data organization

[1642] The server organizes the collected user data into a consistent format: voice data is converted into text using speech recognition technology, and log and feedback data is converted into JSON or CSV format.

[1643] Input: Raw data (audio data, feedback data, SNS data)

[1644] Output: Text data, JSON format data, CSV format data

[1645] Step 3: Natural Language Processing (NLP)

[1646] The server then performs natural language processing (NLP) on the organized data to extract and tag important keywords and themes, for example by using the TextBlob library to parse the text data.

[1647] Input: Organized data (text, JSON, CSV)

[1648] Output: Text data with keywords

[1649] Step 4: Sentiment analysis

[1650] The server runs the keyworded text data through a sentiment analysis engine to identify the emotional state of the user feedback (positive, negative, neutral, etc.). For example, the TextBlob library performs sentiment analysis.

[1651] Input: Text data with keywords

[1652] Output: Text data with emotion information

[1653] Step 5: Data Clustering

[1654] The server clusters user data based on the text data with emotional information, identifies different user segments, and uses a clustering algorithm to classify users by their attributes.

[1655] Input: Text data with emotion information

[1656] Output: Clustered user data

[1657] Step 6: Trend analysis

[1658] The server performs trend analysis based on the clustered user data, extracts the main issues and requests within a specific period, and identifies user trends and requests based on the analysis results.

[1659] Input: Clustered user data

[1660] Output: Trend analysis results

[1661] Step 7: Generate Output

[1662] The server generates output based on the trend analysis results, visualizing user profiles and user experiences. It creates visual reports (graphs and charts) that show the different needs and challenges of each user segment.

[1663] Input: Trend analysis results

[1664] Output: Visualized reports, dashboards

[1665] Step 8: Update the output

[1666] The server automatically updates the output based on new data, always providing the latest user profile and experience. Newly collected data is analyzed and the generative AI is retrained.

[1667] Input: New user data, existing analysis results

[1668] Output: Modernized reports, dashboards

[1669] The above steps enable the efficient collection, organization, analysis, output generation, and updating of diverse user data, making it possible to understand users in real time and provide improvement measures.

[1670] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1671] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1672] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1673] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1674] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1675] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1676] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1677] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1678] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1679] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1680] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1681] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1682] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1683] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1684] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1685] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1686] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1687] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1688] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1689] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1690] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1691] The following is further disclosed regarding the above embodiment.

[1692] (Claim 1)

[1693] the means by which user data is collected;

[1694] A means of organizing collected user data into a consistent format;

[1695] A means of analyzing the organized data to extract user needs and issues,

[1696] A means of generating output that visualizes user profiles and user experiences based on the analysis results;

[1697] means for updating said output based on new data;

[1698] A system including:

[1699] (Claim 2)

[1700] The system according to claim 1, wherein the collection of user data uses a variety of data including voice data, log data, feedback data, and SNS data.

[1701] (Claim 3)

[1702] 10. The system of claim 1, further comprising means for converting the collected voice data into text data.

[1703] (Claim 4)

[1704] 10. The system of claim 1, further comprising means for analyzing the collected text data using natural language processing techniques.

[1705] (Claim 5)

[1706] 10. The system of claim 1, further comprising means for generating a report or dashboard based on the analysis results.

[1707] (Claim 6)

[1708] 10. The system of claim 1, further comprising means for automatically updating the generated output based on newly collected data.

[1709] (Claim 7)

[1710] 10. The system of claim 1, further comprising means for alerting interested parties when a significant change or anomaly is detected.

[1711] "Example 1"

[1712] (Claim 1)

[1713] the means by which user data is collected;

[1714] A means of organizing collected user data into a consistent format;

[1715] A means of analyzing the organized data using a generative AI model to extract user needs and issues;

[1716] A means of generating output that visualizes user profiles and user experiences based on the analysis results;

[1717] means for updating said output based on new data;

[1718] A system including:

[1719] (Claim 2)

[1720] The system according to claim 1, wherein the collection of user data uses a variety of data including voice data, log data, feedback data, and SNS data.

[1721] (Claim 3)

[1722] 10. The system of claim 1, further comprising means for converting the collected voice data into text data.

[1723] "Application Example 1"

[1724] Claims

[1725] (Claim 1)

[1726] the means by which user data is collected;

[1727] A means of organizing collected user data into a consistent format;

[1728] A means of analyzing the organized data using a generative AI model to extract user needs and issues;

[1729] A means of generating output that visualizes user profiles and user experiences based on the analysis results;

[1730] means for updating said output based on new data;

[1731] A means of collecting user operation logs, purchase history, and posts on social networking services,

[1732] A method for analyzing collected data based on prompt statements and recommending products that meet user needs.

[1733] A system including:

[1734] (Claim 2)

[1735] The system according to claim 1, wherein the collection of user data uses a variety of data including voice data, log data, feedback data, and social networking service data.

[1736] (Claim 3)

[1737] 10. The system of claim 1, further comprising means for converting the collected voice data into text data.

[1738] "Example 2: Combining Emotion Engines"

[1739] (Claim 1)

[1740] the means by which user data is collected;

[1741] A means of organizing collected user data into a consistent format;

[1742] A means of analyzing the organized data using a generative AI model and an emotion recognition engine;

[1743] A means of generating output that visualizes user profiles and user experiences based on the analysis results;

[1744] means for updating said output based on new data;

[1745] A system including:

[1746] (Claim 2)

[1747] The system according to claim 1, wherein the collection of user data uses a variety of data including voice data, log data, feedback data, and SNS data.

[1748] (Claim 3)

[1749] 10. The system of claim 1, further comprising means for converting the collected voice data into text data.

[1750] "Application example 2 when combining emotion engines"

[1751] (Claim 1)

[1752] the means by which user data is collected;

[1753] A means of organizing collected user data into a consistent format;

[1754] A means of analyzing the organized data to extract user needs and issues,

[1755] A means of generating output that visualizes user profiles and user experiences based on the analysis results;

[1756] means for updating said output based on new data;

[1757] a means for utilizing a sentiment analysis engine to identify the emotional state of the user feedback;

[1758] A means to update and visualize the analysis results in real time;

[1759] A system including:

[1760] (Claim 2)

[1761] The system according to claim 1, wherein the collection of user data uses a variety of data including voice data, log data, feedback data, and SNS data.

[1762] (Claim 3)

[1763] 10. The system of claim 1, further comprising means for converting the collected voice data into text data. [Explanation of symbols]

[1764] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. the means by which user data is collected; A means of organizing collected user data into a consistent format; A means of analyzing the organized data to extract user needs and issues, A means of generating output that visualizes user profiles and user experiences based on the analysis results; means for updating said output based on new data; A system including:

2. The system according to claim 1, wherein the collection of user data uses a variety of data including voice data, log data, feedback data, and SNS data.

3. 10. The system of claim 1, further comprising means for converting the collected voice data into text data.

4. The system of claim 1 further comprising means for analyzing the collected text data using natural language processing techniques.

5. The system of claim 1 , further comprising means for generating a report or dashboard based on the analysis results.

6. 10. The system of claim 1, further comprising means for automatically updating the generated output based on newly collected data.

7. The system of claim 1 further comprising means for alerting interested parties when a significant change or anomaly is detected.

Images