System

Abstract

To provide a system for collectively performing authentication of a user, information collection, question response, feedback collection, and big data analysis.SOLUTION: The system includes a means for authenticating a user by receiving authentication information of the user and collating the authentication information with a database, a means for collecting data such as position information and use history of the user with the consent of the authenticated user, a means for analyzing the collected data and grasping characteristics of the user, a means for analyzing a question or consultation received from the user and generating an answer, a means for providing the generated answer to the user, a means for collecting and analyzing feedback from the user, a means for improving accuracy of the system based on the analyzed feedback, and a means for performing big data analysis using accumulated data and utilizing the analysis result for marketing and product development.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people lead busy lives, creating a strong demand for labor-saving and automation. Personalized support tailored to individual needs and situations is particularly important. Conventional systems struggle to perform user authentication, information collection, question handling, and big data analysis all at once, requiring manual operations and the integration of individual systems for each process. This resulted in problems such as a poor user experience and a lack of data consistency. To solve these issues, a system was needed that could automate a series of processes and provide seamless support to users. [Means for solving the problem]

[0005] The present invention provides a system that performs user authentication, information collection, question response, feedback collection, and big data analysis all at once. Specifically, it introduces a means for receiving user authentication information and authenticating the user by comparing it with a database. With the authenticated user's consent, data such as location information and usage history is collected, and this data is analyzed to understand the user's characteristics. Furthermore, it accepts questions and inquiries from users, analyzes them using AI, and generates optimal answers. The generated answers are provided to the user, and feedback from the user is collected and analyzed to improve the accuracy of the system. The system also includes a means for performing big data analysis using the collected data and utilizing it for marketing and product development. This allows for consistent, personalized support to be provided to users, significantly improving business efficiency and the user experience.

[0006] A "user" is an individual person who uses the system.

[0007] "Authentication information" is information for verifying the identity of a user, and typically includes an email address and a password.

[0008] A "database" is a system for efficiently storing and managing large amounts of data.

[0009] "Location information" is data that indicates the user's current location.

[0010] "Usage history" refers to a record of a user's past use of the system.

[0011] "Data collection" is the act of gathering information about users and their activities.

[0012] "Data analysis" is the process of processing collected data to extract information and gain a deeper understanding.

[0013] "Questions and consultations" are the information and advice that users request from the system.

[0014] "Answer" refers to the answer or advice provided by the system in response to a question or inquiry.

[0015] "Feedback" refers to the evaluation or opinion a user gives of the answers or advice provided.

[0016] A "session ID" is a unique identifier given to a user after authentication.

[0017] "Big data analytics" is a technique for analyzing large amounts of complex data to find patterns and trends.

[0018] "Marketing" refers to the strategies and activities used to effectively bring products and services to market.

[0019] "Product development" is a series of processes that involve planning, designing, prototyping, and mass-producing a new product.

[0020] A "system" is an overall structure in which multiple components and means work together to achieve a specific purpose. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[0043] System Program and Processing Overview

[0044] User Login and Authentication

[0045] User: Enters email address and password to log in to a smartphone or specialized device. Device: Sends the entered authentication information to the server. Server: Checks against the database to determine whether authentication was successful, and if authentication is successful, generates a session ID and sends it to the device.

[0046] User information collection

[0047] User: After logging in, the system asks for consent to the collection of location information and usage history. If consent is given, data collection begins. Device: After obtaining consent, the collected data is sent to the server. Server: The collected data is analyzed to understand the user's characteristics.

[0048] Processing user requests

[0049] User: Uses the messaging function to input questions or inquiries into the system. Device: The input request is sent as text data to the server. Server: An AI engine is used to analyze the text data and generate the optimal answer.

[0050] Answers and Feedback

[0051] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends the feedback to the server. Server: Analyzes the feedback and improves the accuracy of the system.

[0052] Big data analysis and its use for marketing information

[0053] Server: Aggregates and analyzes data collected from all users. Identifies user behavior patterns and trends and uses them for marketing and product development.

[0054] Specific examples

[0055] User login and authentication examples

[0056] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[0057] Examples of user information collection

[0058] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[0059] Example of user request processing

[0060] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data and the user's current health status and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[0061] Specific examples of answers and feedback

[0062] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[0063] Specific examples of big data analysis and its use in marketing information

[0064] The server analyzes the data collected from all users and finds a trend, such as "Many users go to parks on weekends." Based on this, it develops related products and formulates an effective marketing strategy.

[0065] The above is a specific embodiment for carrying out the present invention. The present invention aims to provide consistent personalized support and make users' lives more comfortable.

[0066] The processing flow will be explained below.

[0067] User login and authentication process steps

[0068] Step 1:

[0069] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[0070] Step 2:

[0071] The device sends the entered email address and password to the server.

[0072] Step 3:

[0073] The server checks the email address and password against a database to determine whether authentication was successful.

[0074] Step 4:

[0075] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[0076] Step 5:

[0077] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[0078] User information collection process steps

[0079] Step 1:

[0080] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[0081] Step 2:

[0082] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[0083] Step 3:

[0084] The device sends the collected location information and usage history to the server.

[0085] Step 4:

[0086] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[0087] Processing steps for user request processing

[0088] Step 1:

[0089] The user enters a question or request into the message input field within the app.

[0090] Step 2:

[0091] The terminal transmits the input data from the user to the server.

[0092] Step 3:

[0093] The server uses an AI engine to analyze the input data and generate an appropriate answer.

[0094] Step 4:

[0095] The server sends the generated response to the terminal.

[0096] Step 5:

[0097] The terminal notifies the user of the received answer and allows the user to view it.

[0098] Response and feedback processing steps

[0099] Step 1:

[0100] The user enters feedback on the answers provided.

[0101] Step 2:

[0102] The terminal sends the user's feedback to the server.

[0103] Step 3:

[0104] The server stores the received feedback in a database and analyzes the feedback.

[0105] Step 4:

[0106] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[0107] Processing steps for big data analysis and its use in marketing information

[0108] Step 1:

[0109] The server aggregates the data collected from all users to create a large dataset.

[0110] Step 2:

[0111] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[0112] Step 3:

[0113] The server generates insights for marketing strategies and new product development based on the analysis results.

[0114] Step 4:

[0115] The server provides the generated insights to the marketing and product development teams.

[0116] Example 1

[0117] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] In modern society, systems that provide personalized support are required to meet the diverse needs of users. However, current systems do not effectively collect, analyze, or utilize user information, resulting in inconsistent user experiences and making it difficult to improve satisfaction. It is also difficult to effectively utilize collected data for marketing and product development.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0120] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as the user's location information and usage history with the authenticated user's consent, means for analyzing the collected data and understanding the user's characteristics, means for analyzing questions and inquiries from the user using a generative AI model and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, and means for performing big data analysis using the accumulated data and utilizing it for marketing and product development. This allows for consistent, personalized support to be provided to users, improving the user experience, and making it possible to effectively utilize the data for marketing and product development.

[0121] "User authentication information" is information provided by a user to identify themselves and prove their access rights.

[0122] A "database" is a system for storing data in an organized manner and facilitating its retrieval and management.

[0123] "Location information" is data that indicates the user's current geographical location.

[0124] "Usage history" refers to data related to the operation history and behavior patterns of users when they use systems or applications.

[0125] A "collection method" is a method or device for capturing and storing specific data.

[0126] "Analytical tools" are methods and equipment used to analyze collected data and find meaning.

[0127] A "generative AI model" is an algorithm or software that uses artificial intelligence techniques to analyze data and generate answers or suggestions.

[0128] The "answer generation means" refers to a method or algorithm for creating an appropriate answer to a question or inquiry input by a user.

[0129] "Feedback" is data regarding ratings and opinions provided by users.

[0130] "Big data analysis" is a technology that analyzes large amounts of data to extract useful information.

[0131] "Marketing" refers to the activities and strategies used to promote the sale of products and services.

[0132] "Product development" is the process of designing and delivering new products and services.

[0133] "Privacy protection" refers to measures to prevent the leakage or misuse of personal information.

[0134] "Anonymization" is the process of processing data so that it cannot identify a specific individual.

[0135] A "session ID" is an identifier temporarily assigned to identify a session of a specific user.

[0136] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[0137] User Login and Authentication

[0138] To log in to a smartphone or specialized device, a user enters an email address and password. The device sends the entered authentication information to the server. The server compares the information with a database (e.g., MySQL (registered trademark), PostgreSQL) and determines whether the authentication was successful. If the authentication is successful, the server generates a session ID and sends it to the device. The user's login is confirmed when a "Login successful" message is displayed on the device.

[0139] User information collection

[0140] After logging in, the user will see a pop-up requesting consent to the collection of location information and usage history. If they agree, they click "Agree." The device will use its GPS function to obtain location information and collect usage history. The device will then send this data to the server. The server will analyze the received data and understand the user's characteristics. Specifically, the server will perform analysis to identify the locations the user frequently visits and app usage patterns.

[0141] Answering questions

[0142] Users use the messaging function to input questions or inquiries into the system. For example, they might input, "What exercise would you recommend today?" The device sends this request as text data to the server. The server then analyzes the text data using a generative AI model (e.g., BERT, GPT-3 (registered trademark)). Based on the analysis results, the server generates an appropriate answer. For example, the server might generate an answer such as, "The sun is strong today, so I recommend jogging in the evening." The server then sends the generated answer to the device and provides it to the user.

[0143] Feedback collection

[0144] The user enters feedback on the provided answer, for example, "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to improve the accuracy of the system. A specific example is that the server uses the feedback data to update the training dataset of a generative AI model.

[0145] Big data analysis and its use for marketing information

[0146] The server aggregates the data collected from all users and analyzes it using big data analysis tools (e.g., Hadoop, Spark). User behavior patterns and trends are extracted and used for marketing and product development. For example, a trend such as "many users visit the park on weekends" can be identified, and a new product promotion can be developed based on this.

[0147] The above is a specific embodiment of the present invention, which can provide consistent and personalized support to users and improve their quality of life.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] User Login and Authentication

[0150] Step 1:

[0151] A user enters an email address and password to log in to a smartphone or specialized device. This is the input for the process.

[0152] Step 2:

[0153] The terminal sends the authentication information (email address and password) entered by the user to the server. This authentication information is the input for the process.

[0154] Step 3:

[0155] The server receives the authentication information and checks it against a database (e.g., MySQL, PostgreSQL). During this process, it compares the entered authentication information with the information in the database and outputs the result of the check.

[0156] Step 4:

[0157] The server determines whether authentication is successful, and if authentication is successful, generates a session ID. The generated session ID is output.

[0158] Step 5:

[0159] The server sends the generated session ID to the terminal, which stores the session ID and displays a "Login successful" message to the user, confirming that the user has logged in.

[0160] User information collection

[0161] Step 1:

[0162] After logging in, users will be shown a pop-up requesting consent to the collection of location information and usage history. This is the input for processing.

[0163] Step 2:

[0164] If the user agrees to the data collection, they click "Agree." The consent click is the input.

[0165] Step 3:

[0166] The device uses GPS to obtain the user's current location and collects app usage history, which are the inputs for processing.

[0167] Step 4:

[0168] The device sends the collected location information and usage history to the server. This transmission becomes the input.

[0169] Step 5:

[0170] The server analyzes the received data and understands the user's characteristics, such as the locations the user frequently visits and the app usage patterns. The results of this analysis are output.

[0171] Processing user requests

[0172] Step 1:

[0173] Users use the messaging function to input questions or inquiries into the system. For example, they might input, "What exercise would you recommend today?" This is the input for the process.

[0174] Step 2:

[0175] The terminal sends the input request as text data to the server, and the sent text data becomes the input.

[0176] Step 3:

[0177] The server analyzes the received text data using a generative AI model (e.g., BERT, GPT-3). This analysis is the data processing process. For example, it generates appropriate suggestions based on the content of the input request.

[0178] Step 4:

[0179] The server generates the optimal answer based on the analysis results, such as "The sun is strong today, so I recommend jogging in the evening."

[0180] Step 5:

[0181] The server sends the generated answer to the terminal and provides it to the user, who then confirms the answer.

[0182] Answers and Feedback

[0183] Step 1:

[0184] The user enters feedback on the provided answer, for example, "This advice was helpful." This is the input for the process.

[0185] Step 2:

[0186] The device sends feedback to the server, which then becomes the input.

[0187] Step 3:

[0188] The server analyzes the received feedback and extracts from it which parts of the system need improvement. This analysis is the data processing process.

[0189] Step 4:

[0190] The server uses the analyzed feedback to update models and algorithms to improve the accuracy of the system, which is the output.

[0191] Big data analysis and its use for marketing information

[0192] Step 1:

[0193] The server aggregates the data collected from all users, and the aggregated data is the input for processing.

[0194] Step 2:

[0195] The server analyzes the aggregated data using big data analysis tools (e.g., Hadoop, Spark). This analysis is the data processing process.

[0196] Step 3:

[0197] The server extracts user behavior patterns and trends from the analysis results. For example, the output may be a trend such as "Many users visit the park on weekends."

[0198] Step 4:

[0199] The server uses the analysis results for marketing and product development, for example, to plan promotions for new products based on these trends.

[0200] These processing steps enable the system to provide consistent, personalized support to users, improving the user experience and enabling effective use in marketing and product development.

[0201] (Application example 1)

[0202] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] Conventional content delivery systems lack the ability to provide personalized content recommendations based on user preferences and viewing history, making it difficult for users to access the content they desire. Furthermore, the mechanisms for properly collecting and analyzing user feedback and reflecting it in improving the system's accuracy were also inadequate. Furthermore, collected data was not anonymized, leaving many users with concerns about privacy.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0205] In this invention, the server includes a means for recommending content based on the user's viewing history and genre of interest, a means for analyzing the user's input text and making optimal recommendations using a generative AI model, and a means for providing the user with recommendations generated using prompts. This enables the provision of optimal content tailored to the user's preferences and improves the accuracy of the system based on feedback analysis. Furthermore, privacy protection can be strengthened by anonymizing user data.

[0206] "User authentication" is the process of verifying the legitimacy of a user when the user accesses a system using authentication information such as an email address and password.

[0207] "Data collection" is the process of collecting information such as a user's location, usage history, and viewing history in order to store it in the system.

[0208] "Data analysis" is the process of analyzing collected data to understand user behavior patterns and characteristics.

[0209] "Question acceptance" is a process of accepting questions and inquiries from users as input.

[0210] "Answer generation" is the process of analyzing received questions and inquiries and providing the most appropriate answer to the user.

[0211] "Feedback collection" is the process of collecting evaluations and opinions from users and using them to improve the system.

[0212] A "viewing history" is a record of content that a user has viewed in the past.

[0213] "Content recommendation" is the process of providing content suitable for a user based on collected viewing history and genres of interest.

[0214] A "generative AI model" is a model that uses artificial intelligence to analyze user input and generate optimal answers or recommendations.

[0215] A "prompt sentence" is a sentence used to input specific instructions or questions to a generative AI model.

[0216] "Anonymization" is the process of protecting a user's personal information by transforming it so that it cannot be identified.

[0217] "Big data analysis" is the process of analyzing large amounts of data collected from a large number of users and utilizing it for marketing and product development.

[0218] A "session ID" is a unique identifier generated to identify a user after successful user authentication.

[0219] The present invention provides a system that performs user authentication, information collection, question handling, feedback collection, and big data analysis all at once to provide consistent, personalized support to users. This system is specifically implemented as follows.

[0220] User Login and Authentication

[0221] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session ID and sends it to the device. This completes user authentication.

[0222] Data collection

[0223] Authenticated users are asked to consent to the collection of their location information and viewing history. Once consent is given, the device will periodically acquire the user's location information using GPS or other functions and send it along with their viewing history to the server. The server then analyzes the collected data to understand the user's characteristics and behavioral patterns.

[0224] Question acceptance and answer generation

[0225] Users can input questions or inquiries using the device's interface. The device then sends this as text data to the server. The server then uses a generative AI model to analyze the user's input and generate optimal answers or content recommendations. For example, if a user types, "What movie should I watch tonight?", the server might generate an answer such as, "Considering the action movies you've recently watched and the reviews, I recommend 'Inception' tonight," and send it to the device.

[0226] Gathering feedback and improving system accuracy

[0227] When a user enters feedback on the answer they received, the device sends the feedback to the server, which analyzes the feedback and uses it as data to improve the accuracy of the system. This improves the accuracy of answers and content recommendations from the next time onwards.

[0228] Utilizing big data analysis and marketing information

[0229] The server performs big data analysis using the accumulated data from all users, which allows users to identify patterns and trends in their behavior and utilize this information for marketing and product development.

[0230] The specific hardware and software used include smartphones and head-mounted displays (HMDs), web frameworks such as Flask for communication with the server, SQLite for the database, and OpenAI (registered trademark) GPT-3 as the generative AI model.

[0231] Prompt Sentence Examples

[0232] When a user asks the following question:

[0233] "Please tell me what movie I should watch today."

[0234] The server generates a response as follows:

[0235] "Based on your recent action movies and reviews, I recommend Inception tonight because it combines a tight storyline with the action you enjoy."

[0236] As described above, the present invention aims to provide consistent personalized support and make users' lives more comfortable.

[0237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0238] Step 1:

[0239] User Authentication

[0240] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The entered authentication information is sent from the device to the server. The server compares the received authentication information with a database, and if authentication is successful, generates a session ID and sends it to the device. This authenticates the user.

[0241] Input: User's email address and password

[0242] Data processing: Database matching of authentication information

[0243] Output: Session ID

[0244] Step 2:

[0245] Data collection

[0246] Authenticated users are asked to consent to the collection of their location information and viewing history. If the user consents, the device will periodically acquire location information using GPS or other functions and send it along with their viewing history to the server. The server receives and stores this data, and analyzes it to understand the user's behavioral patterns and characteristics.

[0247] Input: User consent, GPS location, viewing history

[0248] Data processing: data collection and storage, analysis of behavioral patterns and characteristics

[0249] Output: Characteristic data

[0250] Step 3:

[0251] Questions accepted

[0252] The user inputs a question or request using the terminal interface. The input request is sent as text data from the terminal to the server, which then receives the text data.

[0253] Input: User's question or inquiry (text data)

[0254] Data processing: Receiving text data

[0255] Output: Unparsed text data

[0256] Step 4:

[0257] Answer generation

[0258] The server analyzes the received text data using a generative AI model. Based on the analysis results, it generates the most suitable answer and content recommendation for the user. For example, in response to the question, "What movie should I watch today?", it will recommend "Inception" based on the user's past viewing history and preferences. The generated answer is sent to the device in text format.

[0259] Input: Unparsed text data

[0260] Data processing: Text analysis, answer generation using generative AI models

[0261] Output: User-optimized answers and content recommendations (text format)

[0262] Step 5:

[0263] Providing answers to users

[0264] The terminal displays the answers received from the server to the user, allowing the user to obtain answers to their questions and content recommendations.

[0265] Input: Best answer or content recommendation from the server (text format)

[0266] Data processing: conversion to user display

[0267] Output: The displayed answer

[0268] Step 6:

[0269] Feedback collection

[0270] The user enters feedback on the answers provided. The feedback is sent from the device to the server, which receives and analyzes the feedback. This data is used to improve the accuracy of the system.

[0271] Input: User feedback (text format)

[0272] Data processing: receiving and analyzing feedback

[0273] Output: Data for accuracy improvement

[0274] Step 7:

[0275] Utilizing big data analysis and marketing information

[0276] The server aggregates data collected from all users and performs big data analysis. This allows users to identify patterns and trends in their behavior and use them in marketing and product development. For example, a pattern could be identified where "many users go to the park on weekends."

[0277] Input: Aggregated data for all users

[0278] Data processing: Big data analysis

[0279] Output: Behavioral patterns and trend information

[0280] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0281] This invention combines an emotion engine with a system that provides consistent personalized support to users to provide a more accurate and emotion-aware service. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[0282] System Program and Processing Overview

[0283] User Login and Authentication

[0284] User: Opens a smartphone or specialized device and enters an email address and password on the app's login screen. Device: The entered authentication information is sent to the server. Server: After checking the database and determining whether authentication was successful, if authentication was successful a session ID is generated and sent to the device.

[0285] User information collection

[0286] User: After logging in, the system asks for consent to the collection of location information and usage history, and if consent is given, data collection begins. Device: After obtaining consent, the system uses the GPS function to obtain current location information and periodically collects usage history and sends it to the server. Server: The received data is stored in a database and analyzed to understand the user's characteristics.

[0287] Processing user requests

[0288] User: Enters a question or inquiry into the message input field within the app. Device: Sends the input data from the user to the server. Server: Analyzes the input data using an AI engine and emotion engine, and generates the optimal answer taking emotions into account.

[0289] Emotion recognition by emotion engine

[0290] Terminal: When a user inputs a question or request for advice, the device collects the user's emotional data using methods such as voice recognition and facial expression analysis. Server: Analyzes the collected emotional data and understands the user's current emotional state. Based on the results, the device generates more appropriate answers and advice tailored to each individual user.

[0291] Answers and Feedback

[0292] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends feedback to the server. Server: Analyzes the feedback and emotion data to improve the accuracy of the system.

[0293] Big data analysis and its use for marketing information

[0294] Server: Integrates data collected from all users and performs big data analysis. Data analysis identifies user behavior patterns and trends, which are used for marketing and product development. Collected emotional data is also included to generate insights based on user emotions.

[0295] Specific examples

[0296] User login and authentication examples

[0297] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[0298] Examples of user information collection

[0299] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[0300] Example of user request processing

[0301] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data, current health status, and emotional state determined by the emotion engine, and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[0302] Specific examples of emotion recognition using emotion engines

[0303] When a user is feeling stressed, they ask, "Work hasn't been going well lately. What should I do?" The device detects the user's stress level through voice recognition and facial expression analysis, and sends that data to the server. The server then uses its emotion engine to generate advice to reduce stress, such as "How about listening to some relaxing music?", and provides it to the user.

[0304] Specific examples of answers and feedback

[0305] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[0306] Specific examples of big data analysis and its use in marketing information

[0307] The server analyzes the data collected from all users and finds a trend that "many users visit parks on weekends." Based on the emotional data, it also finds that many users seek relaxation on weekends. Based on this, related products can be developed and effective marketing strategies can be formulated.

[0308] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[0309] The processing flow will be explained below.

[0310] User login and authentication process steps

[0311] Step 1:

[0312] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[0313] Step 2:

[0314] The device sends the entered email address and password to the server.

[0315] Step 3:

[0316] The server checks the email address and password against a database to determine whether authentication was successful.

[0317] Step 4:

[0318] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[0319] Step 5:

[0320] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[0321] User information collection process steps

[0322] Step 1:

[0323] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[0324] Step 2:

[0325] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[0326] Step 3:

[0327] The device sends the collected location information and usage history to the server.

[0328] Step 4:

[0329] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[0330] Processing steps for user request processing

[0331] Step 1:

[0332] The user enters a question or request into the message input field within the app.

[0333] Step 2:

[0334] The terminal transmits the input data from the user to the server.

[0335] Step 3:

[0336] The server uses an AI engine and an emotion engine to analyze the input data and generate an appropriate answer taking emotions into account.

[0337] Step 4:

[0338] The server sends the generated response to the terminal.

[0339] Step 5:

[0340] The terminal notifies the user of the received answer and allows the user to view it.

[0341] Emotion recognition processing steps by the emotion engine

[0342] Step 1:

[0343] When the terminal inputs a user's question or inquiry, it collects emotional data using means such as voice recognition and facial expression analysis.

[0344] Step 2:

[0345] The device transmits the collected emotion data to the server.

[0346] Step 3:

[0347] The server analyzes the received emotional data and understands the user's current emotional state.

[0348] Step 4:

[0349] The server generates optimal answers and advice for the user based on the emotional data.

[0350] Response and feedback processing steps

[0351] Step 1:

[0352] The user enters feedback on the answers provided.

[0353] Step 2:

[0354] The device sends the feedback to the server.

[0355] Step 3:

[0356] The server stores the received feedback and emotional data in a database and analyzes it.

[0357] Step 4:

[0358] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[0359] Processing steps for big data analysis and its use in marketing information

[0360] Step 1:

[0361] The server aggregates the data collected from all users to create a large dataset.

[0362] Step 2:

[0363] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[0364] Step 3:

[0365] The server generates insights for marketing strategies and new product development based on the analysis results, including emotional data.

[0366] Step 4:

[0367] The server provides the generated insights to the marketing and product development teams.

[0368] Example 2

[0369] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0370] Conventional personalized support systems have difficulty in responding to user emotions, resulting in insufficient user satisfaction. In addition, they often cannot effectively analyze the collected data and utilize it for marketing and product development.

[0371] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving user authentication information and authenticating the user by comparing the authentication information with a database; means for collecting data such as the user's location information and usage history with the authenticated user's consent; means for analyzing the collected data and understanding the user's characteristics; means for accepting input of questions and inquiries from the user; means for analyzing the accepted questions and inquiries using a generative AI model and an emotion engine and generating answers that take emotions into consideration; means for providing the generated answers to the user; means for collecting and analyzing user emotion data; means for providing services adapted to the user's emotional state based on the analyzed emotion data; means for collecting and analyzing user feedback; means for improving system accuracy based on the analyzed feedback; and means for performing big data analysis using accumulated data and utilizing the results in marketing and product development. This makes it possible to provide consistent personalized support that takes users' emotions into consideration, improve user satisfaction, and generate effective marketing information.

[0372] "User Credentials" are the identification data a user uses to log into a system, typically a combination of email address and password.

[0373] "Database" means an information management system for managing and storing user authentication information and other data.

[0374] "Location information" is data indicating the user's current location obtained using GPS or other means.

[0375] "Usage history" refers to historical data of operations performed by a user within the system and functions accessed by the user.

[0376] A "generative AI model" is an artificial intelligence model used to analyze questions and inquiries from users and generate appropriate answers.

[0377] The "emotion engine" is an analytical engine that analyzes the user's emotional data and provides an appropriate response based on that emotion.

[0378] A "session ID" is an identifier temporarily assigned to a user who has been successfully authenticated, and is used to manage the continuation of the session.

[0379] "Big data analysis" is a processing method for analyzing large amounts of accumulated data and extracting useful information.

[0380] "Feedback" refers to the evaluation or opinion a user gives regarding the answers or services provided.

[0381] This invention combines an emotion engine and an AI engine into a system that provides consistent, personalized support to users. This system performs user authentication, information collection, question response, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[0382] User Login and Authentication

[0383] A user uses a smartphone or specialized device to enter their email address and password into the app's login screen. The device sends the entered authentication information to the server, which checks it against a database to determine whether the authentication was successful. If authentication is successful, the server generates a session ID and sends it to the device. For example, if a user enters "user@example.com" and "password123" and authentication is successful, the server generates a session ID "abc123def456" and sends it to the device. The user receives a "Login successful" message.

[0384] User information collection

[0385] After a user logs in, the system asks for consent to the collection of location information and usage history. If the user consents, the device will use its GPS function to obtain current location information and periodically collect usage history and send it to the server. The server stores the received data in a database and analyzes it to understand the user's characteristics. For example, if the user consents to the collection of location information, the device will obtain latitude and longitude and periodically send this information to the server. The server will analyze this information to understand the user's behavioral patterns.

[0386] Processing user requests

[0387] The user enters a question or request into the message input field within the app. For example, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which uses a generative AI model and emotion engine to analyze the input data and generate an optimal answer taking emotion into account. For example, the server generates a response such as "The weather is nice, so I recommend going for a jog" and sends it to the device.

[0388] Emotion recognition by emotion engine

[0389] When a user inputs a question or request for advice, the device collects the user's emotional data using means such as voice recognition and facial expression analysis. For example, when a user asks, "Work hasn't been going well lately. What should I do?", the device analyzes the tone of the voice and facial expression. The server analyzes the collected emotional data, understands the user's current emotional state, and generates answers and advice based on that. For example, it may make suggestions such as, "How about listening to some relaxing music?"

[0390] Answers and Feedback

[0391] The server sends the generated answer to the device, and the user confirms the answer they received. For example, the user enters feedback such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve the accuracy of the system.

[0392] Big data analysis and its use for marketing information

[0393] The server integrates data collected from all users and performs big data analysis. For example, it analyzes user behavioral patterns and emotional data to find trends such as "Many users visit parks on weekends" or "Users want to refresh themselves on weekends." Based on this, it can develop marketing strategies and products and implement effective marketing.

[0394] Specific examples

[0395] User login and authentication examples

[0396] The user enters "user@example.com" and "password123", and the terminal sends this to the server. The server checks the database and, if authentication is successful, generates a session ID "abc123def456" and sends it to the terminal. The user receives a "Login successful" message.

[0397] Examples of user information collection

[0398] When a user agrees to the collection of location information, the device acquires latitude and longitude and sends them to a server, which analyzes this data to understand the user's behavioral patterns.

[0399] Example of user request processing

[0400] The user types, "Tell me what kind of exercise I should do today," and the device sends this to the server. The server uses a generative AI model and emotion engine to generate a response such as, "The weather is nice, so I recommend jogging," and sends it to the device.

[0401] Specific examples of emotion recognition using emotion engines

[0402] The user asks, "Work hasn't been going well lately. What should I do?" and the device collects emotional data through voice recognition and facial expression analysis. The server analyzes this data and generates a response such as, "How about listening to some relaxing music?" and sends it to the device.

[0403] Specific examples of answers and feedback

[0404] The user enters feedback into the device, such as "This advice was helpful," and the device sends it to the server, which analyzes the feedback and uses it to improve the accuracy of the system.

[0405] Specific examples of big data analysis and its use in marketing information

[0406] The server analyzes the data collected from all users and finds a trend, such as "Many users visit the park on weekends." This, along with emotional data, is used for marketing strategies and product development.

[0407] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[0408] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0409] Step 1:

[0410] The user enters their email address and password into the app's login screen using a smartphone or specialized device.

[0411] Input: A user enters the email address "user@example.com" and the password "password123".

[0412] Output: The authentication information sent to the device.

[0413] Step 2:

[0414] The terminal sends the entered authentication information to the server. The transmitted data is encrypted to ensure the security of the communication.

[0415] Input: The authentication information entered by the user.

[0416] Output: The encrypted authentication information sent to the server.

[0417] Step 3:

[0418] The server compares the received authentication information with the database and determines whether the authentication was successful.

[0419] Input: Encrypted authentication information sent from the device.

[0420] Output: Authentication success / failure and, if successful, the new session ID "abc123def456".

[0421] Step 4:

[0422] The server sends the authentication result and session ID to the terminal.

[0423] Input: Authentication result and session ID.

[0424] Output: Authentication result and session ID sent to the terminal.

[0425] Step 5:

[0426] The terminal receives the message from the server and displays "Login successful" to the user.

[0427] Input: Authentication result and session ID sent by the server.

[0428] Output: User is told "Login successful" and the session ID is displayed.

[0429] Step 6:

[0430] After logging in, the user is asked to consent to the collection of location information and usage history by the system, and agrees.

[0431] Input: User consent choice (e.g. clicking the "I agree" button).

[0432] Output: Start of terminal data collection.

[0433] Step 7:

[0434] The device uses GPS to obtain the current location and also periodically records app usage history.

[0435] Input: Consent given by the user.

[0436] Output: Obtained location information (latitude and longitude) and usage history data.

[0437] Step 8:

[0438] The device sends the location information and usage history it has acquired to the server. The data is encrypted and sent securely to the server.

[0439] Input: Captured location and usage data.

[0440] Output: The encrypted data sent to the server.

[0441] Step 9:

[0442] The server stores the received data in a database and analyzes it to understand the user's characteristics.

[0443] Input: Location and usage data sent from your device.

[0444] Output: Saving to database and analysis results (user behavior patterns and characteristics).

[0445] Step 10:

[0446] The user types a question or inquiry into a message input field within the app (e.g., "What exercise should I do today?").

[0447] Input: User-generated text for questions or inquiries.

[0448] Output: Input data ready for transmission.

[0449] Step 11:

[0450] The terminal transmits the input data from the user to the server.

[0451] Input: String data from the user.

[0452] Output: The question or question data sent to the server.

[0453] Step 12:

[0454] The server analyzes the input data using a generative AI model and emotion engine, and generates the optimal answer taking emotions into account.

[0455] Input: Questions and inquiries submitted by users, as well as sentiment data.

[0456] Data processing: Analyze the content of questions and consultations and emotional state.

[0457] Output: The best answer based on the analysis results (e.g., "The weather is nice, so I recommend going for a jog").

[0458] Step 13:

[0459] The server generates a response and sends it to the terminal.

[0460] Input: The generated answer data.

[0461] Output: The response data sent to the device.

[0462] Step 14:

[0463] The terminal receives the response from the server and displays it to the user.

[0464] Input: The response data sent from the server.

[0465] Output: Display of the answer to the user.

[0466] Step 15:

[0467] When the terminal inputs a user's question or inquiry, it collects the user's emotional data using means such as voice recognition and facial expression analysis.

[0468] Input: User's voice and facial expression data.

[0469] Output: Collected emotion data.

[0470] Step 16:

[0471] The device transmits the collected emotion data to a server.

[0472] Input: Collected emotion data.

[0473] Output: Emotion data sent to the server.

[0474] Step 17:

[0475] The server analyzes the collected emotional data to understand the user's current emotional state.

[0476] Input: Emotion data sent from the device.

[0477] Data processing: Analysis of emotion data.

[0478] Output: Analysis results about the user's emotional state.

[0479] Step 18:

[0480] The user reviews the answers provided and enters feedback into the device (e.g., "This advice was helpful").

[0481] Input: User-entered text feedback.

[0482] Output: Input data ready for transmission.

[0483] Step 19:

[0484] The device sends the feedback to the server.

[0485] Input: Feedback data entered by the user.

[0486] Output: Feedback data sent to the server.

[0487] Step 20:

[0488] The server analyzes the feedback it receives and uses it to improve the accuracy of the system.

[0489] Input: Feedback data sent from the device.

[0490] Data processing: Analysis of feedback data.

[0491] Output: Analysis results for improving system accuracy.

[0492] Step 21:

[0493] The server integrates the data collected from all users and performs big data analysis.

[0494] Input: All accumulated user data.

[0495] Data processing: Data analysis using big data analysis software.

[0496] Output: Analysis of user behavior patterns and trends.

[0497] Step 22:

[0498] The server uses the insights gained from the analysis results to inform marketing strategies and product development.

[0499] Input: Results of big data analysis.

[0500] Output: Marketing strategy and product development plan.

[0501] The system obtained in this way is capable of providing consistent personalized support that takes into account the user's emotions, contributing to improved user satisfaction and the generation of effective marketing information.

[0502] (Application example 2)

[0503] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0504] Conventional systems provide personalized support without considering the user's emotional state, resulting in a limited user experience and the inability to provide optimal advice or product recommendations based on emotions. Furthermore, there is a lack of analysis using user feedback and big data, which can make it difficult to improve the accuracy of the system or to formulate marketing strategies.

[0505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0506] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as location information and usage history with the user's consent, means for analyzing the collected data and understanding the user's characteristics, means for accepting questions and inquiries from the user, means for analyzing the accepted questions and inquiries and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, means for performing big data analysis using the accumulated data and utilizing it for marketing and product development, and means for analyzing the user's emotional state using an emotion engine and suggesting optimal products based on the user's emotions. This enables highly accurate personalized support that takes user emotions into consideration.

[0507] definition statement

[0508] "User Credentials" are information used to verify a user's identity, typically including an email address and password.

[0509] A "database" is a system for storing and managing data in an organized manner. It stores user authentication information and collected data.

[0510] "User consent" is the act of a user permitting a system to collect and use their data. This is important from the perspective of protecting user privacy.

[0511] "Location information" is data that indicates a user's current location. It is often collected using GPS functionality.

[0512] "Usage history" refers to behavioral data when a user uses a system or application, including the pages visited and the duration of use.

[0513] "Question or consultation input" refers to a question or message requesting advice that a user inputs into the system.

[0514] "Generating an answer" means that the system creates an appropriate answer to a question or inquiry from a user.

[0515] "Feedback" refers to the evaluation or opinion a user gives in response to a received answer.

[0516] "Big data analytics" is an analytical technique that integrates large amounts of data and finds patterns and trends.

[0517] "Marketing and product development" refers to the activities of planning and developing new products and services based on collected data.

[0518] An "emotion engine" is a technology that analyzes a user's emotional state and responds optimally based on that.

[0519] A "session ID" is a temporary identifier that is generated when a user logs into a system. It is provided to the user after successful authentication.

[0520] "Anonymization" is the process of removing or obscuring personally identifiable information from collected data in order to protect user privacy.

[0521] MODE FOR CARRYING OUT THE INVENTION

[0522] The present invention includes an emotion-aware virtual shopping assistant system. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing consistent personalized services that take user emotions into consideration. Specific implementation methods for this system are described in detail below.

[0523] Hardware and software used

[0524] 1. Hardware:

[0525] Smartphone

[0526] Specialized devices (e.g. tablets)

[0527] 2. Software:

[0528] requests (Python HTTP request library)

[0529] Server-side API (e.g. Django, Flask)

[0530] Emotion engine (e.g., Microsoft® Azure® Emotion API)

[0531] Big data analysis engines (e.g., Apache (registered trademark) Hadoop, Apache Spark)

[0532] Program processing

[0533] User Login and Authentication

[0534] The server receives the email address and password entered by the user on a smartphone or specialized device and performs authentication by comparing them with a database. If authentication is successful, the server generates a session ID and sends it to the device. Through this process, user authentication is established.

[0535] User information collection

[0536] With the consent of the authenticated user, the server uses the device's GPS function to collect current location information and transmits usage history to the server. The collected data is analyzed using a big data analysis engine on the server side to understand the user's usage patterns and characteristics.

[0537] Handling user inquiries and enquiries

[0538] The device sends a message from the user to the server, which analyzes the message, uses an emotion engine to understand the user's emotional state, and generates an optimal answer. The generated answer is sent to the device and provided to the user.

[0539] Feedback collection and analysis

[0540] When a user enters feedback on a given answer, the device sends the feedback to the server, which analyzes the collected feedback and uses it to improve the accuracy of the system.

[0541] Big data analysis and its use for marketing information

[0542] The collected data is integrated on a server and analyzed by a big data analysis engine. The analysis results are used in marketing and product development, and insights based on user sentiment are generated.

[0543] Specific examples

[0544] User Questions

[0545] When a user types, "I've been feeling tired from work lately, and I want a product that helps me relax," the device sends this request to the server. The server uses an emotion engine to detect "fatigue" and "stress" and suggests products that will help you relax (e.g., an aroma diffuser).

[0546] Example prompt sentence:

[0547] Based on the following specifications, please generate a program to create a virtual shopping assistant system that combines an emotion engine.

[0548] The user enters their email address and password on the login screen.

[0549] Authentication information is sent to the server, and a session ID is generated upon successful authentication.

[0550] After logging in, location and usage data is collected

[0551] Recognizes emotions from user input and suggests optimal products

[0552] Use the Python language to send HTTP requests using the requests library.

[0553] Assumes Django or Flask as the server-side API

[0554] Examples:

[0555] If a user types, "I've been tired from work lately and I want a product that helps me relax," the emotion engine will detect "fatigue" and "stress" and suggest products that will help you relax (e.g., an aroma diffuser).

[0556] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0557] Specific explanation of processing steps

[0558] Step 1:

[0559] User Login and Authentication

[0560] Input: The user enters their email address and password into their smartphone.

[0561] Specific operation: The device sends the entered authentication information to the server. The server checks the information against a database, determines whether the authentication was successful, and if the authentication was successful, generates a session ID and sends it to the device.

[0562] Output: If authentication is successful, the server generates a session ID and the user receives a "Login successful" message.

[0563] Step 2:

[0564] User information collection

[0565] Input: An authenticated user consents to the collection of location and usage information.

[0566] Specific operation: The device periodically collects its current location information using the GPS function and sends it along with its usage history to the server, which then stores the received data in a database.

[0567] Output: The server periodically receives location information and usage history and stores the data in a database.

[0568] Step 3:

[0569] Input of questions and inquiries

[0570] Input: The user types a question or request into the app's message input field.

[0571] How it works: The device sends the user's input to the server, which then analyzes it using an AI engine and emotion engine.

[0572] Output: The server is ready to understand the question or request as a result of the analysis.

[0573] Step 4:

[0574] Analyzing input data and generating answers

[0575] Input: The server receives input such as questions and inquiries, as well as emotion data.

[0576] How it works: The server's emotion engine analyzes the user's emotional state, and the AI ​​engine generates the optimal answer, customizing the answer using past data and the user's current emotional state.

[0577] Output: The server sends the generated answer to the user's device.

[0578] Step 5:

[0579] Providing answers and collecting feedback

[0580] Input: The user receives and confirms the provided answer on the terminal.

[0581] Specific operation: The device displays the answer to the user, and the user enters feedback on the answer. The feedback is sent from the device to the server.

[0582] Output: The server stores the received feedback as analysis results to help improve accuracy later.

[0583] Step 6:

[0584] Big data analysis and its use for marketing information

[0585] Input: The server uses location information, usage history, emotional data, and feedback data collected from all users.

[0586] How it works: The server's big data analysis engine analyzes this data to discover user behavior patterns and trends, and generates information based on the insights gained to be used in marketing and product development.

[0587] Output: The server outputs trend analysis results and insights for marketing strategies.

[0588] Step 7:

[0589] Continuously updating the emotion engine and improving system accuracy

[0590] Input: User feedback and parsed data.

[0591] Specific operation: The server analyzes the feedback and collected emotion data and makes updates to continuously improve the accuracy of the emotion engine and the entire system.

[0592] Output: The server implements a new version of the emotion engine with improved accuracy, improving the overall system performance.

[0593] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0594] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0595] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0596] [Second embodiment]

[0597] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0598] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0599] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0600] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0601] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0602] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0603] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0604] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0605] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0606] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0607] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0608] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0609] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[0610] System Program and Processing Overview

[0611] User Login and Authentication

[0612] User: Enters email address and password to log in to a smartphone or specialized device. Device: Sends the entered authentication information to the server. Server: Checks against the database to determine whether authentication was successful, and if authentication is successful, generates a session ID and sends it to the device.

[0613] User information collection

[0614] User: After logging in, the system asks for consent to the collection of location information and usage history. If consent is given, data collection begins. Device: After obtaining consent, the collected data is sent to the server. Server: The collected data is analyzed to understand the user's characteristics.

[0615] Processing user requests

[0616] User: Uses the messaging function to input questions or inquiries into the system. Device: The input request is sent as text data to the server. Server: An AI engine is used to analyze the text data and generate the optimal answer.

[0617] Answers and Feedback

[0618] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends the feedback to the server. Server: Analyzes the feedback and improves the accuracy of the system.

[0619] Big data analysis and its use for marketing information

[0620] Server: Aggregates and analyzes data collected from all users. Identifies user behavior patterns and trends and uses them for marketing and product development.

[0621] Specific examples

[0622] User login and authentication examples

[0623] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[0624] Examples of user information collection

[0625] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[0626] Example of user request processing

[0627] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data and the user's current health status and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[0628] Specific examples of answers and feedback

[0629] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[0630] Specific examples of big data analysis and its use in marketing information

[0631] The server analyzes the data collected from all users and finds a trend, such as "Many users go to parks on weekends." Based on this, it develops related products and formulates an effective marketing strategy.

[0632] The above is a specific embodiment for carrying out the present invention. The present invention aims to provide consistent personalized support and make users' lives more comfortable.

[0633] The processing flow will be explained below.

[0634] User login and authentication process steps

[0635] Step 1:

[0636] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[0637] Step 2:

[0638] The device sends the entered email address and password to the server.

[0639] Step 3:

[0640] The server checks the email address and password against a database to determine whether authentication was successful.

[0641] Step 4:

[0642] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[0643] Step 5:

[0644] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[0645] User information collection process steps

[0646] Step 1:

[0647] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[0648] Step 2:

[0649] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[0650] Step 3:

[0651] The device sends the collected location information and usage history to the server.

[0652] Step 4:

[0653] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[0654] Processing steps for user request processing

[0655] Step 1:

[0656] The user enters a question or request into the message input field within the app.

[0657] Step 2:

[0658] The terminal transmits the input data from the user to the server.

[0659] Step 3:

[0660] The server uses an AI engine to analyze the input data and generate an appropriate answer.

[0661] Step 4:

[0662] The server sends the generated response to the terminal.

[0663] Step 5:

[0664] The terminal notifies the user of the received answer and allows the user to view it.

[0665] Response and feedback processing steps

[0666] Step 1:

[0667] The user enters feedback on the answers provided.

[0668] Step 2:

[0669] The terminal sends the user's feedback to the server.

[0670] Step 3:

[0671] The server stores the received feedback in a database and analyzes the feedback.

[0672] Step 4:

[0673] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[0674] Processing steps for big data analysis and its use in marketing information

[0675] Step 1:

[0676] The server aggregates the data collected from all users to create a large dataset.

[0677] Step 2:

[0678] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[0679] Step 3:

[0680] The server generates insights for marketing strategies and new product development based on the analysis results.

[0681] Step 4:

[0682] The server provides the generated insights to the marketing and product development teams.

[0683] Example 1

[0684] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] In modern society, systems that provide personalized support are required to meet the diverse needs of users. However, current systems do not effectively collect, analyze, or utilize user information, resulting in inconsistent user experiences and making it difficult to improve satisfaction. It is also difficult to effectively utilize collected data for marketing and product development.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0687] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as the user's location information and usage history with the authenticated user's consent, means for analyzing the collected data and understanding the user's characteristics, means for analyzing questions and inquiries from the user using a generative AI model and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, and means for performing big data analysis using the accumulated data and utilizing it for marketing and product development. This allows for consistent, personalized support to be provided to users, improving the user experience, and making it possible to effectively utilize the data for marketing and product development.

[0688] "User authentication information" is information provided by a user to identify themselves and prove their access rights.

[0689] A "database" is a system for storing data in an organized manner and facilitating its retrieval and management.

[0690] "Location information" is data that indicates the user's current geographical location.

[0691] "Usage history" refers to data related to the operation history and behavior patterns of users when they use systems or applications.

[0692] A "collection method" is a method or device for capturing and storing specific data.

[0693] "Analytical tools" are methods and equipment used to analyze collected data and find meaning.

[0694] A "generative AI model" is an algorithm or software that uses artificial intelligence techniques to analyze data and generate answers or suggestions.

[0695] The "answer generation means" refers to a method or algorithm for creating an appropriate answer to a question or inquiry input by a user.

[0696] "Feedback" is data regarding ratings and opinions provided by users.

[0697] "Big data analysis" is a technology that analyzes large amounts of data to extract useful information.

[0698] "Marketing" refers to the activities and strategies used to promote the sale of products and services.

[0699] "Product development" is the process of designing and delivering new products and services.

[0700] "Privacy protection" refers to measures to prevent the leakage or misuse of personal information.

[0701] "Anonymization" is the process of processing data so that it cannot identify a specific individual.

[0702] A "session ID" is an identifier temporarily assigned to identify a session of a specific user.

[0703] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[0704] User Login and Authentication

[0705] To log in to a smartphone or specialized device, a user enters an email address and password. The device sends the entered authentication information to the server. The server compares it with a database (e.g., MySQL, PostgreSQL) and determines whether the authentication was successful. If the authentication is successful, the server generates a session ID and sends it to the device. The user's login is confirmed when a "Login successful" message is displayed on the device.

[0706] User information collection

[0707] After logging in, the user will see a pop-up requesting consent to the collection of location information and usage history. If they agree, they click "Agree." The device will use its GPS function to obtain location information and collect usage history. The device will then send this data to the server. The server will analyze the received data and understand the user's characteristics. Specifically, the server will perform analysis to identify the locations the user frequently visits and app usage patterns.

[0708] Answering questions

[0709] Users use the messaging function to input questions or inquiries into the system. For example, they might type, "What exercise would you recommend today?" The device sends this request as text data to the server. The server then analyzes the text data using a generative AI model (e.g., BERT, GPT-3). Based on the analysis results, the server generates an appropriate answer. For example, the server might generate an answer such as, "The sun is strong today, so I recommend jogging in the evening." The server then sends the generated answer to the device and provides it to the user.

[0710] Feedback collection

[0711] The user enters feedback on the provided answer, for example, "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to improve the accuracy of the system. A specific example is that the server uses the feedback data to update the training dataset of a generative AI model.

[0712] Big data analysis and its use for marketing information

[0713] The server aggregates the data collected from all users and analyzes it using big data analysis tools (e.g., Hadoop, Spark). User behavior patterns and trends are extracted and used for marketing and product development. For example, a trend such as "many users visit the park on weekends" can be identified, and a new product promotion can be developed based on this.

[0714] The above is a specific embodiment of the present invention, which can provide consistent and personalized support to users and improve their quality of life.

[0715] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0716] User Login and Authentication

[0717] Step 1:

[0718] A user enters an email address and password to log in to a smartphone or specialized device. This is the input for the process.

[0719] Step 2:

[0720] The terminal sends the authentication information (email address and password) entered by the user to the server. This authentication information is the input for the process.

[0721] Step 3:

[0722] The server receives the authentication information and checks it against a database (e.g., MySQL, PostgreSQL). During this process, it compares the entered authentication information with the information in the database and outputs the result of the check.

[0723] Step 4:

[0724] The server determines whether authentication is successful, and if authentication is successful, generates a session ID. The generated session ID is output.

[0725] Step 5:

[0726] The server sends the generated session ID to the terminal, which stores the session ID and displays a "Login successful" message to the user, confirming that the user has logged in.

[0727] User information collection

[0728] Step 1:

[0729] After logging in, users will be shown a pop-up requesting consent to the collection of location information and usage history. This is the input for processing.

[0730] Step 2:

[0731] If the user agrees to the data collection, they click "Agree." The consent click is the input.

[0732] Step 3:

[0733] The device uses GPS to obtain the user's current location and collects app usage history, which are the inputs for processing.

[0734] Step 4:

[0735] The device sends the collected location information and usage history to the server. This transmission becomes the input.

[0736] Step 5:

[0737] The server analyzes the received data and understands the user's characteristics, such as the locations the user frequently visits and the app usage patterns. The results of this analysis are output.

[0738] Processing user requests

[0739] Step 1:

[0740] Users use the messaging function to input questions or inquiries into the system. For example, they might input, "What exercise would you recommend today?" This is the input for the process.

[0741] Step 2:

[0742] The terminal sends the input request as text data to the server, and the sent text data becomes the input.

[0743] Step 3:

[0744] The server analyzes the received text data using a generative AI model (e.g., BERT, GPT-3). This analysis is the data processing process. For example, it generates appropriate suggestions based on the content of the input request.

[0745] Step 4:

[0746] The server generates the optimal answer based on the analysis results, such as "The sun is strong today, so I recommend jogging in the evening."

[0747] Step 5:

[0748] The server sends the generated answer to the terminal and provides it to the user, who then confirms the answer.

[0749] Answers and Feedback

[0750] Step 1:

[0751] The user enters feedback on the provided answer, for example, "This advice was helpful." This is the input for the process.

[0752] Step 2:

[0753] The device sends feedback to the server, which then becomes the input.

[0754] Step 3:

[0755] The server analyzes the received feedback and extracts from it which parts of the system need improvement. This analysis is the data processing process.

[0756] Step 4:

[0757] The server uses the analyzed feedback to update models and algorithms to improve the accuracy of the system, which is the output.

[0758] Big data analysis and its use for marketing information

[0759] Step 1:

[0760] The server aggregates the data collected from all users, and the aggregated data is the input for processing.

[0761] Step 2:

[0762] The server analyzes the aggregated data using big data analysis tools (e.g., Hadoop, Spark). This analysis is the data processing process.

[0763] Step 3:

[0764] The server extracts user behavior patterns and trends from the analysis results. For example, the output may be a trend such as "Many users visit the park on weekends."

[0765] Step 4:

[0766] The server uses the analysis results for marketing and product development, for example, to plan promotions for new products based on these trends.

[0767] These processing steps enable the system to provide consistent, personalized support to users, improving the user experience and enabling effective use in marketing and product development.

[0768] (Application example 1)

[0769] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0770] Conventional content delivery systems lack the ability to provide personalized content recommendations based on user preferences and viewing history, making it difficult for users to access the content they desire. Furthermore, the mechanisms for properly collecting and analyzing user feedback and reflecting it in improving the system's accuracy were also inadequate. Furthermore, collected data was not anonymized, leaving many users with concerns about privacy.

[0771] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0772] In this invention, the server includes a means for recommending content based on the user's viewing history and genre of interest, a means for analyzing the user's input text and making optimal recommendations using a generative AI model, and a means for providing the user with recommendations generated using prompts. This enables the provision of optimal content tailored to the user's preferences and improves the accuracy of the system based on feedback analysis. Furthermore, privacy protection can be strengthened by anonymizing user data.

[0773] "User authentication" is the process of verifying the legitimacy of a user when the user accesses a system using authentication information such as an email address and password.

[0774] "Data collection" is the process of collecting information such as a user's location, usage history, and viewing history in order to store it in the system.

[0775] "Data analysis" is the process of analyzing collected data to understand user behavior patterns and characteristics.

[0776] "Question acceptance" is a process of accepting questions and inquiries from users as input.

[0777] "Answer generation" is the process of analyzing received questions and inquiries and providing the most appropriate answer to the user.

[0778] "Feedback collection" is the process of collecting evaluations and opinions from users and using them to improve the system.

[0779] A "viewing history" is a record of content that a user has viewed in the past.

[0780] "Content recommendation" is the process of providing content suitable for a user based on collected viewing history and genres of interest.

[0781] A "generative AI model" is a model that uses artificial intelligence to analyze user input and generate optimal answers or recommendations.

[0782] A "prompt sentence" is a sentence used to input specific instructions or questions to a generative AI model.

[0783] "Anonymization" is the process of protecting a user's personal information by transforming it so that it cannot be identified.

[0784] "Big data analysis" is the process of analyzing large amounts of data collected from a large number of users and utilizing it for marketing and product development.

[0785] A "session ID" is a unique identifier generated to identify a user after successful user authentication.

[0786] The present invention provides a system that performs user authentication, information collection, question handling, feedback collection, and big data analysis all at once to provide consistent, personalized support to users. This system is specifically implemented as follows.

[0787] User Login and Authentication

[0788] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session ID and sends it to the device. This completes user authentication.

[0789] Data collection

[0790] Authenticated users are asked to consent to the collection of their location information and viewing history. Once consent is given, the device will periodically acquire the user's location information using GPS or other functions and send it along with their viewing history to the server. The server then analyzes the collected data to understand the user's characteristics and behavioral patterns.

[0791] Question acceptance and answer generation

[0792] Users can input questions or inquiries using the device's interface. The device then sends this as text data to the server. The server then uses a generative AI model to analyze the user's input and generate optimal answers or content recommendations. For example, if a user types, "What movie should I watch tonight?", the server might generate an answer such as, "Considering the action movies you've recently watched and the reviews, I recommend 'Inception' tonight," and send it to the device.

[0793] Gathering feedback and improving system accuracy

[0794] When a user enters feedback on the answer they received, the device sends the feedback to the server, which analyzes the feedback and uses it as data to improve the accuracy of the system. This improves the accuracy of answers and content recommendations from the next time onwards.

[0795] Utilizing big data analysis and marketing information

[0796] The server performs big data analysis using the accumulated data from all users, which allows users to identify patterns and trends in their behavior and utilize this information for marketing and product development.

[0797] The specific hardware and software used include smartphones and head-mounted displays (HMDs), web frameworks such as Flask for communication with the server, SQLite for the database, and OpenAI GPT-3 as the generative AI model.

[0798] Prompt Sentence Examples

[0799] When a user asks the following question:

[0800] "Please tell me what movie I should watch today."

[0801] The server generates a response as follows:

[0802] "Based on your recent action movies and reviews, I recommend Inception tonight because it combines a tight storyline with the action you enjoy."

[0803] As described above, the present invention aims to provide consistent personalized support and make users' lives more comfortable.

[0804] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0805] Step 1:

[0806] User Authentication

[0807] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The entered authentication information is sent from the device to the server. The server compares the received authentication information with a database, and if authentication is successful, generates a session ID and sends it to the device. This authenticates the user.

[0808] Input: User's email address and password

[0809] Data processing: Database matching of authentication information

[0810] Output: Session ID

[0811] Step 2:

[0812] Data collection

[0813] Authenticated users are asked to consent to the collection of their location information and viewing history. If the user consents, the device will periodically acquire location information using GPS or other functions and send it along with their viewing history to the server. The server receives and stores this data, and analyzes it to understand the user's behavioral patterns and characteristics.

[0814] Input: User consent, GPS location, viewing history

[0815] Data processing: data collection and storage, analysis of behavioral patterns and characteristics

[0816] Output: Characteristic data

[0817] Step 3:

[0818] Questions accepted

[0819] The user inputs a question or request using the terminal interface. The input request is sent as text data from the terminal to the server, which then receives the text data.

[0820] Input: User's question or inquiry (text data)

[0821] Data processing: Receiving text data

[0822] Output: Unparsed text data

[0823] Step 4:

[0824] Answer generation

[0825] The server analyzes the received text data using a generative AI model. Based on the analysis results, it generates the most suitable answer and content recommendation for the user. For example, in response to the question, "What movie should I watch today?", it will recommend "Inception" based on the user's past viewing history and preferences. The generated answer is sent to the device in text format.

[0826] Input: Unparsed text data

[0827] Data processing: Text analysis, answer generation using generative AI models

[0828] Output: User-optimized answers and content recommendations (text format)

[0829] Step 5:

[0830] Providing answers to users

[0831] The terminal displays the answers received from the server to the user, allowing the user to obtain answers to their questions and content recommendations.

[0832] Input: Best answer or content recommendation from the server (text format)

[0833] Data processing: conversion to user display

[0834] Output: The displayed answer

[0835] Step 6:

[0836] Feedback collection

[0837] The user enters feedback on the answers provided. The feedback is sent from the device to the server, which receives and analyzes the feedback. This data is used to improve the accuracy of the system.

[0838] Input: User feedback (text format)

[0839] Data processing: receiving and analyzing feedback

[0840] Output: Data for accuracy improvement

[0841] Step 7:

[0842] Utilizing big data analysis and marketing information

[0843] The server aggregates data collected from all users and performs big data analysis. This allows users to identify patterns and trends in their behavior and use them in marketing and product development. For example, a pattern could be identified where "many users go to the park on weekends."

[0844] Input: Aggregated data for all users

[0845] Data processing: Big data analysis

[0846] Output: Behavioral patterns and trend information

[0847] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0848] This invention combines an emotion engine with a system that provides consistent personalized support to users to provide a more accurate and emotion-aware service. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[0849] System Program and Processing Overview

[0850] User Login and Authentication

[0851] User: Opens a smartphone or specialized device and enters an email address and password on the app's login screen. Device: The entered authentication information is sent to the server. Server: After checking the database and determining whether authentication was successful, if authentication was successful a session ID is generated and sent to the device.

[0852] User information collection

[0853] User: After logging in, the system asks for consent to the collection of location information and usage history, and if consent is given, data collection begins. Device: After obtaining consent, the system uses the GPS function to obtain current location information and periodically collects usage history and sends it to the server. Server: The received data is stored in a database and analyzed to understand the user's characteristics.

[0854] Processing user requests

[0855] User: Enters a question or inquiry into the message input field within the app. Device: Sends the input data from the user to the server. Server: Analyzes the input data using an AI engine and emotion engine, and generates the optimal answer taking emotions into account.

[0856] Emotion recognition by emotion engine

[0857] Terminal: When a user inputs a question or request for advice, the device collects the user's emotional data using methods such as voice recognition and facial expression analysis. Server: Analyzes the collected emotional data and understands the user's current emotional state. Based on the results, the device generates more appropriate answers and advice tailored to each individual user.

[0858] Answers and Feedback

[0859] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends feedback to the server. Server: Analyzes the feedback and emotion data to improve the accuracy of the system.

[0860] Big data analysis and its use for marketing information

[0861] Server: Integrates data collected from all users and performs big data analysis. Data analysis identifies user behavior patterns and trends, which are used for marketing and product development. Collected emotional data is also included to generate insights based on user emotions.

[0862] Specific examples

[0863] User login and authentication examples

[0864] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[0865] Examples of user information collection

[0866] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[0867] Example of user request processing

[0868] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data, current health status, and emotional state determined by the emotion engine, and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[0869] Specific examples of emotion recognition using emotion engines

[0870] When a user is feeling stressed, they ask, "Work hasn't been going well lately. What should I do?" The device detects the user's stress level through voice recognition and facial expression analysis, and sends that data to the server. The server then uses its emotion engine to generate advice to reduce stress, such as "How about listening to some relaxing music?", and provides it to the user.

[0871] Specific examples of answers and feedback

[0872] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[0873] Specific examples of big data analysis and its use in marketing information

[0874] The server analyzes the data collected from all users and finds a trend that "many users visit parks on weekends." Based on the emotional data, it also finds that many users seek relaxation on weekends. Based on this, related products can be developed and effective marketing strategies can be formulated.

[0875] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[0876] The processing flow will be explained below.

[0877] User login and authentication process steps

[0878] Step 1:

[0879] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[0880] Step 2:

[0881] The device sends the entered email address and password to the server.

[0882] Step 3:

[0883] The server checks the email address and password against a database to determine whether authentication was successful.

[0884] Step 4:

[0885] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[0886] Step 5:

[0887] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[0888] User information collection process steps

[0889] Step 1:

[0890] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[0891] Step 2:

[0892] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[0893] Step 3:

[0894] The device sends the collected location information and usage history to the server.

[0895] Step 4:

[0896] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[0897] Processing steps for user request processing

[0898] Step 1:

[0899] The user enters a question or request into the message input field within the app.

[0900] Step 2:

[0901] The terminal transmits the input data from the user to the server.

[0902] Step 3:

[0903] The server uses an AI engine and an emotion engine to analyze the input data and generate an appropriate answer taking emotions into account.

[0904] Step 4:

[0905] The server sends the generated response to the terminal.

[0906] Step 5:

[0907] The terminal notifies the user of the received answer and allows the user to view it.

[0908] Emotion recognition processing steps by the emotion engine

[0909] Step 1:

[0910] When the terminal inputs a user's question or inquiry, it collects emotional data using means such as voice recognition and facial expression analysis.

[0911] Step 2:

[0912] The device transmits the collected emotion data to the server.

[0913] Step 3:

[0914] The server analyzes the received emotional data and understands the user's current emotional state.

[0915] Step 4:

[0916] The server generates optimal answers and advice for the user based on the emotional data.

[0917] Response and feedback processing steps

[0918] Step 1:

[0919] The user enters feedback on the answers provided.

[0920] Step 2:

[0921] The device sends the feedback to the server.

[0922] Step 3:

[0923] The server stores the received feedback and emotional data in a database and analyzes it.

[0924] Step 4:

[0925] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[0926] Processing steps for big data analysis and its use in marketing information

[0927] Step 1:

[0928] The server aggregates the data collected from all users to create a large dataset.

[0929] Step 2:

[0930] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[0931] Step 3:

[0932] The server generates insights for marketing strategies and new product development based on the analysis results, including emotional data.

[0933] Step 4:

[0934] The server provides the generated insights to the marketing and product development teams.

[0935] Example 2

[0936] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0937] Conventional personalized support systems have difficulty in responding to user emotions, resulting in insufficient user satisfaction. In addition, they often cannot effectively analyze the collected data and utilize it for marketing and product development.

[0938] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving user authentication information and authenticating the user by comparing the authentication information with a database; means for collecting data such as the user's location information and usage history with the authenticated user's consent; means for analyzing the collected data and understanding the user's characteristics; means for accepting input of questions and inquiries from the user; means for analyzing the accepted questions and inquiries using a generative AI model and an emotion engine and generating answers that take emotions into consideration; means for providing the generated answers to the user; means for collecting and analyzing user emotion data; means for providing services adapted to the user's emotional state based on the analyzed emotion data; means for collecting and analyzing user feedback; means for improving system accuracy based on the analyzed feedback; and means for performing big data analysis using accumulated data and utilizing the results in marketing and product development. This makes it possible to provide consistent personalized support that takes users' emotions into consideration, improve user satisfaction, and generate effective marketing information.

[0939] "User Credentials" are the identification data a user uses to log into a system, typically a combination of email address and password.

[0940] "Database" means an information management system for managing and storing user authentication information and other data.

[0941] "Location information" is data indicating the user's current location obtained using GPS or other means.

[0942] "Usage history" refers to historical data of operations performed by a user within the system and functions accessed by the user.

[0943] A "generative AI model" is an artificial intelligence model used to analyze questions and inquiries from users and generate appropriate answers.

[0944] The "emotion engine" is an analytical engine that analyzes the user's emotional data and provides an appropriate response based on that emotion.

[0945] A "session ID" is an identifier temporarily assigned to a user who has been successfully authenticated, and is used to manage the continuation of the session.

[0946] "Big data analysis" is a processing method for analyzing large amounts of accumulated data and extracting useful information.

[0947] "Feedback" refers to the evaluation or opinion a user gives regarding the answers or services provided.

[0948] This invention combines an emotion engine and an AI engine into a system that provides consistent, personalized support to users. This system performs user authentication, information collection, question response, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[0949] User Login and Authentication

[0950] A user uses a smartphone or specialized device to enter their email address and password into the app's login screen. The device sends the entered authentication information to the server, which checks it against a database to determine whether the authentication was successful. If authentication is successful, the server generates a session ID and sends it to the device. For example, if a user enters "user@example.com" and "password123" and authentication is successful, the server generates a session ID "abc123def456" and sends it to the device. The user receives a "Login successful" message.

[0951] User information collection

[0952] After a user logs in, the system asks for consent to the collection of location information and usage history. If the user consents, the device will use its GPS function to obtain current location information and periodically collect usage history and send it to the server. The server stores the received data in a database and analyzes it to understand the user's characteristics. For example, if the user consents to the collection of location information, the device will obtain latitude and longitude and periodically send this information to the server. The server will analyze this information to understand the user's behavioral patterns.

[0953] Processing user requests

[0954] The user enters a question or request into the message input field within the app. For example, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which uses a generative AI model and emotion engine to analyze the input data and generate an optimal answer taking emotion into account. For example, the server generates a response such as "The weather is nice, so I recommend going for a jog" and sends it to the device.

[0955] Emotion recognition by emotion engine

[0956] When a user inputs a question or request for advice, the device collects the user's emotional data using means such as voice recognition and facial expression analysis. For example, when a user asks, "Work hasn't been going well lately. What should I do?", the device analyzes the tone of the voice and facial expression. The server analyzes the collected emotional data, understands the user's current emotional state, and generates answers and advice based on that. For example, it may make suggestions such as, "How about listening to some relaxing music?"

[0957] Answers and Feedback

[0958] The server sends the generated answer to the device, and the user confirms the answer they received. For example, the user enters feedback such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve the accuracy of the system.

[0959] Big data analysis and its use for marketing information

[0960] The server integrates data collected from all users and performs big data analysis. For example, it analyzes user behavioral patterns and emotional data to find trends such as "Many users visit parks on weekends" or "Users want to refresh themselves on weekends." Based on this, it can develop marketing strategies and products and implement effective marketing.

[0961] Specific examples

[0962] User login and authentication examples

[0963] The user enters "user@example.com" and "password123", and the terminal sends this to the server. The server checks the database and, if authentication is successful, generates a session ID "abc123def456" and sends it to the terminal. The user receives a "Login successful" message.

[0964] Examples of user information collection

[0965] When a user agrees to the collection of location information, the device acquires latitude and longitude and sends them to a server, which analyzes this data to understand the user's behavioral patterns.

[0966] Example of user request processing

[0967] The user types, "Tell me what kind of exercise I should do today," and the device sends this to the server. The server uses a generative AI model and emotion engine to generate a response such as, "The weather is nice, so I recommend jogging," and sends it to the device.

[0968] Specific examples of emotion recognition using emotion engines

[0969] The user asks, "Work hasn't been going well lately. What should I do?" and the device collects emotional data through voice recognition and facial expression analysis. The server analyzes this data and generates a response such as, "How about listening to some relaxing music?" and sends it to the device.

[0970] Specific examples of answers and feedback

[0971] The user enters feedback into the device, such as "This advice was helpful," and the device sends it to the server, which analyzes the feedback and uses it to improve the accuracy of the system.

[0972] Specific examples of big data analysis and its use in marketing information

[0973] The server analyzes the data collected from all users and finds a trend, such as "Many users visit the park on weekends." This, along with emotional data, is used for marketing strategies and product development.

[0974] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[0975] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0976] Step 1:

[0977] The user enters their email address and password into the app's login screen using a smartphone or specialized device.

[0978] Input: A user enters the email address "user@example.com" and the password "password123".

[0979] Output: The authentication information sent to the device.

[0980] Step 2:

[0981] The terminal sends the entered authentication information to the server. The transmitted data is encrypted to ensure the security of the communication.

[0982] Input: The authentication information entered by the user.

[0983] Output: The encrypted authentication information sent to the server.

[0984] Step 3:

[0985] The server compares the received authentication information with the database and determines whether the authentication was successful.

[0986] Input: Encrypted authentication information sent from the device.

[0987] Output: Authentication success / failure and, if successful, the new session ID "abc123def456".

[0988] Step 4:

[0989] The server sends the authentication result and session ID to the terminal.

[0990] Input: Authentication result and session ID.

[0991] Output: Authentication result and session ID sent to the terminal.

[0992] Step 5:

[0993] The terminal receives the message from the server and displays "Login successful" to the user.

[0994] Input: Authentication result and session ID sent by the server.

[0995] Output: User is told "Login successful" and the session ID is displayed.

[0996] Step 6:

[0997] After logging in, the user is asked to consent to the collection of location information and usage history by the system, and agrees.

[0998] Input: User consent choice (e.g. clicking the "I agree" button).

[0999] Output: Start of terminal data collection.

[1000] Step 7:

[1001] The device uses GPS to obtain the current location and also periodically records app usage history.

[1002] Input: Consent given by the user.

[1003] Output: Obtained location information (latitude and longitude) and usage history data.

[1004] Step 8:

[1005] The device sends the location information and usage history it has acquired to the server. The data is encrypted and sent securely to the server.

[1006] Input: Captured location and usage data.

[1007] Output: The encrypted data sent to the server.

[1008] Step 9:

[1009] The server stores the received data in a database and analyzes it to understand the user's characteristics.

[1010] Input: Location and usage data sent from your device.

[1011] Output: Saving to database and analysis results (user behavior patterns and characteristics).

[1012] Step 10:

[1013] The user types a question or inquiry into a message input field within the app (e.g., "What exercise should I do today?").

[1014] Input: User-generated text for questions or inquiries.

[1015] Output: Input data ready for transmission.

[1016] Step 11:

[1017] The terminal transmits the input data from the user to the server.

[1018] Input: String data from the user.

[1019] Output: The question or question data sent to the server.

[1020] Step 12:

[1021] The server analyzes the input data using a generative AI model and emotion engine, and generates the optimal answer taking emotions into account.

[1022] Input: Questions and inquiries submitted by users, as well as sentiment data.

[1023] Data processing: Analyze the content of questions and consultations and emotional state.

[1024] Output: The best answer based on the analysis results (e.g., "The weather is nice, so I recommend going for a jog").

[1025] Step 13:

[1026] The server generates a response and sends it to the terminal.

[1027] Input: The generated answer data.

[1028] Output: The response data sent to the device.

[1029] Step 14:

[1030] The terminal receives the response from the server and displays it to the user.

[1031] Input: The response data sent from the server.

[1032] Output: Display of the answer to the user.

[1033] Step 15:

[1034] When the terminal inputs a user's question or inquiry, it collects the user's emotional data using means such as voice recognition and facial expression analysis.

[1035] Input: User's voice and facial expression data.

[1036] Output: Collected emotion data.

[1037] Step 16:

[1038] The device transmits the collected emotion data to a server.

[1039] Input: Collected emotion data.

[1040] Output: Emotion data sent to the server.

[1041] Step 17:

[1042] The server analyzes the collected emotional data to understand the user's current emotional state.

[1043] Input: Emotion data sent from the device.

[1044] Data processing: Analysis of emotion data.

[1045] Output: Analysis results about the user's emotional state.

[1046] Step 18:

[1047] The user reviews the answers provided and enters feedback into the device (e.g., "This advice was helpful").

[1048] Input: User-entered text feedback.

[1049] Output: Input data ready for transmission.

[1050] Step 19:

[1051] The device sends the feedback to the server.

[1052] Input: Feedback data entered by the user.

[1053] Output: Feedback data sent to the server.

[1054] Step 20:

[1055] The server analyzes the feedback it receives and uses it to improve the accuracy of the system.

[1056] Input: Feedback data sent from the device.

[1057] Data processing: Analysis of feedback data.

[1058] Output: Analysis results for improving system accuracy.

[1059] Step 21:

[1060] The server integrates the data collected from all users and performs big data analysis.

[1061] Input: All accumulated user data.

[1062] Data processing: Data analysis using big data analysis software.

[1063] Output: Analysis of user behavior patterns and trends.

[1064] Step 22:

[1065] The server uses the insights gained from the analysis results to inform marketing strategies and product development.

[1066] Input: Results of big data analysis.

[1067] Output: Marketing strategy and product development plan.

[1068] The system obtained in this way is capable of providing consistent personalized support that takes into account the user's emotions, contributing to improved user satisfaction and the generation of effective marketing information.

[1069] (Application example 2)

[1070] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1071] Conventional systems provide personalized support without considering the user's emotional state, resulting in a limited user experience and the inability to provide optimal advice or product recommendations based on emotions. Furthermore, there is a lack of analysis using user feedback and big data, which can make it difficult to improve the accuracy of the system or to formulate marketing strategies.

[1072] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1073] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as location information and usage history with the user's consent, means for analyzing the collected data and understanding the user's characteristics, means for accepting questions and inquiries from the user, means for analyzing the accepted questions and inquiries and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, means for performing big data analysis using the accumulated data and utilizing it for marketing and product development, and means for analyzing the user's emotional state using an emotion engine and suggesting optimal products based on the user's emotions. This enables highly accurate personalized support that takes user emotions into consideration.

[1074] definition statement

[1075] "User Credentials" are information used to verify a user's identity, typically including an email address and password.

[1076] A "database" is a system for storing and managing data in an organized manner. It stores user authentication information and collected data.

[1077] "User consent" is the act of a user permitting a system to collect and use their data. This is important from the perspective of protecting user privacy.

[1078] "Location information" is data that indicates a user's current location. It is often collected using GPS functionality.

[1079] "Usage history" refers to behavioral data when a user uses a system or application, including the pages visited and the duration of use.

[1080] "Question or consultation input" refers to a question or message requesting advice that a user inputs into the system.

[1081] "Generating an answer" means that the system creates an appropriate answer to a question or inquiry from a user.

[1082] "Feedback" refers to the evaluation or opinion a user gives in response to a received answer.

[1083] "Big data analytics" is an analytical technique that integrates large amounts of data and finds patterns and trends.

[1084] "Marketing and product development" refers to the activities of planning and developing new products and services based on collected data.

[1085] An "emotion engine" is a technology that analyzes a user's emotional state and responds optimally based on that.

[1086] A "session ID" is a temporary identifier that is generated when a user logs into a system. It is provided to the user after successful authentication.

[1087] "Anonymization" is the process of removing or obscuring personally identifiable information from collected data in order to protect user privacy.

[1088] MODE FOR CARRYING OUT THE INVENTION

[1089] The present invention includes an emotion-aware virtual shopping assistant system. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing consistent personalized services that take user emotions into consideration. Specific implementation methods for this system are described in detail below.

[1090] Hardware and software used

[1091] 1. Hardware:

[1092] Smartphone

[1093] Specialized devices (e.g. tablets)

[1094] 2. Software:

[1095] requests (Python HTTP request library)

[1096] Server-side API (e.g. Django, Flask)

[1097] Emotion engine (e.g. Microsoft Azure Emotion API)

[1098] Big data analysis engines (e.g., Apache Hadoop, Apache Spark)

[1099] Program processing

[1100] User Login and Authentication

[1101] The server receives the email address and password entered by the user on a smartphone or specialized device and performs authentication by comparing them with a database. If authentication is successful, the server generates a session ID and sends it to the device. Through this process, user authentication is established.

[1102] User information collection

[1103] With the consent of the authenticated user, the server uses the device's GPS function to collect current location information and transmits usage history to the server. The collected data is analyzed using a big data analysis engine on the server side to understand the user's usage patterns and characteristics.

[1104] Handling user inquiries and enquiries

[1105] The device sends a message from the user to the server, which analyzes the message, uses an emotion engine to understand the user's emotional state, and generates an optimal answer. The generated answer is sent to the device and provided to the user.

[1106] Feedback collection and analysis

[1107] When a user enters feedback on a given answer, the device sends the feedback to the server, which analyzes the collected feedback and uses it to improve the accuracy of the system.

[1108] Big data analysis and its use for marketing information

[1109] The collected data is integrated on a server and analyzed by a big data analysis engine. The analysis results are used in marketing and product development, and insights based on user sentiment are generated.

[1110] Specific examples

[1111] User Questions

[1112] When a user types, "I've been feeling tired from work lately, and I want a product that helps me relax," the device sends this request to the server. The server uses an emotion engine to detect "fatigue" and "stress" and suggests products that will help you relax (e.g., an aroma diffuser).

[1113] Example prompt sentence:

[1114] Based on the following specifications, please generate a program to create a virtual shopping assistant system that combines an emotion engine.

[1115] The user enters their email address and password on the login screen.

[1116] Authentication information is sent to the server, and a session ID is generated upon successful authentication.

[1117] After logging in, location and usage data is collected

[1118] Recognizes emotions from user input and suggests optimal products

[1119] Use the Python language to send HTTP requests using the requests library.

[1120] Assumes Django or Flask as the server-side API

[1121] Examples:

[1122] If a user types, "I've been tired from work lately and I want a product that helps me relax," the emotion engine will detect "fatigue" and "stress" and suggest products that will help you relax (e.g., an aroma diffuser).

[1123] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1124] Specific explanation of processing steps

[1125] Step 1:

[1126] User Login and Authentication

[1127] Input: The user enters their email address and password into their smartphone.

[1128] Specific operation: The device sends the entered authentication information to the server. The server checks the information against a database, determines whether the authentication was successful, and if the authentication was successful, generates a session ID and sends it to the device.

[1129] Output: If authentication is successful, the server generates a session ID and the user receives a "Login successful" message.

[1130] Step 2:

[1131] User information collection

[1132] Input: An authenticated user consents to the collection of location and usage information.

[1133] Specific operation: The device periodically collects its current location information using the GPS function and sends it along with its usage history to the server, which then stores the received data in a database.

[1134] Output: The server periodically receives location information and usage history and stores the data in a database.

[1135] Step 3:

[1136] Input of questions and inquiries

[1137] Input: The user types a question or request into the app's message input field.

[1138] How it works: The device sends the user's input to the server, which then analyzes it using an AI engine and emotion engine.

[1139] Output: The server is ready to understand the question or request as a result of the analysis.

[1140] Step 4:

[1141] Analyzing input data and generating answers

[1142] Input: The server receives input such as questions and inquiries, as well as emotion data.

[1143] How it works: The server's emotion engine analyzes the user's emotional state, and the AI ​​engine generates the optimal answer, customizing the answer using past data and the user's current emotional state.

[1144] Output: The server sends the generated answer to the user's device.

[1145] Step 5:

[1146] Providing answers and collecting feedback

[1147] Input: The user receives and confirms the provided answer on the terminal.

[1148] Specific operation: The device displays the answer to the user, and the user enters feedback on the answer. The feedback is sent from the device to the server.

[1149] Output: The server stores the received feedback as analysis results to help improve accuracy later.

[1150] Step 6:

[1151] Big data analysis and its use for marketing information

[1152] Input: The server uses location information, usage history, emotional data, and feedback data collected from all users.

[1153] How it works: The server's big data analysis engine analyzes this data to discover user behavior patterns and trends, and generates information based on the insights gained to be used in marketing and product development.

[1154] Output: The server outputs trend analysis results and insights for marketing strategies.

[1155] Step 7:

[1156] Continuously updating the emotion engine and improving system accuracy

[1157] Input: User feedback and parsed data.

[1158] Specific operation: The server analyzes the feedback and collected emotion data and makes updates to continuously improve the accuracy of the emotion engine and the entire system.

[1159] Output: The server implements a new version of the emotion engine with improved accuracy, improving the overall system performance.

[1160] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1161] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1162] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1163] [Third embodiment]

[1164] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1165] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1166] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1167] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1168] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1169] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1170] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1171] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1172] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1173] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1174] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1175] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1176] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[1177] System Program and Processing Overview

[1178] User Login and Authentication

[1179] User: Enters email address and password to log in to a smartphone or specialized device. Device: Sends the entered authentication information to the server. Server: Checks against the database to determine whether authentication was successful, and if authentication is successful, generates a session ID and sends it to the device.

[1180] User information collection

[1181] User: After logging in, the system asks for consent to the collection of location information and usage history. If consent is given, data collection begins. Device: After obtaining consent, the collected data is sent to the server. Server: The collected data is analyzed to understand the user's characteristics.

[1182] Processing user requests

[1183] User: Uses the messaging function to input questions or inquiries into the system. Device: The input request is sent as text data to the server. Server: An AI engine is used to analyze the text data and generate the optimal answer.

[1184] Answers and Feedback

[1185] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends the feedback to the server. Server: Analyzes the feedback and improves the accuracy of the system.

[1186] Big data analysis and its use for marketing information

[1187] Server: Aggregates and analyzes data collected from all users. Identifies user behavior patterns and trends and uses them for marketing and product development.

[1188] Specific examples

[1189] User login and authentication examples

[1190] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[1191] Examples of user information collection

[1192] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[1193] Example of user request processing

[1194] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data and the user's current health status and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[1195] Specific examples of answers and feedback

[1196] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[1197] Specific examples of big data analysis and its use in marketing information

[1198] The server analyzes the data collected from all users and finds a trend, such as "Many users go to parks on weekends." Based on this, it develops related products and formulates an effective marketing strategy.

[1199] The above is a specific embodiment for carrying out the present invention. The present invention aims to provide consistent personalized support and make users' lives more comfortable.

[1200] The processing flow will be explained below.

[1201] User login and authentication process steps

[1202] Step 1:

[1203] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[1204] Step 2:

[1205] The device sends the entered email address and password to the server.

[1206] Step 3:

[1207] The server checks the email address and password against a database to determine whether authentication was successful.

[1208] Step 4:

[1209] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[1210] Step 5:

[1211] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[1212] User information collection process steps

[1213] Step 1:

[1214] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[1215] Step 2:

[1216] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[1217] Step 3:

[1218] The device sends the collected location information and usage history to the server.

[1219] Step 4:

[1220] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[1221] Processing steps for user request processing

[1222] Step 1:

[1223] The user enters a question or request into the message input field within the app.

[1224] Step 2:

[1225] The terminal transmits the input data from the user to the server.

[1226] Step 3:

[1227] The server uses an AI engine to analyze the input data and generate an appropriate answer.

[1228] Step 4:

[1229] The server sends the generated response to the terminal.

[1230] Step 5:

[1231] The terminal notifies the user of the received answer and allows the user to view it.

[1232] Response and feedback processing steps

[1233] Step 1:

[1234] The user enters feedback on the answers provided.

[1235] Step 2:

[1236] The terminal sends the user's feedback to the server.

[1237] Step 3:

[1238] The server stores the received feedback in a database and analyzes the feedback.

[1239] Step 4:

[1240] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[1241] Processing steps for big data analysis and its use in marketing information

[1242] Step 1:

[1243] The server aggregates the data collected from all users to create a large dataset.

[1244] Step 2:

[1245] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[1246] Step 3:

[1247] The server generates insights for marketing strategies and new product development based on the analysis results.

[1248] Step 4:

[1249] The server provides the generated insights to the marketing and product development teams.

[1250] Example 1

[1251] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1252] In modern society, systems that provide personalized support are required to meet the diverse needs of users. However, current systems do not effectively collect, analyze, or utilize user information, resulting in inconsistent user experiences and making it difficult to improve satisfaction. It is also difficult to effectively utilize collected data for marketing and product development.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1254] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as the user's location information and usage history with the authenticated user's consent, means for analyzing the collected data and understanding the user's characteristics, means for analyzing questions and inquiries from the user using a generative AI model and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, and means for performing big data analysis using the accumulated data and utilizing it for marketing and product development. This allows for consistent, personalized support to be provided to users, improving the user experience, and making it possible to effectively utilize the data for marketing and product development.

[1255] "User authentication information" is information provided by a user to identify themselves and prove their access rights.

[1256] A "database" is a system for storing data in an organized manner and facilitating its retrieval and management.

[1257] "Location information" is data that indicates the user's current geographical location.

[1258] "Usage history" refers to data related to the operation history and behavior patterns of users when they use systems or applications.

[1259] A "collection method" is a method or device for capturing and storing specific data.

[1260] "Analytical tools" are methods and equipment used to analyze collected data and find meaning.

[1261] A "generative AI model" is an algorithm or software that uses artificial intelligence techniques to analyze data and generate answers or suggestions.

[1262] The "answer generation means" refers to a method or algorithm for creating an appropriate answer to a question or inquiry input by a user.

[1263] "Feedback" is data regarding ratings and opinions provided by users.

[1264] "Big data analysis" is a technology that analyzes large amounts of data to extract useful information.

[1265] "Marketing" refers to the activities and strategies used to promote the sale of products and services.

[1266] "Product development" is the process of designing and delivering new products and services.

[1267] "Privacy protection" refers to measures to prevent the leakage or misuse of personal information.

[1268] "Anonymization" is the process of processing data so that it cannot identify a specific individual.

[1269] A "session ID" is an identifier temporarily assigned to identify a session of a specific user.

[1270] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[1271] User Login and Authentication

[1272] To log in to a smartphone or specialized device, a user enters an email address and password. The device sends the entered authentication information to the server. The server compares it with a database (e.g., MySQL, PostgreSQL) and determines whether the authentication was successful. If the authentication is successful, the server generates a session ID and sends it to the device. The user's login is confirmed when a "Login successful" message is displayed on the device.

[1273] User information collection

[1274] After logging in, the user will see a pop-up requesting consent to the collection of location information and usage history. If they agree, they click "Agree." The device will use its GPS function to obtain location information and collect usage history. The device will then send this data to the server. The server will analyze the received data and understand the user's characteristics. Specifically, the server will perform analysis to identify the locations the user frequently visits and app usage patterns.

[1275] Answering questions

[1276] Users use the messaging function to input questions or inquiries into the system. For example, they might type, "What exercise would you recommend today?" The device sends this request as text data to the server. The server then analyzes the text data using a generative AI model (e.g., BERT, GPT-3). Based on the analysis results, the server generates an appropriate answer. For example, the server might generate an answer such as, "The sun is strong today, so I recommend jogging in the evening." The server then sends the generated answer to the device and provides it to the user.

[1277] Feedback collection

[1278] The user enters feedback on the provided answer, for example, "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to improve the accuracy of the system. A specific example is that the server uses the feedback data to update the training dataset of a generative AI model.

[1279] Big data analysis and its use for marketing information

[1280] The server aggregates the data collected from all users and analyzes it using big data analysis tools (e.g., Hadoop, Spark). User behavior patterns and trends are extracted and used for marketing and product development. For example, a trend such as "many users visit the park on weekends" can be identified, and a new product promotion can be developed based on this.

[1281] The above is a specific embodiment of the present invention, which can provide consistent and personalized support to users and improve their quality of life.

[1282] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1283] User Login and Authentication

[1284] Step 1:

[1285] A user enters an email address and password to log in to a smartphone or specialized device. This is the input for the process.

[1286] Step 2:

[1287] The terminal sends the authentication information (email address and password) entered by the user to the server. This authentication information is the input for the process.

[1288] Step 3:

[1289] The server receives the authentication information and checks it against a database (e.g., MySQL, PostgreSQL). During this process, it compares the entered authentication information with the information in the database and outputs the result of the check.

[1290] Step 4:

[1291] The server determines whether authentication is successful, and if authentication is successful, generates a session ID. The generated session ID is output.

[1292] Step 5:

[1293] The server sends the generated session ID to the terminal, which stores the session ID and displays a "Login successful" message to the user, confirming that the user has logged in.

[1294] User information collection

[1295] Step 1:

[1296] After logging in, users will be shown a pop-up requesting consent to the collection of location information and usage history. This is the input for processing.

[1297] Step 2:

[1298] If the user agrees to the data collection, they click "Agree." The consent click is the input.

[1299] Step 3:

[1300] The device uses GPS to obtain the user's current location and collects app usage history, which are the inputs for processing.

[1301] Step 4:

[1302] The device sends the collected location information and usage history to the server. This transmission becomes the input.

[1303] Step 5:

[1304] The server analyzes the received data and understands the user's characteristics, such as the locations the user frequently visits and the app usage patterns. The results of this analysis are output.

[1305] Processing user requests

[1306] Step 1:

[1307] Users use the messaging function to input questions or inquiries into the system. For example, they might input, "What exercise would you recommend today?" This is the input for the process.

[1308] Step 2:

[1309] The terminal sends the input request as text data to the server, and the sent text data becomes the input.

[1310] Step 3:

[1311] The server analyzes the received text data using a generative AI model (e.g., BERT, GPT-3). This analysis is the data processing process. For example, it generates appropriate suggestions based on the content of the input request.

[1312] Step 4:

[1313] The server generates the optimal answer based on the analysis results, such as "The sun is strong today, so I recommend jogging in the evening."

[1314] Step 5:

[1315] The server sends the generated answer to the terminal and provides it to the user, who then confirms the answer.

[1316] Answers and Feedback

[1317] Step 1:

[1318] The user enters feedback on the provided answer, for example, "This advice was helpful." This is the input for the process.

[1319] Step 2:

[1320] The device sends feedback to the server, which then becomes the input.

[1321] Step 3:

[1322] The server analyzes the received feedback and extracts from it which parts of the system need improvement. This analysis is the data processing process.

[1323] Step 4:

[1324] The server uses the analyzed feedback to update models and algorithms to improve the accuracy of the system, which is the output.

[1325] Big data analysis and its use for marketing information

[1326] Step 1:

[1327] The server aggregates the data collected from all users, and the aggregated data is the input for processing.

[1328] Step 2:

[1329] The server analyzes the aggregated data using big data analysis tools (e.g., Hadoop, Spark). This analysis is the data processing process.

[1330] Step 3:

[1331] The server extracts user behavior patterns and trends from the analysis results. For example, the output may be a trend such as "Many users visit the park on weekends."

[1332] Step 4:

[1333] The server uses the analysis results for marketing and product development, for example, to plan promotions for new products based on these trends.

[1334] These processing steps enable the system to provide consistent, personalized support to users, improving the user experience and enabling effective use in marketing and product development.

[1335] (Application example 1)

[1336] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1337] Conventional content delivery systems lack the ability to provide personalized content recommendations based on user preferences and viewing history, making it difficult for users to access the content they desire. Furthermore, the mechanisms for properly collecting and analyzing user feedback and reflecting it in improving the system's accuracy were also inadequate. Furthermore, collected data was not anonymized, leaving many users with concerns about privacy.

[1338] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1339] In this invention, the server includes a means for recommending content based on the user's viewing history and genre of interest, a means for analyzing the user's input text and making optimal recommendations using a generative AI model, and a means for providing the user with recommendations generated using prompts. This enables the provision of optimal content tailored to the user's preferences and improves the accuracy of the system based on feedback analysis. Furthermore, privacy protection can be strengthened by anonymizing user data.

[1340] "User authentication" is the process of verifying the legitimacy of a user when the user accesses a system using authentication information such as an email address and password.

[1341] "Data collection" is the process of collecting information such as a user's location, usage history, and viewing history in order to store it in the system.

[1342] "Data analysis" is the process of analyzing collected data to understand user behavior patterns and characteristics.

[1343] "Question acceptance" is a process of accepting questions and inquiries from users as input.

[1344] "Answer generation" is the process of analyzing received questions and inquiries and providing the most appropriate answer to the user.

[1345] "Feedback collection" is the process of collecting evaluations and opinions from users and using them to improve the system.

[1346] A "viewing history" is a record of content that a user has viewed in the past.

[1347] "Content recommendation" is the process of providing content suitable for a user based on collected viewing history and genres of interest.

[1348] A "generative AI model" is a model that uses artificial intelligence to analyze user input and generate optimal answers or recommendations.

[1349] A "prompt sentence" is a sentence used to input specific instructions or questions to a generative AI model.

[1350] "Anonymization" is the process of protecting a user's personal information by transforming it so that it cannot be identified.

[1351] "Big data analysis" is the process of analyzing large amounts of data collected from a large number of users and utilizing it for marketing and product development.

[1352] A "session ID" is a unique identifier generated to identify a user after successful user authentication.

[1353] The present invention provides a system that performs user authentication, information collection, question handling, feedback collection, and big data analysis all at once to provide consistent, personalized support to users. This system is specifically implemented as follows.

[1354] User Login and Authentication

[1355] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session ID and sends it to the device. This completes user authentication.

[1356] Data collection

[1357] Authenticated users are asked to consent to the collection of their location information and viewing history. Once consent is given, the device will periodically acquire the user's location information using GPS or other functions and send it along with their viewing history to the server. The server then analyzes the collected data to understand the user's characteristics and behavioral patterns.

[1358] Question acceptance and answer generation

[1359] Users can input questions or inquiries using the device's interface. The device then sends this as text data to the server. The server then uses a generative AI model to analyze the user's input and generate optimal answers or content recommendations. For example, if a user types, "What movie should I watch tonight?", the server might generate an answer such as, "Considering the action movies you've recently watched and the reviews, I recommend 'Inception' tonight," and send it to the device.

[1360] Gathering feedback and improving system accuracy

[1361] When a user enters feedback on the answer they received, the device sends the feedback to the server, which analyzes the feedback and uses it as data to improve the accuracy of the system. This improves the accuracy of answers and content recommendations from the next time onwards.

[1362] Utilizing big data analysis and marketing information

[1363] The server performs big data analysis using the accumulated data from all users, which allows users to identify patterns and trends in their behavior and utilize this information for marketing and product development.

[1364] The specific hardware and software used include smartphones and head-mounted displays (HMDs), web frameworks such as Flask for communication with the server, SQLite for the database, and OpenAI GPT-3 as the generative AI model.

[1365] Prompt Sentence Examples

[1366] When a user asks the following question:

[1367] "Please tell me what movie I should watch today."

[1368] The server generates a response as follows:

[1369] "Based on your recent action movies and reviews, I recommend Inception tonight because it combines a tight storyline with the action you enjoy."

[1370] As described above, the present invention aims to provide consistent personalized support and make users' lives more comfortable.

[1371] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1372] Step 1:

[1373] User Authentication

[1374] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The entered authentication information is sent from the device to the server. The server compares the received authentication information with a database, and if authentication is successful, generates a session ID and sends it to the device. This authenticates the user.

[1375] Input: User's email address and password

[1376] Data processing: Database matching of authentication information

[1377] Output: Session ID

[1378] Step 2:

[1379] Data collection

[1380] Authenticated users are asked to consent to the collection of their location information and viewing history. If the user consents, the device will periodically acquire location information using GPS or other functions and send it along with their viewing history to the server. The server receives and stores this data, and analyzes it to understand the user's behavioral patterns and characteristics.

[1381] Input: User consent, GPS location, viewing history

[1382] Data processing: data collection and storage, analysis of behavioral patterns and characteristics

[1383] Output: Characteristic data

[1384] Step 3:

[1385] Questions accepted

[1386] The user inputs a question or request using the terminal interface. The input request is sent as text data from the terminal to the server, which then receives the text data.

[1387] Input: User's question or inquiry (text data)

[1388] Data processing: Receiving text data

[1389] Output: Unparsed text data

[1390] Step 4:

[1391] Answer generation

[1392] The server analyzes the received text data using a generative AI model. Based on the analysis results, it generates the most suitable answer and content recommendation for the user. For example, in response to the question, "What movie should I watch today?", it will recommend "Inception" based on the user's past viewing history and preferences. The generated answer is sent to the device in text format.

[1393] Input: Unparsed text data

[1394] Data processing: Text analysis, answer generation using generative AI models

[1395] Output: User-optimized answers and content recommendations (text format)

[1396] Step 5:

[1397] Providing answers to users

[1398] The terminal displays the answers received from the server to the user, allowing the user to obtain answers to their questions and content recommendations.

[1399] Input: Best answer or content recommendation from the server (text format)

[1400] Data processing: conversion to user display

[1401] Output: The displayed answer

[1402] Step 6:

[1403] Feedback collection

[1404] The user enters feedback on the answers provided. The feedback is sent from the device to the server, which receives and analyzes the feedback. This data is used to improve the accuracy of the system.

[1405] Input: User feedback (text format)

[1406] Data processing: receiving and analyzing feedback

[1407] Output: Data for accuracy improvement

[1408] Step 7:

[1409] Utilizing big data analysis and marketing information

[1410] The server aggregates data collected from all users and performs big data analysis. This allows users to identify patterns and trends in their behavior and use them in marketing and product development. For example, a pattern could be identified where "many users go to the park on weekends."

[1411] Input: Aggregated data for all users

[1412] Data processing: Big data analysis

[1413] Output: Behavioral patterns and trend information

[1414] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1415] This invention combines an emotion engine with a system that provides consistent personalized support to users to provide a more accurate and emotion-aware service. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[1416] System Program and Processing Overview

[1417] User Login and Authentication

[1418] User: Opens a smartphone or specialized device and enters an email address and password on the app's login screen. Device: The entered authentication information is sent to the server. Server: After checking the database and determining whether authentication was successful, if authentication was successful a session ID is generated and sent to the device.

[1419] User information collection

[1420] User: After logging in, the system asks for consent to the collection of location information and usage history, and if consent is given, data collection begins. Device: After obtaining consent, the system uses the GPS function to obtain current location information and periodically collects usage history and sends it to the server. Server: The received data is stored in a database and analyzed to understand the user's characteristics.

[1421] Processing user requests

[1422] User: Enters a question or inquiry into the message input field within the app. Device: Sends the input data from the user to the server. Server: Analyzes the input data using an AI engine and emotion engine, and generates the optimal answer taking emotions into account.

[1423] Emotion recognition by emotion engine

[1424] Terminal: When a user inputs a question or request for advice, the device collects the user's emotional data using methods such as voice recognition and facial expression analysis. Server: Analyzes the collected emotional data and understands the user's current emotional state. Based on the results, the device generates more appropriate answers and advice tailored to each individual user.

[1425] Answers and Feedback

[1426] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends feedback to the server. Server: Analyzes the feedback and emotion data to improve the accuracy of the system.

[1427] Big data analysis and its use for marketing information

[1428] Server: Integrates data collected from all users and performs big data analysis. Data analysis identifies user behavior patterns and trends, which are used for marketing and product development. Collected emotional data is also included to generate insights based on user emotions.

[1429] Specific examples

[1430] User login and authentication examples

[1431] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[1432] Examples of user information collection

[1433] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[1434] Example of user request processing

[1435] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data, current health status, and emotional state determined by the emotion engine, and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[1436] Specific examples of emotion recognition using emotion engines

[1437] When a user is feeling stressed, they ask, "Work hasn't been going well lately. What should I do?" The device detects the user's stress level through voice recognition and facial expression analysis, and sends that data to the server. The server then uses its emotion engine to generate advice to reduce stress, such as "How about listening to some relaxing music?", and provides it to the user.

[1438] Specific examples of answers and feedback

[1439] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[1440] Specific examples of big data analysis and its use in marketing information

[1441] The server analyzes the data collected from all users and finds a trend that "many users visit parks on weekends." Based on the emotional data, it also finds that many users seek relaxation on weekends. Based on this, related products can be developed and effective marketing strategies can be formulated.

[1442] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[1443] The processing flow will be explained below.

[1444] User login and authentication process steps

[1445] Step 1:

[1446] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[1447] Step 2:

[1448] The device sends the entered email address and password to the server.

[1449] Step 3:

[1450] The server checks the email address and password against a database to determine whether authentication was successful.

[1451] Step 4:

[1452] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[1453] Step 5:

[1454] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[1455] User information collection process steps

[1456] Step 1:

[1457] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[1458] Step 2:

[1459] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[1460] Step 3:

[1461] The device sends the collected location information and usage history to the server.

[1462] Step 4:

[1463] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[1464] Processing steps for user request processing

[1465] Step 1:

[1466] The user enters a question or request into the message input field within the app.

[1467] Step 2:

[1468] The terminal transmits the input data from the user to the server.

[1469] Step 3:

[1470] The server uses an AI engine and an emotion engine to analyze the input data and generate an appropriate answer taking emotions into account.

[1471] Step 4:

[1472] The server sends the generated response to the terminal.

[1473] Step 5:

[1474] The terminal notifies the user of the received answer and allows the user to view it.

[1475] Emotion recognition processing steps by the emotion engine

[1476] Step 1:

[1477] When the terminal inputs a user's question or inquiry, it collects emotional data using means such as voice recognition and facial expression analysis.

[1478] Step 2:

[1479] The device transmits the collected emotion data to the server.

[1480] Step 3:

[1481] The server analyzes the received emotional data and understands the user's current emotional state.

[1482] Step 4:

[1483] The server generates optimal answers and advice for the user based on the emotional data.

[1484] Response and feedback processing steps

[1485] Step 1:

[1486] The user enters feedback on the answers provided.

[1487] Step 2:

[1488] The device sends the feedback to the server.

[1489] Step 3:

[1490] The server stores the received feedback and emotional data in a database and analyzes it.

[1491] Step 4:

[1492] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[1493] Processing steps for big data analysis and its use in marketing information

[1494] Step 1:

[1495] The server aggregates the data collected from all users to create a large dataset.

[1496] Step 2:

[1497] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[1498] Step 3:

[1499] The server generates insights for marketing strategies and new product development based on the analysis results, including emotional data.

[1500] Step 4:

[1501] The server provides the generated insights to the marketing and product development teams.

[1502] Example 2

[1503] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1504] Conventional personalized support systems have difficulty in responding to user emotions, resulting in insufficient user satisfaction. In addition, they often cannot effectively analyze the collected data and utilize it for marketing and product development.

[1505] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving user authentication information and authenticating the user by comparing the authentication information with a database; means for collecting data such as the user's location information and usage history with the authenticated user's consent; means for analyzing the collected data and understanding the user's characteristics; means for accepting input of questions and inquiries from the user; means for analyzing the accepted questions and inquiries using a generative AI model and an emotion engine and generating answers that take emotions into consideration; means for providing the generated answers to the user; means for collecting and analyzing user emotion data; means for providing services adapted to the user's emotional state based on the analyzed emotion data; means for collecting and analyzing user feedback; means for improving system accuracy based on the analyzed feedback; and means for performing big data analysis using accumulated data and utilizing the results in marketing and product development. This makes it possible to provide consistent personalized support that takes users' emotions into consideration, improve user satisfaction, and generate effective marketing information.

[1506] "User Credentials" are the identification data a user uses to log into a system, typically a combination of email address and password.

[1507] "Database" means an information management system for managing and storing user authentication information and other data.

[1508] "Location information" is data indicating the user's current location obtained using GPS or other means.

[1509] "Usage history" refers to historical data of operations performed by a user within the system and functions accessed by the user.

[1510] A "generative AI model" is an artificial intelligence model used to analyze questions and inquiries from users and generate appropriate answers.

[1511] The "emotion engine" is an analytical engine that analyzes the user's emotional data and provides an appropriate response based on that emotion.

[1512] A "session ID" is an identifier temporarily assigned to a user who has been successfully authenticated, and is used to manage the continuation of the session.

[1513] "Big data analysis" is a processing method for analyzing large amounts of accumulated data and extracting useful information.

[1514] "Feedback" refers to the evaluation or opinion a user gives regarding the answers or services provided.

[1515] This invention combines an emotion engine and an AI engine into a system that provides consistent, personalized support to users. This system performs user authentication, information collection, question response, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[1516] User Login and Authentication

[1517] A user uses a smartphone or specialized device to enter their email address and password into the app's login screen. The device sends the entered authentication information to the server, which checks it against a database to determine whether the authentication was successful. If authentication is successful, the server generates a session ID and sends it to the device. For example, if a user enters "user@example.com" and "password123" and authentication is successful, the server generates a session ID "abc123def456" and sends it to the device. The user receives a "Login successful" message.

[1518] User information collection

[1519] After a user logs in, the system asks for consent to the collection of location information and usage history. If the user consents, the device will use its GPS function to obtain current location information and periodically collect usage history and send it to the server. The server stores the received data in a database and analyzes it to understand the user's characteristics. For example, if the user consents to the collection of location information, the device will obtain latitude and longitude and periodically send this information to the server. The server will analyze this information to understand the user's behavioral patterns.

[1520] Processing user requests

[1521] The user enters a question or request into the message input field within the app. For example, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which uses a generative AI model and emotion engine to analyze the input data and generate an optimal answer taking emotion into account. For example, the server generates a response such as "The weather is nice, so I recommend going for a jog" and sends it to the device.

[1522] Emotion recognition by emotion engine

[1523] When a user inputs a question or request for advice, the device collects the user's emotional data using means such as voice recognition and facial expression analysis. For example, when a user asks, "Work hasn't been going well lately. What should I do?", the device analyzes the tone of the voice and facial expression. The server analyzes the collected emotional data, understands the user's current emotional state, and generates answers and advice based on that. For example, it may make suggestions such as, "How about listening to some relaxing music?"

[1524] Answers and Feedback

[1525] The server sends the generated answer to the device, and the user confirms the answer they received. For example, the user enters feedback such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve the accuracy of the system.

[1526] Big data analysis and its use for marketing information

[1527] The server integrates data collected from all users and performs big data analysis. For example, it analyzes user behavioral patterns and emotional data to find trends such as "Many users visit parks on weekends" or "Users want to refresh themselves on weekends." Based on this, it can develop marketing strategies and products and implement effective marketing.

[1528] Specific examples

[1529] User login and authentication examples

[1530] The user enters "user@example.com" and "password123", and the terminal sends this to the server. The server checks the database and, if authentication is successful, generates a session ID "abc123def456" and sends it to the terminal. The user receives a "Login successful" message.

[1531] Examples of user information collection

[1532] When a user agrees to the collection of location information, the device acquires latitude and longitude and sends them to a server, which analyzes this data to understand the user's behavioral patterns.

[1533] Example of user request processing

[1534] The user types, "Tell me what kind of exercise I should do today," and the device sends this to the server. The server uses a generative AI model and emotion engine to generate a response such as, "The weather is nice, so I recommend jogging," and sends it to the device.

[1535] Specific examples of emotion recognition using emotion engines

[1536] The user asks, "Work hasn't been going well lately. What should I do?" and the device collects emotional data through voice recognition and facial expression analysis. The server analyzes this data and generates a response such as, "How about listening to some relaxing music?" and sends it to the device.

[1537] Specific examples of answers and feedback

[1538] The user enters feedback into the device, such as "This advice was helpful," and the device sends it to the server, which analyzes the feedback and uses it to improve the accuracy of the system.

[1539] Specific examples of big data analysis and its use in marketing information

[1540] The server analyzes the data collected from all users and finds a trend, such as "Many users visit the park on weekends." This, along with emotional data, is used for marketing strategies and product development.

[1541] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[1542] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1543] Step 1:

[1544] The user enters their email address and password into the app's login screen using a smartphone or specialized device.

[1545] Input: A user enters the email address "user@example.com" and the password "password123".

[1546] Output: The authentication information sent to the device.

[1547] Step 2:

[1548] The terminal sends the entered authentication information to the server. The transmitted data is encrypted to ensure the security of the communication.

[1549] Input: The authentication information entered by the user.

[1550] Output: The encrypted authentication information sent to the server.

[1551] Step 3:

[1552] The server compares the received authentication information with the database and determines whether the authentication was successful.

[1553] Input: Encrypted authentication information sent from the device.

[1554] Output: Authentication success / failure and, if successful, the new session ID "abc123def456".

[1555] Step 4:

[1556] The server sends the authentication result and session ID to the terminal.

[1557] Input: Authentication result and session ID.

[1558] Output: Authentication result and session ID sent to the terminal.

[1559] Step 5:

[1560] The terminal receives the message from the server and displays "Login successful" to the user.

[1561] Input: Authentication result and session ID sent by the server.

[1562] Output: User is told "Login successful" and the session ID is displayed.

[1563] Step 6:

[1564] After logging in, the user is asked to consent to the collection of location information and usage history by the system, and agrees.

[1565] Input: User consent choice (e.g. clicking the "I agree" button).

[1566] Output: Start of terminal data collection.

[1567] Step 7:

[1568] The device uses GPS to obtain the current location and also periodically records app usage history.

[1569] Input: Consent given by the user.

[1570] Output: Obtained location information (latitude and longitude) and usage history data.

[1571] Step 8:

[1572] The device sends the location information and usage history it has acquired to the server. The data is encrypted and sent securely to the server.

[1573] Input: Captured location and usage data.

[1574] Output: The encrypted data sent to the server.

[1575] Step 9:

[1576] The server stores the received data in a database and analyzes it to understand the user's characteristics.

[1577] Input: Location and usage data sent from your device.

[1578] Output: Saving to database and analysis results (user behavior patterns and characteristics).

[1579] Step 10:

[1580] The user types a question or inquiry into a message input field within the app (e.g., "What exercise should I do today?").

[1581] Input: User-generated text for questions or inquiries.

[1582] Output: Input data ready for transmission.

[1583] Step 11:

[1584] The terminal transmits the input data from the user to the server.

[1585] Input: String data from the user.

[1586] Output: The question or question data sent to the server.

[1587] Step 12:

[1588] The server analyzes the input data using a generative AI model and emotion engine, and generates the optimal answer taking emotions into account.

[1589] Input: Questions and inquiries submitted by users, as well as sentiment data.

[1590] Data processing: Analyze the content of questions and consultations and emotional state.

[1591] Output: The best answer based on the analysis results (e.g., "The weather is nice, so I recommend going for a jog").

[1592] Step 13:

[1593] The server generates a response and sends it to the terminal.

[1594] Input: The generated answer data.

[1595] Output: The response data sent to the device.

[1596] Step 14:

[1597] The terminal receives the response from the server and displays it to the user.

[1598] Input: The response data sent from the server.

[1599] Output: Display of the answer to the user.

[1600] Step 15:

[1601] When the terminal inputs a user's question or inquiry, it collects the user's emotional data using means such as voice recognition and facial expression analysis.

[1602] Input: User's voice and facial expression data.

[1603] Output: Collected emotion data.

[1604] Step 16:

[1605] The device transmits the collected emotion data to a server.

[1606] Input: Collected emotion data.

[1607] Output: Emotion data sent to the server.

[1608] Step 17:

[1609] The server analyzes the collected emotional data to understand the user's current emotional state.

[1610] Input: Emotion data sent from the device.

[1611] Data processing: Analysis of emotion data.

[1612] Output: Analysis results about the user's emotional state.

[1613] Step 18:

[1614] The user reviews the answers provided and enters feedback into the device (e.g., "This advice was helpful").

[1615] Input: User-entered text feedback.

[1616] Output: Input data ready for transmission.

[1617] Step 19:

[1618] The device sends the feedback to the server.

[1619] Input: Feedback data entered by the user.

[1620] Output: Feedback data sent to the server.

[1621] Step 20:

[1622] The server analyzes the feedback it receives and uses it to improve the accuracy of the system.

[1623] Input: Feedback data sent from the device.

[1624] Data processing: Analysis of feedback data.

[1625] Output: Analysis results for improving system accuracy.

[1626] Step 21:

[1627] The server integrates the data collected from all users and performs big data analysis.

[1628] Input: All accumulated user data.

[1629] Data processing: Data analysis using big data analysis software.

[1630] Output: Analysis of user behavior patterns and trends.

[1631] Step 22:

[1632] The server uses the insights gained from the analysis results to inform marketing strategies and product development.

[1633] Input: Results of big data analysis.

[1634] Output: Marketing strategy and product development plan.

[1635] The system obtained in this way is capable of providing consistent personalized support that takes into account the user's emotions, contributing to improved user satisfaction and the generation of effective marketing information.

[1636] (Application example 2)

[1637] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1638] Conventional systems provide personalized support without considering the user's emotional state, resulting in a limited user experience and the inability to provide optimal advice or product recommendations based on emotions. Furthermore, there is a lack of analysis using user feedback and big data, which can make it difficult to improve the accuracy of the system or to formulate marketing strategies.

[1639] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1640] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as location information and usage history with the user's consent, means for analyzing the collected data and understanding the user's characteristics, means for accepting questions and inquiries from the user, means for analyzing the accepted questions and inquiries and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, means for performing big data analysis using the accumulated data and utilizing it for marketing and product development, and means for analyzing the user's emotional state using an emotion engine and suggesting optimal products based on the user's emotions. This enables highly accurate personalized support that takes user emotions into consideration.

[1641] definition statement

[1642] "User Credentials" are information used to verify a user's identity, typically including an email address and password.

[1643] A "database" is a system for storing and managing data in an organized manner. It stores user authentication information and collected data.

[1644] "User consent" is the act of a user permitting a system to collect and use their data. This is important from the perspective of protecting user privacy.

[1645] "Location information" is data that indicates a user's current location. It is often collected using GPS functionality.

[1646] "Usage history" refers to behavioral data when a user uses a system or application, including the pages visited and the duration of use.

[1647] "Question or consultation input" refers to a question or message requesting advice that a user inputs into the system.

[1648] "Generating an answer" means that the system creates an appropriate answer to a question or inquiry from a user.

[1649] "Feedback" refers to the evaluation or opinion a user gives in response to a received answer.

[1650] "Big data analytics" is an analytical technique that integrates large amounts of data and finds patterns and trends.

[1651] "Marketing and product development" refers to the activities of planning and developing new products and services based on collected data.

[1652] An "emotion engine" is a technology that analyzes a user's emotional state and responds optimally based on that.

[1653] A "session ID" is a temporary identifier that is generated when a user logs into a system. It is provided to the user after successful authentication.

[1654] "Anonymization" is the process of removing or obscuring personally identifiable information from collected data in order to protect user privacy.

[1655] MODE FOR CARRYING OUT THE INVENTION

[1656] The present invention includes an emotion-aware virtual shopping assistant system. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing consistent personalized services that take user emotions into consideration. Specific implementation methods for this system are described in detail below.

[1657] Hardware and software used

[1658] 1. Hardware:

[1659] Smartphone

[1660] Specialized devices (e.g. tablets)

[1661] 2. Software:

[1662] requests (Python HTTP request library)

[1663] Server-side API (e.g. Django, Flask)

[1664] Emotion engine (e.g. Microsoft Azure Emotion API)

[1665] Big data analysis engines (e.g., Apache Hadoop, Apache Spark)

[1666] Program processing

[1667] User Login and Authentication

[1668] The server receives the email address and password entered by the user on a smartphone or specialized device and performs authentication by comparing them with a database. If authentication is successful, the server generates a session ID and sends it to the device. Through this process, user authentication is established.

[1669] User information collection

[1670] With the consent of the authenticated user, the server uses the device's GPS function to collect current location information and transmits usage history to the server. The collected data is analyzed using a big data analysis engine on the server side to understand the user's usage patterns and characteristics.

[1671] Handling user inquiries and enquiries

[1672] The device sends a message from the user to the server, which analyzes the message, uses an emotion engine to understand the user's emotional state, and generates an optimal answer. The generated answer is sent to the device and provided to the user.

[1673] Feedback collection and analysis

[1674] When a user enters feedback on a given answer, the device sends the feedback to the server, which analyzes the collected feedback and uses it to improve the accuracy of the system.

[1675] Big data analysis and its use for marketing information

[1676] The collected data is integrated on a server and analyzed by a big data analysis engine. The analysis results are used in marketing and product development, and insights based on user sentiment are generated.

[1677] Specific examples

[1678] User Questions

[1679] When a user types, "I've been feeling tired from work lately, and I want a product that helps me relax," the device sends this request to the server. The server uses an emotion engine to detect "fatigue" and "stress" and suggests products that will help you relax (e.g., an aroma diffuser).

[1680] Example prompt sentence:

[1681] Based on the following specifications, please generate a program to create a virtual shopping assistant system that combines an emotion engine.

[1682] The user enters their email address and password on the login screen.

[1683] Authentication information is sent to the server, and a session ID is generated upon successful authentication.

[1684] After logging in, location and usage data is collected

[1685] Recognizes emotions from user input and suggests optimal products

[1686] Use the Python language to send HTTP requests using the requests library.

[1687] Assumes Django or Flask as the server-side API

[1688] Examples:

[1689] If a user types, "I've been tired from work lately and I want a product that helps me relax," the emotion engine will detect "fatigue" and "stress" and suggest products that will help you relax (e.g., an aroma diffuser).

[1690] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1691] Specific explanation of processing steps

[1692] Step 1:

[1693] User Login and Authentication

[1694] Input: The user enters their email address and password into their smartphone.

[1695] Specific operation: The device sends the entered authentication information to the server. The server checks the information against a database, determines whether the authentication was successful, and if the authentication was successful, generates a session ID and sends it to the device.

[1696] Output: If authentication is successful, the server generates a session ID and the user receives a "Login successful" message.

[1697] Step 2:

[1698] User information collection

[1699] Input: An authenticated user consents to the collection of location and usage information.

[1700] Specific operation: The device periodically collects its current location information using the GPS function and sends it along with its usage history to the server, which then stores the received data in a database.

[1701] Output: The server periodically receives location information and usage history and stores the data in a database.

[1702] Step 3:

[1703] Input of questions and inquiries

[1704] Input: The user types a question or request into the app's message input field.

[1705] How it works: The device sends the user's input to the server, which then analyzes it using an AI engine and emotion engine.

[1706] Output: The server is ready to understand the question or request as a result of the analysis.

[1707] Step 4:

[1708] Analyzing input data and generating answers

[1709] Input: The server receives input such as questions and inquiries, as well as emotion data.

[1710] How it works: The server's emotion engine analyzes the user's emotional state, and the AI ​​engine generates the optimal answer, customizing the answer using past data and the user's current emotional state.

[1711] Output: The server sends the generated answer to the user's device.

[1712] Step 5:

[1713] Providing answers and collecting feedback

[1714] Input: The user receives and confirms the provided answer on the terminal.

[1715] Specific operation: The device displays the answer to the user, and the user enters feedback on the answer. The feedback is sent from the device to the server.

[1716] Output: The server stores the received feedback as analysis results to help improve accuracy later.

[1717] Step 6:

[1718] Big data analysis and its use for marketing information

[1719] Input: The server uses location information, usage history, emotional data, and feedback data collected from all users.

[1720] How it works: The server's big data analysis engine analyzes this data to discover user behavior patterns and trends, and generates information based on the insights gained to be used in marketing and product development.

[1721] Output: The server outputs trend analysis results and insights for marketing strategies.

[1722] Step 7:

[1723] Continuously updating the emotion engine and improving system accuracy

[1724] Input: User feedback and parsed data.

[1725] Specific operation: The server analyzes the feedback and collected emotion data and makes updates to continuously improve the accuracy of the emotion engine and the entire system.

[1726] Output: The server implements a new version of the emotion engine with improved accuracy, improving the overall system performance.

[1727] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1728] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1729] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1730] [Fourth embodiment]

[1731] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1732] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1733] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1734] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1735] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1736] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1737] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1738] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1739] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1740] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1741] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1742] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1743] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1744] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[1745] System Program and Processing Overview

[1746] User Login and Authentication

[1747] User: Enters email address and password to log in to a smartphone or specialized device. Device: Sends the entered authentication information to the server. Server: Checks against the database to determine whether authentication was successful, and if authentication is successful, generates a session ID and sends it to the device.

[1748] User information collection

[1749] User: After logging in, the system asks for consent to the collection of location information and usage history. If consent is given, data collection begins. Device: After obtaining consent, the collected data is sent to the server. Server: The collected data is analyzed to understand the user's characteristics.

[1750] Processing user requests

[1751] User: Uses the messaging function to input questions or inquiries into the system. Device: The input request is sent as text data to the server. Server: An AI engine is used to analyze the text data and generate the optimal answer.

[1752] Answers and Feedback

[1753] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends the feedback to the server. Server: Analyzes the feedback and improves the accuracy of the system.

[1754] Big data analysis and its use for marketing information

[1755] Server: Aggregates and analyzes data collected from all users. Identifies user behavior patterns and trends and uses them for marketing and product development.

[1756] Specific examples

[1757] User login and authentication examples

[1758] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[1759] Examples of user information collection

[1760] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[1761] Example of user request processing

[1762] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data and the user's current health status and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[1763] Specific examples of answers and feedback

[1764] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[1765] Specific examples of big data analysis and its use in marketing information

[1766] The server analyzes the data collected from all users and finds a trend, such as "Many users go to parks on weekends." Based on this, it develops related products and formulates an effective marketing strategy.

[1767] The above is a specific embodiment for carrying out the present invention. The present invention aims to provide consistent personalized support and make users' lives more comfortable.

[1768] The processing flow will be explained below.

[1769] User login and authentication process steps

[1770] Step 1:

[1771] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[1772] Step 2:

[1773] The device sends the entered email address and password to the server.

[1774] Step 3:

[1775] The server checks the email address and password against a database to determine whether authentication was successful.

[1776] Step 4:

[1777] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[1778] Step 5:

[1779] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[1780] User information collection process steps

[1781] Step 1:

[1782] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[1783] Step 2:

[1784] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[1785] Step 3:

[1786] The device sends the collected location information and usage history to the server.

[1787] Step 4:

[1788] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[1789] Processing steps for user request processing

[1790] Step 1:

[1791] The user enters a question or request into the message input field within the app.

[1792] Step 2:

[1793] The terminal transmits the input data from the user to the server.

[1794] Step 3:

[1795] The server uses an AI engine to analyze the input data and generate an appropriate answer.

[1796] Step 4:

[1797] The server sends the generated response to the terminal.

[1798] Step 5:

[1799] The terminal notifies the user of the received answer and allows the user to view it.

[1800] Response and feedback processing steps

[1801] Step 1:

[1802] The user enters feedback on the answers provided.

[1803] Step 2:

[1804] The terminal sends the user's feedback to the server.

[1805] Step 3:

[1806] The server stores the received feedback in a database and analyzes the feedback.

[1807] Step 4:

[1808] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[1809] Processing steps for big data analysis and its use in marketing information

[1810] Step 1:

[1811] The server aggregates the data collected from all users to create a large dataset.

[1812] Step 2:

[1813] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[1814] Step 3:

[1815] The server generates insights for marketing strategies and new product development based on the analysis results.

[1816] Step 4:

[1817] The server provides the generated insights to the marketing and product development teams.

[1818] Example 1

[1819] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1820] In modern society, systems that provide personalized support are required to meet the diverse needs of users. However, current systems do not effectively collect, analyze, or utilize user information, resulting in inconsistent user experiences and making it difficult to improve satisfaction. It is also difficult to effectively utilize collected data for marketing and product development.

[1821] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1822] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as the user's location information and usage history with the authenticated user's consent, means for analyzing the collected data and understanding the user's characteristics, means for analyzing questions and inquiries from the user using a generative AI model and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, and means for performing big data analysis using the accumulated data and utilizing it for marketing and product development. This allows for consistent, personalized support to be provided to users, improving the user experience, and making it possible to effectively utilize the data for marketing and product development.

[1823] "User authentication information" is information provided by a user to identify themselves and prove their access rights.

[1824] A "database" is a system for storing data in an organized manner and facilitating its retrieval and management.

[1825] "Location information" is data that indicates the user's current geographical location.

[1826] "Usage history" refers to data related to the operation history and behavior patterns of users when they use systems or applications.

[1827] A "collection method" is a method or device for capturing and storing specific data.

[1828] "Analytical tools" are methods and equipment used to analyze collected data and find meaning.

[1829] A "generative AI model" is an algorithm or software that uses artificial intelligence techniques to analyze data and generate answers or suggestions.

[1830] The "answer generation means" refers to a method or algorithm for creating an appropriate answer to a question or inquiry input by a user.

[1831] "Feedback" is data regarding ratings and opinions provided by users.

[1832] "Big data analysis" is a technology that analyzes large amounts of data to extract useful information.

[1833] "Marketing" refers to the activities and strategies used to promote the sale of products and services.

[1834] "Product development" is the process of designing and delivering new products and services.

[1835] "Privacy protection" refers to measures to prevent the leakage or misuse of personal information.

[1836] "Anonymization" is the process of processing data so that it cannot identify a specific individual.

[1837] A "session ID" is an identifier temporarily assigned to identify a session of a specific user.

[1838] This invention is a system for providing consistent, personalized support to users. This system supports users' daily lives by performing user authentication, information collection, question handling, feedback collection, and big data analysis all at once. The collected data is also used for marketing and product development.

[1839] User Login and Authentication

[1840] To log in to a smartphone or specialized device, a user enters an email address and password. The device sends the entered authentication information to the server. The server compares it with a database (e.g., MySQL, PostgreSQL) and determines whether the authentication was successful. If the authentication is successful, the server generates a session ID and sends it to the device. The user's login is confirmed when a "Login successful" message is displayed on the device.

[1841] User information collection

[1842] After logging in, the user will see a pop-up requesting consent to the collection of location information and usage history. If they agree, they click "Agree." The device will use its GPS function to obtain location information and collect usage history. The device will then send this data to the server. The server will analyze the received data and understand the user's characteristics. Specifically, the server will perform analysis to identify the locations the user frequently visits and app usage patterns.

[1843] Answering questions

[1844] Users use the messaging function to input questions or inquiries into the system. For example, they might type, "What exercise would you recommend today?" The device sends this request as text data to the server. The server then analyzes the text data using a generative AI model (e.g., BERT, GPT-3). Based on the analysis results, the server generates an appropriate answer. For example, the server might generate an answer such as, "The sun is strong today, so I recommend jogging in the evening." The server then sends the generated answer to the device and provides it to the user.

[1845] Feedback collection

[1846] The user enters feedback on the provided answer, for example, "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to improve the accuracy of the system. A specific example is that the server uses the feedback data to update the training dataset of a generative AI model.

[1847] Big data analysis and its use for marketing information

[1848] The server aggregates the data collected from all users and analyzes it using big data analysis tools (e.g., Hadoop, Spark). User behavior patterns and trends are extracted and used for marketing and product development. For example, a trend such as "many users visit the park on weekends" can be identified, and a new product promotion can be developed based on this.

[1849] The above is a specific embodiment of the present invention, which can provide consistent and personalized support to users and improve their quality of life.

[1850] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1851] User Login and Authentication

[1852] Step 1:

[1853] A user enters an email address and password to log in to a smartphone or specialized device. This is the input for the process.

[1854] Step 2:

[1855] The terminal sends the authentication information (email address and password) entered by the user to the server. This authentication information is the input for the process.

[1856] Step 3:

[1857] The server receives the authentication information and checks it against a database (e.g., MySQL, PostgreSQL). During this process, it compares the entered authentication information with the information in the database and outputs the result of the check.

[1858] Step 4:

[1859] The server determines whether authentication is successful, and if authentication is successful, generates a session ID. The generated session ID is output.

[1860] Step 5:

[1861] The server sends the generated session ID to the terminal, which stores the session ID and displays a "Login successful" message to the user, confirming that the user has logged in.

[1862] User information collection

[1863] Step 1:

[1864] After logging in, users will be shown a pop-up requesting consent to the collection of location information and usage history. This is the input for processing.

[1865] Step 2:

[1866] If the user agrees to the data collection, they click "Agree." The consent click is the input.

[1867] Step 3:

[1868] The device uses GPS to obtain the user's current location and collects app usage history, which are the inputs for processing.

[1869] Step 4:

[1870] The device sends the collected location information and usage history to the server. This transmission becomes the input.

[1871] Step 5:

[1872] The server analyzes the received data and understands the user's characteristics, such as the locations the user frequently visits and the app usage patterns. The results of this analysis are output.

[1873] Processing user requests

[1874] Step 1:

[1875] Users use the messaging function to input questions or inquiries into the system. For example, they might input, "What exercise would you recommend today?" This is the input for the process.

[1876] Step 2:

[1877] The terminal sends the input request as text data to the server, and the sent text data becomes the input.

[1878] Step 3:

[1879] The server analyzes the received text data using a generative AI model (e.g., BERT, GPT-3). This analysis is the data processing process. For example, it generates appropriate suggestions based on the content of the input request.

[1880] Step 4:

[1881] The server generates the optimal answer based on the analysis results, such as "The sun is strong today, so I recommend jogging in the evening."

[1882] Step 5:

[1883] The server sends the generated answer to the terminal and provides it to the user, who then confirms the answer.

[1884] Answers and Feedback

[1885] Step 1:

[1886] The user enters feedback on the provided answer, for example, "This advice was helpful." This is the input for the process.

[1887] Step 2:

[1888] The device sends feedback to the server, which then becomes the input.

[1889] Step 3:

[1890] The server analyzes the received feedback and extracts from it which parts of the system need improvement. This analysis is the data processing process.

[1891] Step 4:

[1892] The server uses the analyzed feedback to update models and algorithms to improve the accuracy of the system, which is the output.

[1893] Big data analysis and its use for marketing information

[1894] Step 1:

[1895] The server aggregates the data collected from all users, and the aggregated data is the input for processing.

[1896] Step 2:

[1897] The server analyzes the aggregated data using big data analysis tools (e.g., Hadoop, Spark). This analysis is the data processing process.

[1898] Step 3:

[1899] The server extracts user behavior patterns and trends from the analysis results. For example, the output may be a trend such as "Many users visit the park on weekends."

[1900] Step 4:

[1901] The server uses the analysis results for marketing and product development, for example, to plan promotions for new products based on these trends.

[1902] These processing steps enable the system to provide consistent, personalized support to users, improving the user experience and enabling effective use in marketing and product development.

[1903] (Application example 1)

[1904] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1905] Conventional content delivery systems lack the ability to provide personalized content recommendations based on user preferences and viewing history, making it difficult for users to access the content they desire. Furthermore, the mechanisms for properly collecting and analyzing user feedback and reflecting it in improving the system's accuracy were also inadequate. Furthermore, collected data was not anonymized, leaving many users with concerns about privacy.

[1906] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1907] In this invention, the server includes a means for recommending content based on the user's viewing history and genre of interest, a means for analyzing the user's input text and making optimal recommendations using a generative AI model, and a means for providing the user with recommendations generated using prompts. This enables the provision of optimal content tailored to the user's preferences and improves the accuracy of the system based on feedback analysis. Furthermore, privacy protection can be strengthened by anonymizing user data.

[1908] "User authentication" is the process of verifying the legitimacy of a user when the user accesses a system using authentication information such as an email address and password.

[1909] "Data collection" is the process of collecting information such as a user's location, usage history, and viewing history in order to store it in the system.

[1910] "Data analysis" is the process of analyzing collected data to understand user behavior patterns and characteristics.

[1911] "Question acceptance" is a process of accepting questions and inquiries from users as input.

[1912] "Answer generation" is the process of analyzing received questions and inquiries and providing the most appropriate answer to the user.

[1913] "Feedback collection" is the process of collecting evaluations and opinions from users and using them to improve the system.

[1914] A "viewing history" is a record of content that a user has viewed in the past.

[1915] "Content recommendation" is the process of providing content suitable for a user based on collected viewing history and genres of interest.

[1916] A "generative AI model" is a model that uses artificial intelligence to analyze user input and generate optimal answers or recommendations.

[1917] A "prompt sentence" is a sentence used to input specific instructions or questions to a generative AI model.

[1918] "Anonymization" is the process of protecting a user's personal information by transforming it so that it cannot be identified.

[1919] "Big data analysis" is the process of analyzing large amounts of data collected from a large number of users and utilizing it for marketing and product development.

[1920] A "session ID" is a unique identifier generated to identify a user after successful user authentication.

[1921] The present invention provides a system that performs user authentication, information collection, question handling, feedback collection, and big data analysis all at once to provide consistent, personalized support to users. This system is specifically implemented as follows.

[1922] User Login and Authentication

[1923] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session ID and sends it to the device. This completes user authentication.

[1924] Data collection

[1925] Authenticated users are asked to consent to the collection of their location information and viewing history. Once consent is given, the device will periodically acquire the user's location information using GPS or other functions and send it along with their viewing history to the server. The server then analyzes the collected data to understand the user's characteristics and behavioral patterns.

[1926] Question acceptance and answer generation

[1927] Users can input questions or inquiries using the device's interface. The device then sends this as text data to the server. The server then uses a generative AI model to analyze the user's input and generate optimal answers or content recommendations. For example, if a user types, "What movie should I watch tonight?", the server might generate an answer such as, "Considering the action movies you've recently watched and the reviews, I recommend 'Inception' tonight," and send it to the device.

[1928] Gathering feedback and improving system accuracy

[1929] When a user enters feedback on the answer they received, the device sends the feedback to the server, which analyzes the feedback and uses it as data to improve the accuracy of the system. This improves the accuracy of answers and content recommendations from the next time onwards.

[1930] Utilizing big data analysis and marketing information

[1931] The server performs big data analysis using the accumulated data from all users, which allows users to identify patterns and trends in their behavior and utilize this information for marketing and product development.

[1932] The specific hardware and software used include smartphones and head-mounted displays (HMDs), web frameworks such as Flask for communication with the server, SQLite for the database, and OpenAI GPT-3 as the generative AI model.

[1933] Prompt Sentence Examples

[1934] When a user asks the following question:

[1935] "Please tell me what movie I should watch today."

[1936] The server generates a response as follows:

[1937] "Based on your recent action movies and reviews, I recommend Inception tonight because it combines a tight storyline with the action you enjoy."

[1938] As described above, the present invention aims to provide consistent personalized support and make users' lives more comfortable.

[1939] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1940] Step 1:

[1941] User Authentication

[1942] The user opens a login screen on a device such as a smartphone or head-mounted display (HMD) and enters their email address and password. The entered authentication information is sent from the device to the server. The server compares the received authentication information with a database, and if authentication is successful, generates a session ID and sends it to the device. This authenticates the user.

[1943] Input: User's email address and password

[1944] Data processing: Database matching of authentication information

[1945] Output: Session ID

[1946] Step 2:

[1947] Data collection

[1948] Authenticated users are asked to consent to the collection of their location information and viewing history. If the user consents, the device will periodically acquire location information using GPS or other functions and send it along with their viewing history to the server. The server receives and stores this data, and analyzes it to understand the user's behavioral patterns and characteristics.

[1949] Input: User consent, GPS location, viewing history

[1950] Data processing: data collection and storage, analysis of behavioral patterns and characteristics

[1951] Output: Characteristic data

[1952] Step 3:

[1953] Questions accepted

[1954] The user inputs a question or request using the terminal interface. The input request is sent as text data from the terminal to the server, which then receives the text data.

[1955] Input: User's question or inquiry (text data)

[1956] Data processing: Receiving text data

[1957] Output: Unparsed text data

[1958] Step 4:

[1959] Answer generation

[1960] The server analyzes the received text data using a generative AI model. Based on the analysis results, it generates the most suitable answer and content recommendation for the user. For example, in response to the question, "What movie should I watch today?", it will recommend "Inception" based on the user's past viewing history and preferences. The generated answer is sent to the device in text format.

[1961] Input: Unparsed text data

[1962] Data processing: Text analysis, answer generation using generative AI models

[1963] Output: User-optimized answers and content recommendations (text format)

[1964] Step 5:

[1965] Providing answers to users

[1966] The terminal displays the answers received from the server to the user, allowing the user to obtain answers to their questions and content recommendations.

[1967] Input: Best answer or content recommendation from the server (text format)

[1968] Data processing: conversion to user display

[1969] Output: The displayed answer

[1970] Step 6:

[1971] Feedback collection

[1972] The user enters feedback on the answers provided. The feedback is sent from the device to the server, which receives and analyzes the feedback. This data is used to improve the accuracy of the system.

[1973] Input: User feedback (text format)

[1974] Data processing: receiving and analyzing feedback

[1975] Output: Data for accuracy improvement

[1976] Step 7:

[1977] Utilizing big data analysis and marketing information

[1978] The server aggregates data collected from all users and performs big data analysis. This allows users to identify patterns and trends in their behavior and use them in marketing and product development. For example, a pattern could be identified where "many users go to the park on weekends."

[1979] Input: Aggregated data for all users

[1980] Data processing: Big data analysis

[1981] Output: Behavioral patterns and trend information

[1982] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1983] This invention combines an emotion engine with a system that provides consistent personalized support to users to provide a more accurate and emotion-aware service. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[1984] System Program and Processing Overview

[1985] User Login and Authentication

[1986] User: Opens a smartphone or specialized device and enters an email address and password on the app's login screen. Device: The entered authentication information is sent to the server. Server: After checking the database and determining whether authentication was successful, if authentication was successful a session ID is generated and sent to the device.

[1987] User information collection

[1988] User: After logging in, the system asks for consent to the collection of location information and usage history, and if consent is given, data collection begins. Device: After obtaining consent, the system uses the GPS function to obtain current location information and periodically collects usage history and sends it to the server. Server: The received data is stored in a database and analyzed to understand the user's characteristics.

[1989] Processing user requests

[1990] User: Enters a question or inquiry into the message input field within the app. Device: Sends the input data from the user to the server. Server: Analyzes the input data using an AI engine and emotion engine, and generates the optimal answer taking emotions into account.

[1991] Emotion recognition by emotion engine

[1992] Terminal: When a user inputs a question or request for advice, the device collects the user's emotional data using methods such as voice recognition and facial expression analysis. Server: Analyzes the collected emotional data and understands the user's current emotional state. Based on the results, the device generates more appropriate answers and advice tailored to each individual user.

[1993] Answers and Feedback

[1994] Server: Sends the generated answer to the device. User: Checks the received answer and enters feedback into the device. Device: Sends feedback to the server. Server: Analyzes the feedback and emotion data to improve the accuracy of the system.

[1995] Big data analysis and its use for marketing information

[1996] Server: Integrates data collected from all users and performs big data analysis. Data analysis identifies user behavior patterns and trends, which are used for marketing and product development. Collected emotional data is also included to generate insights based on user emotions.

[1997] Specific examples

[1998] User login and authentication examples

[1999] A user opens the app and enters their email address and password on the login screen. The device sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session ID and sends it to the device. The user receives a "Login successful" message.

[2000] Examples of user information collection

[2001] After logging in, the user selects "Agree" on the consent screen for location information collection. The device uses GPS to obtain current location information and periodically collects and sends usage history to the server. The server analyzes this data to understand the user's frequently visited locations and app usage patterns.

[2002] Example of user request processing

[2003] The user types, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which takes into account past data, current health status, and emotional state determined by the emotion engine, and generates a response, "The weather is nice, so I recommend jogging," and sends it to the device. The user replies, "Thank you."

[2004] Specific examples of emotion recognition using emotion engines

[2005] When a user is feeling stressed, they ask, "Work hasn't been going well lately. What should I do?" The device detects the user's stress level through voice recognition and facial expression analysis, and sends that data to the server. The server then uses its emotion engine to generate advice to reduce stress, such as "How about listening to some relaxing music?", and provides it to the user.

[2006] Specific examples of answers and feedback

[2007] The user can then input feedback on the provided answer, such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve accuracy.

[2008] Specific examples of big data analysis and its use in marketing information

[2009] The server analyzes the data collected from all users and finds a trend that "many users visit parks on weekends." Based on the emotional data, it also finds that many users seek relaxation on weekends. Based on this, related products can be developed and effective marketing strategies can be formulated.

[2010] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[2011] The processing flow will be explained below.

[2012] User login and authentication process steps

[2013] Step 1:

[2014] The user opens their smartphone or specialized device and enters their email address and password on the app's login screen.

[2015] Step 2:

[2016] The device sends the entered email address and password to the server.

[2017] Step 3:

[2018] The server checks the email address and password against a database to determine whether authentication was successful.

[2019] Step 4:

[2020] If the server is successful in authentication, it generates a session ID and sends it to the terminal, otherwise it generates an error message.

[2021] Step 5:

[2022] The terminal displays the authentication result received from the server to the user, for example, a message indicating successful login or an error message.

[2023] User information collection process steps

[2024] Step 1:

[2025] After logging in, the user will be taken to a consent screen regarding the collection of location information and usage history, where they will select "Agree."

[2026] Step 2:

[2027] With consent, the device uses GPS to collect the user's current location information and periodically collects usage history.

[2028] Step 3:

[2029] The device sends the collected location information and usage history to the server.

[2030] Step 4:

[2031] The server stores the received data in a database and analyzes it to understand the user's characteristics and behavioral patterns.

[2032] Processing steps for user request processing

[2033] Step 1:

[2034] The user enters a question or request into the message input field within the app.

[2035] Step 2:

[2036] The terminal transmits the input data from the user to the server.

[2037] Step 3:

[2038] The server uses an AI engine and an emotion engine to analyze the input data and generate an appropriate answer taking emotions into account.

[2039] Step 4:

[2040] The server sends the generated response to the terminal.

[2041] Step 5:

[2042] The terminal notifies the user of the received answer and allows the user to view it.

[2043] Emotion recognition processing steps by the emotion engine

[2044] Step 1:

[2045] When the terminal inputs a user's question or inquiry, it collects emotional data using means such as voice recognition and facial expression analysis.

[2046] Step 2:

[2047] The device transmits the collected emotion data to the server.

[2048] Step 3:

[2049] The server analyzes the received emotional data and understands the user's current emotional state.

[2050] Step 4:

[2051] The server generates optimal answers and advice for the user based on the emotional data.

[2052] Response and feedback processing steps

[2053] Step 1:

[2054] The user enters feedback on the answers provided.

[2055] Step 2:

[2056] The device sends the feedback to the server.

[2057] Step 3:

[2058] The server stores the received feedback and emotional data in a database and analyzes it.

[2059] Step 4:

[2060] The server adjusts the AI ​​engine's algorithm based on the analysis results to improve the accuracy of future answers.

[2061] Processing steps for big data analysis and its use in marketing information

[2062] Step 1:

[2063] The server aggregates the data collected from all users to create a large dataset.

[2064] Step 2:

[2065] The server uses big data analysis tools to analyze the data and extract user behavior patterns and trends.

[2066] Step 3:

[2067] The server generates insights for marketing strategies and new product development based on the analysis results, including emotional data.

[2068] Step 4:

[2069] The server provides the generated insights to the marketing and product development teams.

[2070] Example 2

[2071] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2072] Conventional personalized support systems have difficulty in responding to user emotions, resulting in insufficient user satisfaction. In addition, they often cannot effectively analyze the collected data and utilize it for marketing and product development.

[2073] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving user authentication information and authenticating the user by comparing the authentication information with a database; means for collecting data such as the user's location information and usage history with the authenticated user's consent; means for analyzing the collected data and understanding the user's characteristics; means for accepting input of questions and inquiries from the user; means for analyzing the accepted questions and inquiries using a generative AI model and an emotion engine and generating answers that take emotions into consideration; means for providing the generated answers to the user; means for collecting and analyzing user emotion data; means for providing services adapted to the user's emotional state based on the analyzed emotion data; means for collecting and analyzing user feedback; means for improving system accuracy based on the analyzed feedback; and means for performing big data analysis using accumulated data and utilizing the results in marketing and product development. This makes it possible to provide consistent personalized support that takes users' emotions into consideration, improve user satisfaction, and generate effective marketing information.

[2074] "User Credentials" are the identification data a user uses to log into a system, typically a combination of email address and password.

[2075] "Database" means an information management system for managing and storing user authentication information and other data.

[2076] "Location information" is data indicating the user's current location obtained using GPS or other means.

[2077] "Usage history" refers to historical data of operations performed by a user within the system and functions accessed by the user.

[2078] A "generative AI model" is an artificial intelligence model used to analyze questions and inquiries from users and generate appropriate answers.

[2079] The "emotion engine" is an analytical engine that analyzes the user's emotional data and provides an appropriate response based on that emotion.

[2080] A "session ID" is an identifier temporarily assigned to a user who has been successfully authenticated, and is used to manage the continuation of the session.

[2081] "Big data analysis" is a processing method for analyzing large amounts of accumulated data and extracting useful information.

[2082] "Feedback" refers to the evaluation or opinion a user gives regarding the answers or services provided.

[2083] This invention combines an emotion engine and an AI engine into a system that provides consistent, personalized support to users. This system performs user authentication, information collection, question response, feedback collection, emotion recognition, and big data analysis all at once, providing comprehensive support for users' lives. The collected data can also be used for marketing and product development.

[2084] User Login and Authentication

[2085] A user uses a smartphone or specialized device to enter their email address and password into the app's login screen. The device sends the entered authentication information to the server, which checks it against a database to determine whether the authentication was successful. If authentication is successful, the server generates a session ID and sends it to the device. For example, if a user enters "user@example.com" and "password123" and authentication is successful, the server generates a session ID "abc123def456" and sends it to the device. The user receives a "Login successful" message.

[2086] User information collection

[2087] After a user logs in, the system asks for consent to the collection of location information and usage history. If the user consents, the device will use its GPS function to obtain current location information and periodically collect usage history and send it to the server. The server stores the received data in a database and analyzes it to understand the user's characteristics. For example, if the user consents to the collection of location information, the device will obtain latitude and longitude and periodically send this information to the server. The server will analyze this information to understand the user's behavioral patterns.

[2088] Processing user requests

[2089] The user enters a question or request into the message input field within the app. For example, "Please tell me what kind of exercise I should do today." The device sends this request to the server, which uses a generative AI model and emotion engine to analyze the input data and generate an optimal answer taking emotion into account. For example, the server generates a response such as "The weather is nice, so I recommend going for a jog" and sends it to the device.

[2090] Emotion recognition by emotion engine

[2091] When a user inputs a question or request for advice, the device collects the user's emotional data using means such as voice recognition and facial expression analysis. For example, when a user asks, "Work hasn't been going well lately. What should I do?", the device analyzes the tone of the voice and facial expression. The server analyzes the collected emotional data, understands the user's current emotional state, and generates answers and advice based on that. For example, it may make suggestions such as, "How about listening to some relaxing music?"

[2092] Answers and Feedback

[2093] The server sends the generated answer to the device, and the user confirms the answer they received. For example, the user enters feedback such as "This advice was helpful." The device then sends the feedback to the server, which analyzes it and uses it to further improve the accuracy of the system.

[2094] Big data analysis and its use for marketing information

[2095] The server integrates data collected from all users and performs big data analysis. For example, it analyzes user behavioral patterns and emotional data to find trends such as "Many users visit parks on weekends" or "Users want to refresh themselves on weekends." Based on this, it can develop marketing strategies and products and implement effective marketing.

[2096] Specific examples

[2097] User login and authentication examples

[2098] The user enters "user@example.com" and "password123", and the terminal sends this to the server. The server checks the database and, if authentication is successful, generates a session ID "abc123def456" and sends it to the terminal. The user receives a "Login successful" message.

[2099] Examples of user information collection

[2100] When a user agrees to the collection of location information, the device acquires latitude and longitude and sends them to a server, which analyzes this data to understand the user's behavioral patterns.

[2101] Example of user request processing

[2102] The user types, "Tell me what kind of exercise I should do today," and the device sends this to the server. The server uses a generative AI model and emotion engine to generate a response such as, "The weather is nice, so I recommend jogging," and sends it to the device.

[2103] Specific examples of emotion recognition using emotion engines

[2104] The user asks, "Work hasn't been going well lately. What should I do?" and the device collects emotional data through voice recognition and facial expression analysis. The server analyzes this data and generates a response such as, "How about listening to some relaxing music?" and sends it to the device.

[2105] Specific examples of answers and feedback

[2106] The user enters feedback into the device, such as "This advice was helpful," and the device sends it to the server, which analyzes the feedback and uses it to improve the accuracy of the system.

[2107] Specific examples of big data analysis and its use in marketing information

[2108] The server analyzes the data collected from all users and finds a trend, such as "Many users visit the park on weekends." This, along with emotional data, is used for marketing strategies and product development.

[2109] The above is a specific embodiment of the present invention that combines an emotion engine. The present invention aims to provide consistent personalized support that takes into account the user's emotions, making the user's life more comfortable.

[2110] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2111] Step 1:

[2112] The user enters their email address and password into the app's login screen using a smartphone or specialized device.

[2113] Input: A user enters the email address "user@example.com" and the password "password123".

[2114] Output: The authentication information sent to the device.

[2115] Step 2:

[2116] The terminal sends the entered authentication information to the server. The transmitted data is encrypted to ensure the security of the communication.

[2117] Input: The authentication information entered by the user.

[2118] Output: The encrypted authentication information sent to the server.

[2119] Step 3:

[2120] The server compares the received authentication information with the database and determines whether the authentication was successful.

[2121] Input: Encrypted authentication information sent from the device.

[2122] Output: Authentication success / failure and, if successful, the new session ID "abc123def456".

[2123] Step 4:

[2124] The server sends the authentication result and session ID to the terminal.

[2125] Input: Authentication result and session ID.

[2126] Output: Authentication result and session ID sent to the terminal.

[2127] Step 5:

[2128] The terminal receives the message from the server and displays "Login successful" to the user.

[2129] Input: Authentication result and session ID sent by the server.

[2130] Output: User is told "Login successful" and the session ID is displayed.

[2131] Step 6:

[2132] After logging in, the user is asked to consent to the collection of location information and usage history by the system, and agrees.

[2133] Input: User consent choice (e.g. clicking the "I agree" button).

[2134] Output: Start of terminal data collection.

[2135] Step 7:

[2136] The device uses GPS to obtain the current location and also periodically records app usage history.

[2137] Input: Consent given by the user.

[2138] Output: Obtained location information (latitude and longitude) and usage history data.

[2139] Step 8:

[2140] The device sends the location information and usage history it has acquired to the server. The data is encrypted and sent securely to the server.

[2141] Input: Captured location and usage data.

[2142] Output: The encrypted data sent to the server.

[2143] Step 9:

[2144] The server stores the received data in a database and analyzes it to understand the user's characteristics.

[2145] Input: Location and usage data sent from your device.

[2146] Output: Saving to database and analysis results (user behavior patterns and characteristics).

[2147] Step 10:

[2148] The user types a question or inquiry into a message input field within the app (e.g., "What exercise should I do today?").

[2149] Input: User-generated text for questions or inquiries.

[2150] Output: Input data ready for transmission.

[2151] Step 11:

[2152] The terminal transmits the input data from the user to the server.

[2153] Input: String data from the user.

[2154] Output: The question or question data sent to the server.

[2155] Step 12:

[2156] The server analyzes the input data using a generative AI model and emotion engine, and generates the optimal answer taking emotions into account.

[2157] Input: Questions and inquiries submitted by users, as well as sentiment data.

[2158] Data processing: Analyze the content of questions and consultations and emotional state.

[2159] Output: The best answer based on the analysis results (e.g., "The weather is nice, so I recommend going for a jog").

[2160] Step 13:

[2161] The server generates a response and sends it to the terminal.

[2162] Input: The generated answer data.

[2163] Output: The response data sent to the device.

[2164] Step 14:

[2165] The terminal receives the response from the server and displays it to the user.

[2166] Input: The response data sent from the server.

[2167] Output: Display of the answer to the user.

[2168] Step 15:

[2169] When the terminal inputs a user's question or inquiry, it collects the user's emotional data using means such as voice recognition and facial expression analysis.

[2170] Input: User's voice and facial expression data.

[2171] Output: Collected emotion data.

[2172] Step 16:

[2173] The device transmits the collected emotion data to a server.

[2174] Input: Collected emotion data.

[2175] Output: Emotion data sent to the server.

[2176] Step 17:

[2177] The server analyzes the collected emotional data to understand the user's current emotional state.

[2178] Input: Emotion data sent from the device.

[2179] Data processing: Analysis of emotion data.

[2180] Output: Analysis results about the user's emotional state.

[2181] Step 18:

[2182] The user reviews the answers provided and enters feedback into the device (e.g., "This advice was helpful").

[2183] Input: User-entered text feedback.

[2184] Output: Input data ready for transmission.

[2185] Step 19:

[2186] The device sends the feedback to the server.

[2187] Input: Feedback data entered by the user.

[2188] Output: Feedback data sent to the server.

[2189] Step 20:

[2190] The server analyzes the feedback it receives and uses it to improve the accuracy of the system.

[2191] Input: Feedback data sent from the device.

[2192] Data processing: Analysis of feedback data.

[2193] Output: Analysis results for improving system accuracy.

[2194] Step 21:

[2195] The server integrates the data collected from all users and performs big data analysis.

[2196] Input: All accumulated user data.

[2197] Data processing: Data analysis using big data analysis software.

[2198] Output: Analysis of user behavior patterns and trends.

[2199] Step 22:

[2200] The server uses the insights gained from the analysis results to inform marketing strategies and product development.

[2201] Input: Results of big data analysis.

[2202] Output: Marketing strategy and product development plan.

[2203] The system obtained in this way is capable of providing consistent personalized support that takes into account the user's emotions, contributing to improved user satisfaction and the generation of effective marketing information.

[2204] (Application example 2)

[2205] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2206] Conventional systems provide personalized support without considering the user's emotional state, resulting in a limited user experience and the inability to provide optimal advice or product recommendations based on emotions. Furthermore, there is a lack of analysis using user feedback and big data, which can make it difficult to improve the accuracy of the system or to formulate marketing strategies.

[2207] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2208] In this invention, the server includes means for receiving user authentication information and authenticating the user by comparing the authentication information with a database, means for collecting data such as location information and usage history with the user's consent, means for analyzing the collected data and understanding the user's characteristics, means for accepting questions and inquiries from the user, means for analyzing the accepted questions and inquiries and generating answers, means for providing the generated answers to the user, means for collecting and analyzing feedback from the user, means for improving the accuracy of the system based on the analyzed feedback, means for performing big data analysis using the accumulated data and utilizing it for marketing and product development, and means for analyzing the user's emotional state using an emotion engine and suggesting optimal products based on the user's emotions. This enables highly accurate personalized support that takes user emotions into consideration.

[2209] definition statement

[2210] "User Credentials" are information used to verify a user's identity, typically including an email address and password.

[2211] A "database" is a system for storing and managing data in an organized manner. It stores user authentication information and collected data.

[2212] "User consent" is the act of a user permitting a system to collect and use their data. This is important from the perspective of protecting user privacy.

[2213] "Location information" is data that indicates a user's current location. It is often collected using GPS functionality.

[2214] "Usage history" refers to behavioral data when a user uses a system or application, including the pages visited and the duration of use.

[2215] "Question or consultation input" refers to a question or message requesting advice that a user inputs into the system.

[2216] "Generating an answer" means that the system creates an appropriate answer to a question or inquiry from a user.

[2217] "Feedback" refers to the evaluation or opinion a user gives in response to a received answer.

[2218] "Big data analytics" is an analytical technique that integrates large amounts of data and finds patterns and trends.

[2219] "Marketing and product development" refers to the activities of planning and developing new products and services based on collected data.

[2220] An "emotion engine" is a technology that analyzes a user's emotional state and responds optimally based on that.

[2221] A "session ID" is a temporary identifier that is generated when a user logs into a system. It is provided to the user after successful authentication.

[2222] "Anonymization" is the process of removing or obscuring personally identifiable information from collected data in order to protect user privacy.

[2223] MODE FOR CARRYING OUT THE INVENTION

[2224] The present invention includes an emotion-aware virtual shopping assistant system. This system performs user authentication, information collection, question handling, feedback collection, emotion recognition, and big data analysis all at once, providing consistent personalized services that take user emotions into consideration. Specific implementation methods for this system are described in detail below.

[2225] Hardware and software used

[2226] 1. Hardware:

[2227] Smartphone

[2228] Specialized devices (e.g. tablets)

[2229] 2. Software:

[2230] requests (Python HTTP request library)

[2231] Server-side API (e.g. Django, Flask)

[2232] Emotion engine (e.g. Microsoft Azure Emotion API)

[2233] Big data analysis engines (e.g., Apache Hadoop, Apache Spark)

[2234] Program processing

[2235] User Login and Authentication

[2236] The server receives the email address and password entered by the user on a smartphone or specialized device and performs authentication by comparing them with a database. If authentication is successful, the server generates a session ID and sends it to the device. Through this process, user authentication is established.

[2237] User information collection

[2238] With the consent of the authenticated user, the server uses the device's GPS function to collect current location information and transmits usage history to the server. The collected data is analyzed using a big data analysis engine on the server side to understand the user's usage patterns and characteristics.

[2239] Handling user inquiries and enquiries

[2240] The device sends a message from the user to the server, which analyzes the message, uses an emotion engine to understand the user's emotional state, and generates an optimal answer. The generated answer is sent to the device and provided to the user.

[2241] Feedback collection and analysis

[2242] When a user enters feedback on a given answer, the device sends the feedback to the server, which analyzes the collected feedback and uses it to improve the accuracy of the system.

[2243] Big data analysis and its use for marketing information

[2244] The collected data is integrated on a server and analyzed by a big data analysis engine. The analysis results are used in marketing and product development, and insights based on user sentiment are generated.

[2245] Specific examples

[2246] User Questions

[2247] When a user types, "I've been feeling tired from work lately, and I want a product that helps me relax," the device sends this request to the server. The server uses an emotion engine to detect "fatigue" and "stress" and suggests products that will help you relax (e.g., an aroma diffuser).

[2248] Example prompt sentence:

[2249] Based on the following specifications, please generate a program to create a virtual shopping assistant system that combines an emotion engine.

[2250] The user enters their email address and password on the login screen.

[2251] Authentication information is sent to the server, and a session ID is generated upon successful authentication.

[2252] After logging in, location and usage data is collected

[2253] Recognizes emotions from user input and suggests optimal products

[2254] Use the Python language to send HTTP requests using the requests library.

[2255] Assumes Django or Flask as the server-side API

[2256] Examples:

[2257] If a user types, "I've been tired from work lately and I want a product that helps me relax," the emotion engine will detect "fatigue" and "stress" and suggest products that will help you relax (e.g., an aroma diffuser).

[2258] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2259] Specific explanation of processing steps

[2260] Step 1:

[2261] User Login and Authentication

[2262] Input: The user enters their email address and password into their smartphone.

[2263] Specific operation: The device sends the entered authentication information to the server. The server checks the information against a database, determines whether the authentication was successful, and if the authentication was successful, generates a session ID and sends it to the device.

[2264] Output: If authentication is successful, the server generates a session ID and the user receives a "Login successful" message.

[2265] Step 2:

[2266] User information collection

[2267] Input: An authenticated user consents to the collection of location and usage information.

[2268] Specific operation: The device periodically collects its current location information using the GPS function and sends it along with its usage history to the server, which then stores the received data in a database.

[2269] Output: The server periodically receives location information and usage history and stores the data in a database.

[2270] Step 3:

[2271] Input of questions and inquiries

[2272] Input: The user types a question or request into the app's message input field.

[2273] How it works: The device sends the user's input to the server, which then analyzes it using an AI engine and emotion engine.

[2274] Output: The server is ready to understand the question or request as a result of the analysis.

[2275] Step 4:

[2276] Analyzing input data and generating answers

[2277] Input: The server receives input such as questions and inquiries, as well as emotion data.

[2278] How it works: The server's emotion engine analyzes the user's emotional state, and the AI ​​engine generates the optimal answer, customizing the answer using past data and the user's current emotional state.

[2279] Output: The server sends the generated answer to the user's device.

[2280] Step 5:

[2281] Providing answers and collecting feedback

[2282] Input: The user receives and confirms the provided answer on the terminal.

[2283] Specific operation: The device displays the answer to the user, and the user enters feedback on the answer. The feedback is sent from the device to the server.

[2284] Output: The server stores the received feedback as analysis results to help improve accuracy later.

[2285] Step 6:

[2286] Big data analysis and its use for marketing information

[2287] Input: The server uses location information, usage history, emotional data, and feedback data collected from all users.

[2288] How it works: The server's big data analysis engine analyzes this data to discover user behavior patterns and trends, and generates information based on the insights gained to be used in marketing and product development.

[2289] Output: The server outputs trend analysis results and insights for marketing strategies.

[2290] Step 7:

[2291] Continuously updating the emotion engine and improving system accuracy

[2292] Input: User feedback and parsed data.

[2293] Specific operation: The server analyzes the feedback and collected emotion data and makes updates to continuously improve the accuracy of the emotion engine and the entire system.

[2294] Output: The server implements a new version of the emotion engine with improved accuracy, improving the overall system performance.

[2295] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2296] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicat...

Claims

1. means for receiving user authentication information and authenticating the user by matching the authentication information with a database; A means for collecting data such as user location information and usage history with the consent of the authenticated user; A means for analyzing the collected data and understanding the characteristics of users; A means for accepting input of questions and inquiries from users; A means for analyzing received questions and inquiries and generating answers; means for providing the generated answer to the user; a means for collecting and analyzing user feedback; a means for improving the accuracy of the system based on the analyzed feedback; and The accumulated data will be used for big data analysis and utilized for marketing and product development. A system including:

2. 2. The system of claim 1, further comprising: means for generating a session ID and providing the session ID to the user upon successful authentication of the user.

3. The system according to claim 1 , further comprising means for anonymizing data to protect privacy when collecting data such as user location information and usage history.

Images