system

Abstract

We provide the system. [Solution] A means for receiving inquiry data entered by the user, The means for pre-processing the received query data, Based on the pre-processed data described above, a means of analyzing the problem using a database or AI model and searching for a solution is provided. The above solution is to be sent back to the user's device, A means of receiving user feedback on the above solution, Based on the user feedback mentioned above, we will provide additional support options. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present invention relates to a system that can receive support from a smartphone or other devices without visiting a store. Conventional support systems require users to go directly to the store, which is inconvenient for users who are far away or have physical limitations. Also, since the information for problem-solving is dispersed, it has been difficult for users to solve problems on their own. Therefore, there is a need for a support system that is more convenient for users and can solve problems efficiently.

Means for Solving the Problems

[0005] To solve these problems, the present invention provides the following means. First, a means for receiving inquiry data entered by the user is provided. Next, a means for pre-processing this received inquiry data is provided. Furthermore, a means for analyzing the problem using a database or AI model based on this pre-processed data and searching for a solution is provided. The searched solution is returned to the user's terminal. Furthermore, a means for receiving feedback from the user is provided, and a means for presenting additional support options based on the feedback is provided. With this system, users can efficiently solve problems from home, and support that is easy to use is available even for users who are far away or have physical limitations.

[0006] A "user" refers to an individual or organization that uses this system to request inquiries or support.

[0007] "Input data" refers to information that users provide to the system, including inquiries and problems they have encountered.

[0008] "Preprocessing" refers to the process of cleaning and normalizing data entered by users in order to make it easier to analyze.

[0009] A "database" refers to an information aggregation device that stores information and solutions for analyzing query data.

[0010] An "AI model" refers to an algorithm or program that uses artificial intelligence to analyze input data and search for an appropriate solution.

[0011] "Analysis" refers to the process of understanding input data, identifying problems, and finding appropriate solutions.

[0012] "Solution" refers to the specific actions or instructions provided in response to a user's inquiry or problem.

[0013] "Terminal" refers to any device used by a user to access this system, such as a smartphone or computer.

[0014] "Feedback" refers to the reactions and results that users provide regarding a solution.

[0015] "Additional support options" refer to further assistance provided when the initial troubleshooting methods fail to resolve the issue. These include options such as live chat and remote access. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. This system includes a series of means for providing quick and accurate solutions to user problems and inquiries.

[0038] Receive user input

[0039] First, the user accesses the system and enters the problem or question they need support for. For example, they might enter, "My smartphone screen goes dark." This triggers the system to receive the user's problem.

[0040] Server analysis of the problem

[0041] Next, the terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[0042] Providing a response

[0043] Once the server finds a solution, that information is sent back to the device. The device then displays the solution received from the server to the user. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[0044] Follow-up

[0045] The user tries the suggested solution and provides feedback on the results. This feedback is then sent back to the server from the device. If the user reports that the issue is "not resolved," the server offers additional support options, including live chat and remote access.

[0046] Specific example

[0047] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[0048] 1. The user enters "My smartphone won't charge."

[0049] 2. The terminal sends this input to the server.

[0050] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0051] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[0052] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[0053] 6. The server receives this feedback and offers live chat as an additional support option.

[0054] In this way, the system provides a series of means for users to efficiently solve problems from their homes. It is easy to use and provides convenient support for users who live far away or who have physical limitations.

[0055] The following describes the processing flow.

[0056] Step 1:

[0057] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark."

[0058] Step 2:

[0059] The terminal sends user input to the server. This process is carried out via HTTP requests.

[0060] Step 3:

[0061] The server preprocesses the received query data. This preprocessing includes removing unnecessary spaces and special characters.

[0062] Step 4:

[0063] The server analyzes the problem using databases and AI models based on pre-processed data. Based on the analysis, an appropriate solution is found.

[0064] Step 5:

[0065] The server returns the searched solution to the user's terminal. This process also takes place via an HTTP response.

[0066] Step 6:

[0067] The device displays the received solution to the user. For example, it might suggest a specific solution such as "Disable automatic brightness adjustment in settings."

[0068] Step 7:

[0069] The user tries to find a solution. For example, they open the settings menu and disable automatic brightness adjustment.

[0070] Step 8:

[0071] The user provides feedback on the results. They answer "Yes" or "No" to the question, "Has the problem been resolved?".

[0072] Step 9:

[0073] The device sends user feedback to the server. This process is also carried out via HTTP requests.

[0074] Step 10:

[0075] The server receives feedback from the user, and if the feedback is "no," it offers additional support options. For example, live chat or remote access options may be provided.

[0076] Step 11:

[0077] The user selects additional support options. For example, they might decide to use live chat.

[0078] Step 12:

[0079] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[0080] This series of steps allows users to efficiently resolve problems from home. The improved speed and convenience of support significantly enhances the user experience.

[0081] (Example 1)

[0082] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0083] In modern society, there is a growing demand for users to receive technical support remotely from their homes, requiring rapid and accurate problem analysis and solutions. However, traditional systems often rely on manual processes for processing inquiry data and analyzing problems, making them inefficient and hindering user satisfaction. Furthermore, responding quickly to user feedback is difficult. A system is needed to improve this situation and provide remote support efficiently and effectively.

[0084] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0085] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, and means for presenting additional support options based on the user's feedback. This enables users to receive quick and accurate support remotely without having to visit a store. Furthermore, by providing additional support options based on user feedback, higher user satisfaction can be achieved.

[0086] "Inquiry data" refers to information related to problems or questions that users submit to the system.

[0087] "Preprocessing" is the process of removing unnecessary spaces and special characters from received data and preparing it for analysis.

[0088] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to analyze data, identify problems, and search for solutions.

[0089] A "database" is a collection of information that is structured, stored, and made searchable, containing relevant information such as user inquiries and past solutions.

[0090] A "solution" refers to the specific steps or instructions provided to resolve a user's inquiry.

[0091] A "terminal" is an electronic device used by a user to access a support system and receive solutions.

[0092] "Feedback" refers to information that users send back to the system, such as their evaluation of the results of the proposed solutions or any additional requests.

[0093] "Additional support options" refer to further assistance or services provided to users when the initial troubleshooting method fails to resolve their problem.

[0094] "Real-time communication" refers to methods that allow users and support staff to interact instantly, such as live chat and voice calls.

[0095] "Remote operation support" is a support method in which a support staff member remotely accesses a user's electronic device and performs operations on their behalf.

[0096] This invention relates to a system for users to receive remote support from their homes. Embodiments of this system are described in detail below.

[0097] The system's basic configuration includes three main elements: users, terminals, and servers. Users access the system via the internet using electronic devices such as smartphones and computers. Terminals connect to the system through web browsers or dedicated applications, receiving user input and sending it to the server. The server processes the received data, analyzes it using generative AI models, and provides solutions.

[0098] Hardware and software to be used

[0099] 1. User:

[0100] Users will use typical smartphones (e.g., Android® devices, iPhone®) or computers (e.g., Windows PCs, Macs). These devices require an internet connection.

[0101] 2. Terminal:

[0102] The device uses a web browser (e.g., Google Chrome®, Safari) or a dedicated application (e.g., a custom support app). These are responsible for receiving user input and communicating with the server.

[0103] 3. Server:

[0104] The servers run in a cloud environment (e.g., AWS® EC2 instances, Microsoft® Azure® VMs) and process and analyze query data. The software used includes the following:

[0105] Preprocessing: Python libraries (e.g., NLTK, SpaCy)

[0106] Analysis: Generative AI models (e.g., OpenAI® GPT-4®, Google® BERT)

[0107] Databases: MySQL (registered trademark), PostgreSQL

[0108] Specific actions

[0109] 1. Receive user input.

[0110] Users access the support portal using their smartphones or computers. For example, a user opens a browser and visits the support portal's URL.

[0111] The user enters their problem or question into the input form. For example, they might enter, "My smartphone screen goes dark."

[0112] 2. Sending input data

[0113] The terminal sends the entered data to the server. This communication uses the HTTPS protocol.

[0114] 3. Data preprocessing

[0115] When the server receives data, it first performs preprocessing. For example, it might use a Python library to remove unnecessary spaces and special characters from the input data.

[0116] 4. AI-based analysis

[0117] The server inputs pre-processed data into a generating AI model to identify problems and search for solutions. It also searches databases to supplement relevant information.

[0118] 5. Providing solutions

[0119] The server sends the identified solution back to the terminal. This information is exchanged in JSON format.

[0120] The device displays the received solution to the user. For example, it may show specific steps such as "Disable automatic brightness adjustment in settings."

[0121] 6. Feedback and additional support

[0122] The user tries the suggested solution and provides feedback on the results. For example, if the problem is not resolved, they can enter "Not resolved" in the form and submit it.

[0123] The device sends this feedback to the server.

[0124] The server receives feedback and provides additional support options as needed, including real-time communication and remote assistance.

[0125] Specific example

[0126] This shows the processing flow when a user enters a problem such as "My smartphone won't charge."

[0127] 1. The user enters "My smartphone won't charge."

[0128] 2. The terminal sends this input to the server.

[0129] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0130] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[0131] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[0132] 6. The server receives this feedback and provides real-time communication as an additional support option.

[0133] Example of a prompt:

[0134] User: My smartphone won't charge.

[0135] Server: Please check the charger connection.

[0136] User: I tried it, but it didn't solve the problem.

[0137] Server: Please try using a different cable. If that doesn't solve the problem, we will provide additional support via real-time communication.

[0138] This provides users with a series of tools to efficiently solve problems from home, making it easier to use and providing convenient support for users who live far away or have physical limitations.

[0139] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0140] Step 1:

[0141] The user enters information.

[0142] Users access the support portal via smartphones or computers.

[0143] Input: The user accesses the support portal and enters their inquiry. For example, they might enter, "My smartphone screen is dimming."

[0144] Output: User input is sent to the system via an HTML form.

[0145] Step 2:

[0146] The terminal receives input data and sends it to the server.

[0147] The terminal receives user input and sends it to the server.

[0148] Input: Inquiry data entered by the user in an HTML form.

[0149] Output: JSON formatted data sent to the server via the HTTPS protocol.

[0150] In terms of specific operations, a web browser or dedicated application converts the input data into an appropriate format and sends it to the server.

[0151] Step 3:

[0152] The server performs data preprocessing upon receiving data.

[0153] The server preprocesses the data it receives.

[0154] Input: Query data in JSON format sent from the terminal.

[0155] Output: Preprocessed data (data suitable for analysis, with unnecessary spaces and special characters removed).

[0156] Specifically, data cleaning is performed using Python's NLTK and SpaCy.

[0157] Step 4:

[0158] The server generates data and analyzes it using an AI model.

[0159] The server generates pre-processed data, which is then input into an AI model to identify problems and search for solutions.

[0160] Input: Pre-processed query data.

[0161] Output: Solutions identified by a generative AI model (e.g., OpenAI GPT-4).

[0162] Specifically, the server sends API requests to the generated AI model to analyze the problem and search for solutions.

[0163] Step 5:

[0164] The server searches the database to retrieve supplementary information.

[0165] The server searches the database and retrieves information to complement the solutions obtained from the generated AI model.

[0166] Input: Candidate solutions obtained from a generative AI model.

[0167] Output: Detailed information related to the optimal solution.

[0168] The server queries the database (MySQL, PostgreSQL) to retrieve additional relevant information.

[0169] Step 6:

[0170] The server sends the solution back to the terminal.

[0171] The server sends the identified solution back to the terminal.

[0172] Input: Generative AI models and solutions obtained from databases.

[0173] Output: Data in JSON format containing the solution to be displayed to the user.

[0174] Specifically, the server generates JSON data and sends it back to the terminal via the HTTPS protocol.

[0175] Step 7:

[0176] The device displays the solution to the user.

[0177] The terminal displays the solution it received from the server to the user.

[0178] Input: Resolution data in JSON format sent from the server.

[0179] Output: Solutions displayed in a user-friendly format.

[0180] A web browser or dedicated application parses the JSON data and displays the solution on the screen.

[0181] Step 8:

[0182] Users provide feedback

[0183] Users try the solutions and provide feedback on the results.

[0184] Input: Results and feedback on solutions attempted by the user.

[0185] Output: User feedback data is sent to the system.

[0186] Specific actions include entering "Not resolved" in the feedback form.

[0187] Step 9:

[0188] The server receives feedback and provides additional support.

[0189] The server receives user feedback and presents additional support options.

[0190] Input: User feedback data.

[0191] Output: Additional support options (real-time communication and remote operation assistance).

[0192] Specifically, the server will use the AI ​​model again to analyze the situation and provide the most suitable additional support options.

[0193] This allows the system to provide quick and accurate solutions to user inquiries, as well as offer additional support as needed.

[0194] (Application Example 1)

[0195] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0196] Traditional customer support systems often required users to visit physical stores, which was inconvenient, especially for users living far away or those with physical limitations. Even when remote support systems existed, they frequently lacked the necessary information and tools to resolve problems. This resulted in decreased support efficiency and lower customer satisfaction.

[0197] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0198] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, means for presenting additional support options based on the user's feedback, and means for the user to receive remote support regarding physical stores via their smartphone. This makes it possible for users to quickly and accurately receive inquiries and support regarding physical stores remotely, even from home.

[0199] "User-entered inquiry data" refers to information that users enter into the system when seeking support, and includes data that details the question or problem.

[0200] "Preprocessing" is the process of removing unnecessary spaces and special characters from received query data and converting it into a format suitable for data analysis.

[0201] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and predict how to solve a problem based on a specific algorithm.

[0202] A "database" is a collection of information where query data and solutions are stored, and which can be used for searching and analysis.

[0203] "Searching for solutions" is the process of identifying appropriate solutions using databases and generative AI models based on pre-processed data.

[0204] "Returning the solution to the user's device" refers to the process of sending the identified solution to the user's device for display.

[0205] "User feedback" refers to the opinions and results that users provide in response to the solutions offered.

[0206] "Additional support options" are further support measures provided based on user feedback, such as live chat and remote access.

[0207] "Remote support" refers to a service that allows users to receive support via the internet without needing to visit a physical store.

[0208] This invention relates to a system that allows users to receive remote support via their smartphones without having to visit a physical store. This system receives user inquiry data, preprocesses that data, uses a generated AI model and database to analyze the problem and search for solutions, and finally provides the user with a solution.

[0209] Hardware and software to be used

[0210] 1. Hardware

[0211] Smartphone: Used as a terminal for users to enter inquiry data.

[0212] Server: Receives, preprocesses, analyzes, and processes user feedback.

[0213] 2. Software

[0214] Smartphone application: An application (iOS, Android) for entering details of questions or problems and receiving solutions.

[0215] Server-side AI model: Data analysis and prediction of problem-solving methods (TENSORFLOW®, PyTorch).

[0216] Database: Stores and searches user query history and solutions (MySQL, PostgreSQL).

[0217] System operation

[0218] 1. User input of inquiry data

[0219] The user opens the smartphone application and enters inquiry data. For example, they might enter a question such as, "I don't know how to return a product."

[0220] 2. Receiving and preprocessing data by the server

[0221] The smartphone application sends the entered query data to the server. The server first preprocesses the data, removing unnecessary spaces and special characters.

[0222] 3. Analysis using generative AI models

[0223] The pre-processed data is input into a generating AI model, where the problem is analyzed and solutions are searched for.

[0224] 4. Providing solutions

[0225] The server searches the database for the best solution based on the analysis results and sends it back to the user's smartphone application. For example, a solution such as "Please access the returns page, fill out the returns form, and submit it" might be displayed.

[0226] 5. Receiving feedback and providing additional support

[0227] The user tries the suggested solution and provides feedback on the results. The server receives the feedback and provides additional support options such as live chat or remote access as needed.

[0228] Examples of specific cases and prompt statements

[0229] Specific example:

[0230] If a user enters "I don't know how to return the product," the server preprocesses the data and uses a generation AI model to search for a solution such as "Please access the returns page, fill out the return form, and submit it," and displays it on the user's smartphone.

[0231] Example of a prompt:

[0232] "User input: 'I don't know how to return the product' -> Provide a solution"

[0233] In this way, users can receive quick and accurate inquiries and support regarding physical stores remotely, even from the comfort of their homes. This system is particularly useful for users who live far away or who have physical limitations.

[0234] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0235] Step 1:

[0236] The user launches the smartphone application and enters inquiry data. For example, the user enters "I don't know how to return a product" into the text field and presses the submit button. The input data (text) is retrieved by the app and sent to the next step.

[0237] Step 2:

[0238] The device (smartphone) sends user input data to the server. The transmitted data is received by the server and recorded in the log. The input includes text data, and the output is ready for preprocessing.

[0239] Step 3:

[0240] The server preprocesses the query data it receives. This preprocessing includes removing unnecessary spaces and special characters. Specifically, it removes unwanted parts from the text using regular expressions, etc. The input includes the user's raw data, and the output is clean query data.

[0241] Step 4:

[0242] Preprocessed data is input to a generating AI model on the server side. The generating AI model analyzes the preprocessed, clean data and predicts relevant solutions. Specifically, the AI ​​model (such as TensorFlow or PyTorch) analyzes the input data and outputs the corresponding solutions. The input includes preprocessed, clean data, and the output is the solutions.

[0243] Step 5:

[0244] The server references the database to supplement the solution obtained from the generated AI model with detailed data. For example, a solution such as "Access the returns page, fill out the return form, and submit it" might be identified. The input includes the output of the AI ​​model and information from the database, and the output provides a specific solution.

[0245] Step 6:

[0246] The server sends the identified solution back to the user's smartphone. The details of the solution are displayed on the user's device. The input includes data on the specific solution, and the output is the solution displayed on the user's device.

[0247] Step 7:

[0248] The user tries the suggested solution and provides feedback on the results. For example, they might enter feedback such as, "I accessed the returns page, filled out the return form, and submitted it, but it was not completed." The input includes the text of the feedback, and the output is feedback data sent to the server.

[0249] Step 8:

[0250] The server receives and analyzes user feedback. It then presents additional support options as needed. For example, it might generate a link to offer the user a live chat option. The input includes the feedback content, and the output presents additional support options.

[0251] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0252] This invention relates to a system that allows users to receive remote support for their smartphones or computers without having to visit a store. In particular, it provides a configuration that combines an emotion engine that recognizes the user's emotions and responds accordingly.

[0253] Receive user input

[0254] Users access the system and enter problems or questions they need support for. For example, they might enter, "My smartphone screen goes dark." The user's input is also sent to an emotion engine for sentiment analysis.

[0255] Server analysis of the problem

[0256] The terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Simultaneously, the emotion engine analyzes the user's input to determine their emotions. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[0257] Utilizing the Emotion Engine

[0258] The emotion engine recognizes the user's emotional state and automatically adjusts solutions and responses accordingly. For example, if the user is experiencing anger or frustration, solutions will be provided more quickly, and live chat will be prioritized as an additional support option.

[0259] Providing a response

[0260] Once the server finds a solution, that information, along with the results of the emotion engine's analysis, is sent back to the user's device. The device then displays the solution received from the server and appropriate countermeasures based on the user's emotion. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[0261] Follow-up

[0262] The user tries the suggested solution and provides feedback on the results. This feedback is sent back to the server from the device, and the sentiment engine is updated. If the user provides feedback that the problem is "not resolved," the server will quickly provide, for example, live chat or remote access, based on the sentiment engine's analysis.

[0263] Specific example

[0264] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[0265] 1. The user types "My smartphone won't charge," and the emotion engine recognizes "confusion."

[0266] 2. The terminal sends this input to the server.

[0267] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0268] 4. The server sends the solution and the sentiment engine's analysis results back to the terminal, which then displays them to the user.

[0269] 5. The user tries a solution and provides feedback that it "does not solve the problem," and the emotion engine recognizes "frustration."

[0270] 6. The server receives this feedback and, based on the sentiment engine's analysis results, quickly provides live chat.

[0271] This series of steps allows users to efficiently resolve problems from home and receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[0272] The following describes the processing flow.

[0273] Step 1:

[0274] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark." This input is sent to the emotion engine, which analyzes the user's emotions.

[0275] Step 2:

[0276] The terminal sends the input data from the user to the server together with the emotion engine. The input data includes the user's question and the content of the problem.

[0277] Step 3:

[0278] The server preprocesses the received input data. The preprocessing includes removing unnecessary spaces and special characters. At the same time, emotion analysis by the emotion engine is carried out.

[0279] Step 4:

[0280] The emotion engine analyzes the emotion from the user's input and recognizes the emotional state such as "confusion", "irritation", "uneasiness", etc. It provides the recognized emotional state to the server.

[0281] Step 5:

[0282] Based on the preprocessed data and the analysis results of the emotion engine, the server analyzes the problem using a database or an AI model. As a result of the analysis, it searches for an appropriate solution method.

[0283] Step 6:

[0284] The server returns the searched solution method to the user's terminal together with the analysis results of the emotion engine. For example, it includes a specific instruction such as "Disable automatic brightness adjustment from settings".

[0285] Step 7:

[0286] The terminal displays the received solution method and the corresponding measures according to the emotion to the user. The user tries the presented solution method.

[0287] Step 8:

[0288] After the user tries the solution method, the user inputs feedback on the result. For example, the user answers "yes" or "no" to the question "Has the problem been solved?".

[0289] Step 9:

[0290] The device sends user feedback to the server. This feedback includes information about the success or failure of the resolution and the user's emotional state.

[0291] Step 10:

[0292] The server receives feedback from the user, and if the feedback is "no," it provides additional support options. In this process, it takes into account the results of the emotion engine's analysis; for example, if it recognizes "frustration," it prioritizes offering live chat.

[0293] Step 11:

[0294] The user selects live chat as an additional support option. The device sends that selection to the server.

[0295] Step 12:

[0296] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[0297] Through this series of steps, users can not only efficiently resolve problems from home but also receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[0298] (Example 2)

[0299] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0300] In a system that provides remote support, there is a problem that it is unable to recognize the user's emotions and take appropriate actions accordingly, resulting in a decline in the user experience. Additionally, in conventional support systems, there is also an issue that it is difficult to provide prompt additional support based on the user's feedback.

[0301] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0302] In this invention, the server includes means for receiving inquiry data input by the user, means for preprocessing, and means for analyzing the user's emotions using an emotion analysis engine. As a result, it becomes possible to perform highly accurate problem analysis and search for solution methods according to the user's emotional state. Further, it includes means for analyzing problems using a database and a generative AI model, searching for solution methods, means for returning countermeasures according to the solution methods and emotions to the user's terminal, and means for providing additional support options based on the user's feedback and emotion analysis results. Thereby, the user experience is improved and efficient and prompt problem-solving becomes possible.

[0303] The "user" refers to a person who accesses the system and inputs inquiry data.

[0304] The "inquiry data" refers to information regarding problems or questions for which the user inputs a need for support.

[0305] The "terminal" refers to an information device used by the user to access the system.

[0306] The "server" refers to a main computer system that receives inquiry data, performs preprocessing, analysis, search for solution methods, emotion analysis, and further processing of feedback.

[0307] "Preprocessing" refers to a process of removing unnecessary spaces and special characters from the inquiry data.

[0308] A "sentiment analysis engine" refers to a software component used to analyze a user's emotional state from their inquiry data.

[0309] A "database" refers to a system that stores information for problem analysis and searching for solutions.

[0310] A "generative AI model" refers to a model that uses artificial intelligence technology to generate solutions to problems based on input data.

[0311] "Solution" refers to the instructions or methods provided to resolve a problem entered by the user.

[0312] "Countermeasures" refer to solutions and additional support tailored to the user's emotional state.

[0313] "Feedback" refers to the act of a user submitting the results of trying out a provided solution and their opinions on it.

[0314] "Additional support options" refer to supplementary support methods provided to users, such as live chat and remote access.

[0315] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. In particular, it provides a configuration that combines this system with an emotion analysis engine that recognizes the user's emotions and responds accordingly.

[0316] Hardware and software to be used

[0317] This system uses the following hardware and software:

[0318] Hardware:

[0319] The device the user uses (smartphone, computer, etc.)

[0320] Server (a computer with high-performance processing capabilities)

[0321] software:

[0322] Sentiment analysis engine (e.g., BERT model using HuggingFace's Transformers library)

[0323] Database system (stores information for problem analysis and searching for solutions).

[0324] Generative AI models (e.g., GPT-4, which generate problem-solving methods based on input data)

[0325] Data processing and data calculation

[0326] 1. The user accesses the system and enters inquiry data: The user accesses the system and enters a specific problem or question. For example, they might enter, "My smartphone screen goes dark." This input data is sent to the server via the terminal.

[0327] 2. Server preprocessing: The server preprocesses the received input data. It removes unnecessary spaces and special characters and converts it into a format suitable for analysis.

[0328] 3. Emotion Analysis by Emotion Analysis Engine: The server sends the pre-processed data to the emotion analysis engine, which analyzes the user's emotional state. This determines whether the user is experiencing an emotional state such as "anger" or "confusion."

[0329] 4. Problem analysis and solution search using databases and generative AI models: The server identifies the problem based on the sentiment analysis results and searches for the optimal solution using databases and generative AI models. For example, the prompt might be in the format of, "The user says 'the smartphone screen is dimming,' and their emotion is 'frustrated.' What kind of support is best?"

[0330] 5. Sending solutions and emotional responses to the device: The server sends the found solutions and emotional responses to the device. The device receives this and presents it to the user. An example of a presentation is a specific solution such as "Disable automatic brightness adjustment in settings."

[0331] 6. User Feedback: Users attempt solutions and provide feedback to the server via their device. For example, they might send feedback such as "The problem was not resolved."

[0332] 7. Server-based feedback processing and additional support: The server processes user feedback and performs sentiment analysis again. Additional support options, such as live chat and remote access, are provided as needed.

[0333] Specific example

[0334] Specific examples are given below.

[0335] When a user reports a problem such as "my smartphone won't charge," the server provides a solution through the following process.

[0336] 1. User: Enters "My smartphone won't charge," and the emotion analysis engine recognizes "confusion."

[0337] 2. Terminal: Send this input to the server.

[0338] 3. Server: Preprocesses the input and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0339] 4. Server: Sends the solution and sentiment analysis results back to the terminal.

[0340] 5. Terminal: Displays the returned information to the user.

[0341] 6. User: Tries a solution, sends feedback that it "does not work," and the sentiment analysis engine recognizes "frustration."

[0342] 7. Server: Upon receiving this feedback, the server will promptly provide live chat based on the sentiment analysis results.

[0343] In this way, the system not only efficiently solves user problems but also enables appropriate responses through emotion recognition.

[0344] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0345] Step 1:

[0346] The user accesses the system and enters inquiry data. The user enters a specific problem or question, for example, "My smartphone screen goes dark." This input data is sent to the server via the terminal. The input is in text format, and the output is also inquiry data in text format.

[0347] Step 2:

[0348] The terminal sends user input data to the server. The terminal packets the input data in the appropriate format and quickly sends it to the server via the internet. The input is the user's text input, and the output is the input data sent to the server.

[0349] Step 3:

[0350] The server preprocesses the input data it receives. This preprocessing includes removing unnecessary spaces and special characters. The server cleans up the data using regular expression libraries and other tools. The input is the raw data sent from the terminal, and the output is the clean, preprocessed input data.

[0351] Step 4:

[0352] The server sends pre-processed data to the sentiment analysis engine, which then analyzes the user's emotional state. The sentiment analysis engine uses a BERT model, for example, the HuggingFace Transformers library. The input is pre-processed text data, and the output is the sentiment analysis result (e.g., "irritated," "confused," etc.).

[0353] Step 5:

[0354] The server analyzes the problem and searches for solutions using a database and a generative AI model based on the sentiment analysis results and preprocessed data. For example, it creates a prompt sentence such as, "The user says 'the smartphone screen is getting dark,' and their emotion is 'frustrated.' What kind of support is best?" and inputs it into a generative AI model (e.g., GPT-4). The input is preprocessed data and sentiment analysis results, and the output is a specific solution (e.g., "Disable automatic brightness adjustment in settings").

[0355] Step 6:

[0356] The server sends a solution and emotional response to the user's device. The device receives this and displays it to the user. Specifically, it displays a solution such as "Disable automatic brightness adjustment in settings" on the screen. The input is the solution and emotional response, and the output is the solution instructions displayed on the device.

[0357] Step 7:

[0358] The user attempts the suggested solution and provides feedback on the results. For example, if the solution doesn't work, the user enters "Not solved" and sends it back to the server from their device. The input is the result of attempting the solution, and the output is the feedback data sent to the server.

[0359] Step 8:

[0360] The server receives user feedback and re-analyzes it using an emotion analysis engine. Based on the feedback, the server provides additional support options (e.g., live chat or remote access). The input is the feedback data and the re-analyzed emotion data, and the output is the provision of additional support options.

[0361] This series of steps allows users to efficiently solve problems from home and receive appropriate support based on sentiment analysis.

[0362] (Application Example 2)

[0363] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0364] Traditional remote support systems often lacked the ability to respond to users' emotions, particularly those experiencing confusion or frustration, and frequently failed to provide appropriate support. This resulted in decreased user problem-solving skills and satisfaction. Furthermore, in-store customer support also faced challenges due to physical limitations that made rapid and effective responses difficult.

[0365] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing, means for analyzing the problem using a database or AI model and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback, means for presenting additional support options based on user feedback, and means for performing sentiment analysis and adjusting countermeasures according to the user's emotional state. This enables customized responses according to the user's emotional state, improving the efficiency and satisfaction of customer support in physical stores.

[0366] definition statement

[0367] "Inquiry data" refers to the information that users enter into the system regarding problems or questions.

[0368] "Preprocessing" refers to the process of removing unnecessary spaces and special characters from input data and converting it into a format that is easy to analyze.

[0369] A "database" is a system that stores and manages information such as inquiry data and solutions.

[0370] An "AI model" is an algorithm that uses technologies such as machine learning and deep learning to analyze data and identify problems and find solutions.

[0371] "Emotion analysis" is a technology that infers and analyzes emotions from user input data.

[0372] "Feedback" refers to the evaluation and reporting of results that users provide regarding the solutions offered.

[0373] "Additional support options" refer to support services provided in addition to basic troubleshooting methods, such as live chat and remote access.

[0374] A "device" refers to a device used by a user, such as a smartphone or computer.

[0375] Modes for carrying out the invention

[0376] This invention is a support system for users to remotely resolve problems, and in particular, a system that recognizes the user's emotions and provides appropriate responses.

[0377] System program

[0378] When a user enters inquiry data using a smartphone or computer terminal, the data is sent to the server. The server first preprocesses the data, removing unnecessary spaces and special characters. Then, it analyzes the user's emotional state using an emotion analysis engine. Next, it uses an AI model and database to analyze the problem and search for appropriate solutions. The searched solutions and the results of the emotion analysis are sent back to the terminal and displayed to the user. The user tries the displayed solutions and provides the results as feedback. Based on this feedback, the server provides additional support options. This entire process enables customized responses tailored to the user's emotional state, improving the efficiency and satisfaction of problem solving.

[0379] Hardware and software used

[0380] The following hardware and software will be used to implement the system.

[0381] Hardware: Smartphones, computer terminals, server PCs, or cloud infrastructure (e.g., Google Cloud Platform, Amazon Web Services)

[0382] Software: Sentiment analysis engine (e.g., Google Cloud Natural Language API), AI model (e.g., OpenAI API)

[0383] Data preprocessing is performed using programming languages ​​such as Python, and a sentiment analysis engine and AI model are combined to provide problem analysis and solutions.

[0384] Specific examples and prompt statements

[0385] 1. Specific example:

[0386] When a user enters "I don't know how to use this product," the server analyzes their emotional state and recognizes "confusion." As a result, solutions such as a "link to an online product demo" or a "video guide" are provided.

[0387] 2. Example of a prompt:

[0388] "A user says, 'I don't know how to use this product.' Please suggest a solution."

[0389] This invention will digitally complement in-store customer support, enabling the provision of quick and appropriate assistance. Furthermore, it is expected to improve user satisfaction through responses based on emotion recognition.

[0390] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0391] Processing steps

[0392] Step 1:

[0393] Users use their smartphones or computer terminals to input problems or questions they need help with. This input data is sent to the server as "inquiry data." An example of input data might be, "I don't know how to use this product."

[0394] Step 2:

[0395] The server preprocesses the received query data. This preprocessing includes data manipulation to remove unnecessary spaces and special characters. This results in data in a format suitable for subsequent processing. The input is the user's query data, and the output is the preprocessed data.

[0396] Step 3:

[0397] The server sends pre-processed data to an emotion analysis engine to analyze the user's emotional state. This analysis can identify whether the user is experiencing emotions such as anxiety, confusion, or frustration. The input is pre-processed data, and the output is the analyzed emotion score and emotional state.

[0398] Step 4:

[0399] The server analyzes the problem using a database and a generative AI model based on the sentiment analysis results and pre-processed query data, and searches for solutions. In this step, it generates and inputs appropriate prompt sentences to the AI ​​model to find the best solution. The input is the sentiment analysis results and pre-processed data, and the output is the searched solution.

[0400] Step 5:

[0401] The server returns the searched solutions and sentiment analysis results to the user's terminal. Care is taken to ensure that the solutions and corresponding responses are clearly displayed to the user. The input consists of the solutions and sentiment analysis results, while the output is the solutions displayed on the user's terminal.

[0402] Step 6:

[0403] The user tries the solutions displayed on their device and provides feedback on the results. For example, they may be presented with options such as "Resolved" or "Not resolved," and the user selects one of these.

[0404] Step 7:

[0405] The server receives feedback data and provides further appropriate support options as needed. For example, if the issue is not resolved or the user's anxiety increases, additional assistance such as live chat or remote support is provided quickly. The input is user feedback, and the output is additional support options.

[0406] This series of processes allows users to receive effective support even when they are not in a physical store, leading to improved customer satisfaction.

[0407] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0408] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0409] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0410] [Second Embodiment]

[0411] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0412] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0413] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0414] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0415] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0416] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0417] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0418] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0419] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0420] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0421] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0422] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0423] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. This system includes a series of means for providing quick and accurate solutions to user problems and inquiries.

[0424] Receive user input

[0425] First, the user accesses the system and enters the problem or question they need support for. For example, they might enter, "My smartphone screen goes dark." This triggers the system to receive the user's problem.

[0426] Server analysis of the problem

[0427] Next, the terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[0428] Providing a response

[0429] Once the server finds a solution, that information is sent back to the device. The device then displays the solution received from the server to the user. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[0430] Follow-up

[0431] The user tries the suggested solution and provides feedback on the results. This feedback is then sent back to the server from the device. If the user reports that the issue is "not resolved," the server offers additional support options, including live chat and remote access.

[0432] Specific example

[0433] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[0434] 1. The user enters "My smartphone won't charge."

[0435] 2. The terminal sends this input to the server.

[0436] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0437] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[0438] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[0439] 6. The server receives this feedback and offers live chat as an additional support option.

[0440] In this way, the system provides a series of means for users to efficiently solve problems from their homes. It is easy to use and provides convenient support for users who live far away or who have physical limitations.

[0441] The following describes the processing flow.

[0442] Step 1:

[0443] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark."

[0444] Step 2:

[0445] The terminal sends user input to the server. This process is carried out via HTTP requests.

[0446] Step 3:

[0447] The server preprocesses the received query data. This preprocessing includes removing unnecessary spaces and special characters.

[0448] Step 4:

[0449] The server analyzes the problem using databases and AI models based on pre-processed data. Based on the analysis, an appropriate solution is found.

[0450] Step 5:

[0451] The server returns the searched solution to the user's terminal. This process also takes place via an HTTP response.

[0452] Step 6:

[0453] The device displays the received solution to the user. For example, it might suggest a specific solution such as "Disable automatic brightness adjustment in settings."

[0454] Step 7:

[0455] The user tries to find a solution. For example, they open the settings menu and disable automatic brightness adjustment.

[0456] Step 8:

[0457] The user provides feedback on the results. They answer "Yes" or "No" to the question, "Has the problem been resolved?".

[0458] Step 9:

[0459] The device sends user feedback to the server. This process is also carried out via HTTP requests.

[0460] Step 10:

[0461] The server receives feedback from the user, and if the feedback is "no," it offers additional support options. For example, live chat or remote access options may be provided.

[0462] Step 11:

[0463] The user selects additional support options. For example, they might decide to use live chat.

[0464] Step 12:

[0465] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[0466] This series of steps allows users to efficiently resolve problems from home. The improved speed and convenience of support significantly enhances the user experience.

[0467] (Example 1)

[0468] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0469] In modern society, there is a growing demand for users to receive technical support remotely from their homes, requiring rapid and accurate problem analysis and solutions. However, traditional systems often rely on manual processes for processing inquiry data and analyzing problems, making them inefficient and hindering user satisfaction. Furthermore, responding quickly to user feedback is difficult. A system is needed to improve this situation and provide remote support efficiently and effectively.

[0470] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0471] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, and means for presenting additional support options based on the user's feedback. This enables users to receive quick and accurate support remotely without having to visit a store. Furthermore, by providing additional support options based on user feedback, higher user satisfaction can be achieved.

[0472] "Inquiry data" refers to information related to problems or questions that users submit to the system.

[0473] "Preprocessing" is the process of removing unnecessary spaces and special characters from received data and preparing it for analysis.

[0474] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to analyze data, identify problems, and search for solutions.

[0475] A "database" is a collection of information that is structured, stored, and made searchable, containing relevant information such as user inquiries and past solutions.

[0476] A "solution" refers to the specific steps or instructions provided to resolve a user's inquiry.

[0477] A "terminal" is an electronic device used by a user to access a support system and receive solutions.

[0478] "Feedback" refers to information that users send back to the system, such as their evaluation of the results of the proposed solutions or any additional requests.

[0479] "Additional support options" refer to further assistance or services provided to users when the initial troubleshooting method fails to resolve their problem.

[0480] "Real-time communication" refers to methods that allow users and support staff to interact instantly, such as live chat and voice calls.

[0481] "Remote operation support" is a support method in which a support staff member remotely accesses a user's electronic device and performs operations on their behalf.

[0482] This invention relates to a system for users to receive remote support from their homes. Embodiments of this system are described in detail below.

[0483] The system's basic configuration includes three main elements: users, terminals, and servers. Users access the system via the internet using electronic devices such as smartphones and computers. Terminals connect to the system through web browsers or dedicated applications, receiving user input and sending it to the server. The server processes the received data, analyzes it using generative AI models, and provides solutions.

[0484] Hardware and software to be used

[0485] 1. User:

[0486] Users will use typical smartphones (e.g., Android devices, iPhones) or computers (e.g., Windows PCs, Macs). These devices require an internet connection.

[0487] 2. Terminal:

[0488] The device uses a web browser (e.g., Google Chrome, Safari) or a dedicated application (e.g., a custom support app). These are responsible for receiving user input and communicating with the server.

[0489] 3. Server:

[0490] The servers run in a cloud environment (e.g., AWS EC2 instances, Microsoft Azure VMs) and process and analyze query data. The software used includes the following:

[0491] Preprocessing: Python libraries (e.g., NLTK, SpaCy)

[0492] Analysis: Generative AI models (e.g., OpenAI GPT-4, Google BERT)

[0493] Database: MySQL, PostgreSQL

[0494] Specific actions

[0495] 1. Receive user input.

[0496] Users access the support portal using their smartphones or computers. For example, a user opens a browser and visits the support portal's URL.

[0497] The user enters their problem or question into the input form. For example, they might enter, "My smartphone screen goes dark."

[0498] 2. Sending input data

[0499] The terminal sends the entered data to the server. This communication uses the HTTPS protocol.

[0500] 3. Data preprocessing

[0501] When the server receives data, it first performs preprocessing. For example, it might use a Python library to remove unnecessary spaces and special characters from the input data.

[0502] 4. AI-based analysis

[0503] The server inputs pre-processed data into a generating AI model to identify problems and search for solutions. It also searches databases to supplement relevant information.

[0504] 5. Providing solutions

[0505] The server sends the identified solution back to the terminal. This information is exchanged in JSON format.

[0506] The device displays the received solution to the user. For example, it may show specific steps such as "Disable automatic brightness adjustment in settings."

[0507] 6. Feedback and additional support

[0508] The user tries the suggested solution and provides feedback on the results. For example, if the problem is not resolved, they can enter "Not resolved" in the form and submit it.

[0509] The device sends this feedback to the server.

[0510] The server receives feedback and provides additional support options as needed, including real-time communication and remote assistance.

[0511] Specific example

[0512] This shows the processing flow when a user enters a problem such as "My smartphone won't charge."

[0513] 1. The user enters "My smartphone won't charge."

[0514] 2. The terminal sends this input to the server.

[0515] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0516] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[0517] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[0518] 6. The server receives this feedback and provides real-time communication as an additional support option.

[0519] Example of a prompt:

[0520] User: My smartphone won't charge.

[0521] Server: Please check the charger connection.

[0522] User: I tried it, but it didn't solve the problem.

[0523] Server: Please try using a different cable. If that doesn't solve the problem, we will provide additional support via real-time communication.

[0524] This provides users with a series of tools to efficiently solve problems from home, making it easier to use and providing convenient support for users who live far away or have physical limitations.

[0525] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0526] Step 1:

[0527] The user enters information.

[0528] Users access the support portal via smartphones or computers.

[0529] Input: The user accesses the support portal and enters their inquiry. For example, they might enter, "My smartphone screen is dimming."

[0530] Output: User input is sent to the system via an HTML form.

[0531] Step 2:

[0532] The terminal receives input data and sends it to the server.

[0533] The terminal receives user input and sends it to the server.

[0534] Input: Inquiry data entered by the user in an HTML form.

[0535] Output: JSON formatted data sent to the server via the HTTPS protocol.

[0536] In terms of specific operations, a web browser or dedicated application converts the input data into an appropriate format and sends it to the server.

[0537] Step 3:

[0538] The server performs data preprocessing upon receiving data.

[0539] The server preprocesses the data it receives.

[0540] Input: Query data in JSON format sent from the terminal.

[0541] Output: Preprocessed data (data suitable for analysis, with unnecessary spaces and special characters removed).

[0542] Specifically, data cleaning is performed using Python's NLTK and SpaCy.

[0543] Step 4:

[0544] The server generates data and analyzes it using an AI model.

[0545] The server generates pre-processed data, which is then input into an AI model to identify problems and search for solutions.

[0546] Input: Pre-processed query data.

[0547] Output: Solutions identified by a generative AI model (e.g., OpenAI GPT-4).

[0548] Specifically, the server sends API requests to the generated AI model to analyze the problem and search for solutions.

[0549] Step 5:

[0550] The server searches the database to retrieve supplementary information.

[0551] The server searches the database and retrieves information to complement the solutions obtained from the generated AI model.

[0552] Input: Candidate solutions obtained from a generative AI model.

[0553] Output: Detailed information related to the optimal solution.

[0554] The server queries the database (MySQL, PostgreSQL) to retrieve additional relevant information.

[0555] Step 6:

[0556] The server sends the solution back to the terminal.

[0557] The server sends the identified solution back to the terminal.

[0558] Input: Generative AI models and solutions obtained from databases.

[0559] Output: Data in JSON format containing the solution to be displayed to the user.

[0560] Specifically, the server generates JSON data and sends it back to the terminal via the HTTPS protocol.

[0561] Step 7:

[0562] The device displays the solution to the user.

[0563] The terminal displays the solution it received from the server to the user.

[0564] Input: Resolution data in JSON format sent from the server.

[0565] Output: Solutions displayed in a user-friendly format.

[0566] A web browser or dedicated application parses the JSON data and displays the solution on the screen.

[0567] Step 8:

[0568] Users provide feedback

[0569] Users try the solutions and provide feedback on the results.

[0570] Input: Results and feedback on solutions attempted by the user.

[0571] Output: User feedback data is sent to the system.

[0572] Specific actions include entering "Not resolved" in the feedback form.

[0573] Step 9:

[0574] The server receives feedback and provides additional support.

[0575] The server receives user feedback and presents additional support options.

[0576] Input: User feedback data.

[0577] Output: Additional support options (real-time communication and remote operation assistance).

[0578] Specifically, the server will use the AI ​​model again to analyze the situation and provide the most suitable additional support options.

[0579] This allows the system to provide quick and accurate solutions to user inquiries, as well as offer additional support as needed.

[0580] (Application Example 1)

[0581] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0582] Traditional customer support systems often required users to visit physical stores, which was inconvenient, especially for users living far away or those with physical limitations. Even when remote support systems existed, they frequently lacked the necessary information and tools to resolve problems. This resulted in decreased support efficiency and lower customer satisfaction.

[0583] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0584] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, means for presenting additional support options based on the user's feedback, and means for the user to receive remote support regarding physical stores via their smartphone. This makes it possible for users to quickly and accurately receive inquiries and support regarding physical stores remotely, even from home.

[0585] "User-entered inquiry data" refers to information that users enter into the system when seeking support, and includes data that details the question or problem.

[0586] "Preprocessing" is the process of removing unnecessary spaces and special characters from received query data and converting it into a format suitable for data analysis.

[0587] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and predict how to solve a problem based on a specific algorithm.

[0588] A "database" is a collection of information where query data and solutions are stored, and which can be used for searching and analysis.

[0589] "Searching for solutions" is the process of identifying appropriate solutions using databases and generative AI models based on pre-processed data.

[0590] "Returning the solution to the user's device" refers to the process of sending the identified solution to the user's device for display.

[0591] "User feedback" refers to the opinions and results that users provide in response to the solutions offered.

[0592] "Additional support options" are further support measures provided based on user feedback, such as live chat and remote access.

[0593] "Remote support" refers to a service that allows users to receive support via the internet without needing to visit a physical store.

[0594] This invention relates to a system that allows users to receive remote support via their smartphones without having to visit a physical store. This system receives user inquiry data, preprocesses that data, uses a generated AI model and database to analyze the problem and search for solutions, and finally provides the user with a solution.

[0595] Hardware and software to be used

[0596] 1. Hardware

[0597] Smartphone: Used as a terminal for users to enter inquiry data.

[0598] Server: Receives, preprocesses, analyzes, and processes user feedback.

[0599] 2. Software

[0600] Smartphone application: An application (iOS, Android) for entering details of questions or problems and receiving solutions.

[0601] Server-side AI models: Data analysis and prediction of problem-solving methods (TensorFlow, PyTorch).

[0602] Database: Stores and searches user query history and solutions (MySQL, PostgreSQL).

[0603] System operation

[0604] 1. User input of inquiry data

[0605] The user opens the smartphone application and enters inquiry data. For example, they might enter a question such as, "I don't know how to return a product."

[0606] 2. Receiving and preprocessing data by the server

[0607] The smartphone application sends the entered query data to the server. The server first preprocesses the data, removing unnecessary spaces and special characters.

[0608] 3. Analysis using generative AI models

[0609] The pre-processed data is input into a generating AI model, where the problem is analyzed and solutions are searched for.

[0610] 4. Providing solutions

[0611] The server searches the database for the best solution based on the analysis results and sends it back to the user's smartphone application. For example, a solution such as "Please access the returns page, fill out the returns form, and submit it" might be displayed.

[0612] 5. Receiving feedback and providing additional support

[0613] The user tries the suggested solution and provides feedback on the results. The server receives the feedback and provides additional support options such as live chat or remote access as needed.

[0614] Examples of specific cases and prompt statements

[0615] Specific example:

[0616] If a user enters "I don't know how to return the product," the server preprocesses the data and uses a generation AI model to search for a solution such as "Please access the returns page, fill out the return form, and submit it," and displays it on the user's smartphone.

[0617] Example of a prompt:

[0618] "User input: 'I don't know how to return the product' -> Provide a solution"

[0619] In this way, users can receive quick and accurate inquiries and support regarding physical stores remotely, even from the comfort of their homes. This system is particularly useful for users who live far away or who have physical limitations.

[0620] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0621] Step 1:

[0622] The user launches the smartphone application and enters inquiry data. For example, the user enters "I don't know how to return a product" into the text field and presses the submit button. The input data (text) is retrieved by the app and sent to the next step.

[0623] Step 2:

[0624] The device (smartphone) sends user input data to the server. The transmitted data is received by the server and recorded in the log. The input includes text data, and the output is ready for preprocessing.

[0625] Step 3:

[0626] The server preprocesses the query data it receives. This preprocessing includes removing unnecessary spaces and special characters. Specifically, it removes unwanted parts from the text using regular expressions, etc. The input includes the user's raw data, and the output is clean query data.

[0627] Step 4:

[0628] Preprocessed data is input to a generating AI model on the server side. The generating AI model analyzes the preprocessed, clean data and predicts relevant solutions. Specifically, the AI ​​model (such as TensorFlow or PyTorch) analyzes the input data and outputs the corresponding solutions. The input includes preprocessed, clean data, and the output is the solutions.

[0629] Step 5:

[0630] The server references the database to supplement the solution obtained from the generated AI model with detailed data. For example, a solution such as "Access the returns page, fill out the return form, and submit it" might be identified. The input includes the output of the AI ​​model and information from the database, and the output provides a specific solution.

[0631] Step 6:

[0632] The server sends the identified solution back to the user's smartphone. The details of the solution are displayed on the user's device. The input includes data on the specific solution, and the output is the solution displayed on the user's device.

[0633] Step 7:

[0634] The user tries the suggested solution and provides feedback on the results. For example, they might enter feedback such as, "I accessed the returns page, filled out the return form, and submitted it, but it was not completed." The input includes the text of the feedback, and the output is feedback data sent to the server.

[0635] Step 8:

[0636] The server receives and analyzes user feedback. It then presents additional support options as needed. For example, it might generate a link to offer the user a live chat option. The input includes the feedback content, and the output presents additional support options.

[0637] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0638] This invention relates to a system that allows users to receive remote support for their smartphones or computers without having to visit a store. In particular, it provides a configuration that combines an emotion engine that recognizes the user's emotions and responds accordingly.

[0639] Receive user input

[0640] Users access the system and enter problems or questions they need support for. For example, they might enter, "My smartphone screen goes dark." The user's input is also sent to an emotion engine for sentiment analysis.

[0641] Server analysis of the problem

[0642] The terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Simultaneously, the emotion engine analyzes the user's input to determine their emotions. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[0643] Utilizing the Emotion Engine

[0644] The emotion engine recognizes the user's emotional state and automatically adjusts solutions and responses accordingly. For example, if the user is experiencing anger or frustration, solutions will be provided more quickly, and live chat will be prioritized as an additional support option.

[0645] Providing a response

[0646] Once the server finds a solution, that information, along with the results of the emotion engine's analysis, is sent back to the user's device. The device then displays the solution received from the server and appropriate countermeasures based on the user's emotion. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[0647] Follow-up

[0648] The user tries the suggested solution and provides feedback on the results. This feedback is sent back to the server from the device, and the sentiment engine is updated. If the user provides feedback that the problem is "not resolved," the server will quickly provide, for example, live chat or remote access, based on the sentiment engine's analysis.

[0649] Specific example

[0650] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[0651] 1. The user types "My smartphone won't charge," and the emotion engine recognizes "confusion."

[0652] 2. The terminal sends this input to the server.

[0653] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0654] 4. The server sends the solution and the sentiment engine's analysis results back to the terminal, which then displays them to the user.

[0655] 5. The user tries a solution and provides feedback that it "does not solve the problem," and the emotion engine recognizes "frustration."

[0656] 6. The server receives this feedback and, based on the sentiment engine's analysis results, quickly provides live chat.

[0657] This series of steps allows users to efficiently resolve problems from home and receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[0658] The following describes the processing flow.

[0659] Step 1:

[0660] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark." This input is sent to the emotion engine, which analyzes the user's emotions.

[0661] Step 2:

[0662] The terminal sends user input data to the server along with the emotion engine. The input data includes the user's questions and problem details.

[0663] Step 3:

[0664] The server preprocesses the received input data. This preprocessing includes removing unnecessary spaces and special characters. Simultaneously, sentiment analysis is performed by the sentiment engine.

[0665] Step 4:

[0666] The emotion engine analyzes user input to identify emotions such as "confusion," "irritation," and "anxiety." It then provides the identified emotional state to the server.

[0667] Step 5:

[0668] The server analyzes the problem using databases and AI models based on pre-processed data and the results of the emotion engine's analysis. Based on the analysis, it searches for appropriate solutions.

[0669] Step 6:

[0670] The server returns the searched solution, along with the sentiment engine's analysis results, to the user's device. For example, it may include specific instructions such as "Disable automatic brightness adjustment in settings."

[0671] Step 7:

[0672] The device displays the user with solutions and corresponding responses based on their emotions. The user then tries out the suggested solutions.

[0673] Step 8:

[0674] After trying a solution, the user provides feedback on the result. For example, they might answer "yes" or "no" to the question, "Was the problem solved?"

[0675] Step 9:

[0676] The device sends user feedback to the server. This feedback includes information about the success or failure of the resolution and the user's emotional state.

[0677] Step 10:

[0678] The server receives feedback from the user, and if the feedback is "no," it provides additional support options. In this process, it takes into account the results of the emotion engine's analysis; for example, if it recognizes "frustration," it prioritizes offering live chat.

[0679] Step 11:

[0680] The user selects live chat as an additional support option. The device sends that selection to the server.

[0681] Step 12:

[0682] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[0683] Through this series of steps, users can not only efficiently resolve problems from home but also receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[0684] (Example 2)

[0685] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0686] In systems that provide remote support, there is a problem in that they cannot recognize the user's emotions and respond appropriately accordingly, resulting in a degraded user experience. Furthermore, traditional support systems have the challenge of not being able to provide prompt additional support based on user feedback.

[0687] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0688] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the data, and means for analyzing the user's emotions using an emotion analysis engine. This enables highly accurate problem analysis and solution retrieval tailored to the user's emotional state. The invention also includes means for analyzing problems and retrieving solutions using a database and generative AI models, means for returning solutions and corresponding countermeasures to the user's terminal, and means for providing additional support options based on user feedback and emotion analysis results. This improves the user experience and enables efficient and rapid problem solving.

[0689] A "user" refers to a person who accesses the system and enters inquiry data.

[0690] "Inquiry data" refers to information about problems or questions that users enter that require support.

[0691] A "terminal" refers to an information device used by a user to access a system.

[0692] A "server" refers to a primary computer system that receives query data and processes it through pre-processing, analysis, solution retrieval, sentiment analysis, and feedback.

[0693] "Preprocessing" refers to the process of removing unnecessary spaces and special characters from query data.

[0694] A "sentiment analysis engine" refers to a software component used to analyze a user's emotional state from their inquiry data.

[0695] A "database" refers to a system that stores information for problem analysis and searching for solutions.

[0696] A "generative AI model" refers to a model that uses artificial intelligence technology to generate solutions to problems based on input data.

[0697] "Solution" refers to the instructions or methods provided to resolve a problem entered by the user.

[0698] "Countermeasures" refer to solutions and additional support tailored to the user's emotional state.

[0699] "Feedback" refers to the act of a user submitting the results of trying out a provided solution and their opinions on it.

[0700] "Additional support options" refer to supplementary support methods provided to users, such as live chat and remote access.

[0701] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. In particular, it provides a configuration that combines this system with an emotion analysis engine that recognizes the user's emotions and responds accordingly.

[0702] Hardware and software to be used

[0703] This system uses the following hardware and software:

[0704] Hardware:

[0705] The device the user uses (smartphone, computer, etc.)

[0706] Server (a computer with high-performance processing capabilities)

[0707] software:

[0708] Sentiment analysis engine (e.g., BERT model using HuggingFace's Transformers library)

[0709] Database system (stores information for problem analysis and searching for solutions).

[0710] Generative AI models (e.g., GPT-4, which generate problem-solving methods based on input data)

[0711] Data processing and data calculation

[0712] 1. The user accesses the system and enters inquiry data: The user accesses the system and enters a specific problem or question. For example, they might enter, "My smartphone screen goes dark." This input data is sent to the server via the terminal.

[0713] 2. Server preprocessing: The server preprocesses the received input data. It removes unnecessary spaces and special characters and converts it into a format suitable for analysis.

[0714] 3. Emotion Analysis by Emotion Analysis Engine: The server sends the pre-processed data to the emotion analysis engine, which analyzes the user's emotional state. This determines whether the user is experiencing an emotional state such as "anger" or "confusion."

[0715] 4. Problem analysis and solution search using databases and generative AI models: The server identifies the problem based on the sentiment analysis results and searches for the optimal solution using databases and generative AI models. For example, the prompt might be in the format of, "The user says 'the smartphone screen is dimming,' and their emotion is 'frustrated.' What kind of support is best?"

[0716] 5. Sending solutions and emotional responses to the device: The server sends the found solutions and emotional responses to the device. The device receives this and presents it to the user. An example of a presentation is a specific solution such as "Disable automatic brightness adjustment in settings."

[0717] 6. User Feedback: Users attempt solutions and provide feedback to the server via their device. For example, they might send feedback such as "The problem was not resolved."

[0718] 7. Server-based feedback processing and additional support: The server processes user feedback and performs sentiment analysis again. Additional support options, such as live chat and remote access, are provided as needed.

[0719] Specific example

[0720] Specific examples are given below.

[0721] When a user reports a problem such as "my smartphone won't charge," the server provides a solution through the following process.

[0722] 1. User: Enters "My smartphone won't charge," and the emotion analysis engine recognizes "confusion."

[0723] 2. Terminal: Send this input to the server.

[0724] 3. Server: Preprocesses the input and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0725] 4. Server: Sends the solution and sentiment analysis results back to the terminal.

[0726] 5. Terminal: Displays the returned information to the user.

[0727] 6. User: Tries a solution, sends feedback that it "does not work," and the sentiment analysis engine recognizes "frustration."

[0728] 7. Server: Upon receiving this feedback, the server will promptly provide live chat based on the sentiment analysis results.

[0729] In this way, the system not only efficiently solves user problems but also enables appropriate responses through emotion recognition.

[0730] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0731] Step 1:

[0732] The user accesses the system and enters inquiry data. The user enters a specific problem or question, for example, "My smartphone screen goes dark." This input data is sent to the server via the terminal. The input is in text format, and the output is also inquiry data in text format.

[0733] Step 2:

[0734] The terminal sends user input data to the server. The terminal packets the input data in the appropriate format and quickly sends it to the server via the internet. The input is the user's text input, and the output is the input data sent to the server.

[0735] Step 3:

[0736] The server preprocesses the input data it receives. This preprocessing includes removing unnecessary spaces and special characters. The server cleans up the data using regular expression libraries and other tools. The input is the raw data sent from the terminal, and the output is the clean, preprocessed input data.

[0737] Step 4:

[0738] The server sends pre-processed data to the sentiment analysis engine, which then analyzes the user's emotional state. The sentiment analysis engine uses a BERT model, for example, the HuggingFace Transformers library. The input is pre-processed text data, and the output is the sentiment analysis result (e.g., "irritated," "confused," etc.).

[0739] Step 5:

[0740] The server analyzes the problem and searches for solutions using a database and a generative AI model based on the sentiment analysis results and preprocessed data. For example, it creates a prompt sentence such as, "The user says 'the smartphone screen is getting dark,' and their emotion is 'frustrated.' What kind of support is best?" and inputs it into a generative AI model (e.g., GPT-4). The input is preprocessed data and sentiment analysis results, and the output is a specific solution (e.g., "Disable automatic brightness adjustment in settings").

[0741] Step 6:

[0742] The server sends a solution and emotional response to the user's device. The device receives this and displays it to the user. Specifically, it displays a solution such as "Disable automatic brightness adjustment in settings" on the screen. The input is the solution and emotional response, and the output is the solution instructions displayed on the device.

[0743] Step 7:

[0744] The user attempts the suggested solution and provides feedback on the results. For example, if the solution doesn't work, the user enters "Not solved" and sends it back to the server from their device. The input is the result of attempting the solution, and the output is the feedback data sent to the server.

[0745] Step 8:

[0746] The server receives user feedback and re-analyzes it using an emotion analysis engine. Based on the feedback, the server provides additional support options (e.g., live chat or remote access). The input is the feedback data and the re-analyzed emotion data, and the output is the provision of additional support options.

[0747] This series of steps allows users to efficiently solve problems from home and receive appropriate support based on sentiment analysis.

[0748] (Application Example 2)

[0749] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0750] Traditional remote support systems often lacked the ability to respond to users' emotions, particularly those experiencing confusion or frustration, and frequently failed to provide appropriate support. This resulted in decreased user problem-solving skills and satisfaction. Furthermore, in-store customer support also faced challenges due to physical limitations that made rapid and effective responses difficult.

[0751] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing, means for analyzing the problem using a database or AI model and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback, means for presenting additional support options based on user feedback, and means for performing sentiment analysis and adjusting countermeasures according to the user's emotional state. This enables customized responses according to the user's emotional state, improving the efficiency and satisfaction of customer support in physical stores.

[0752] definition statement

[0753] "Inquiry data" refers to the information that users enter into the system regarding problems or questions.

[0754] "Preprocessing" refers to the process of removing unnecessary spaces and special characters from input data and converting it into a format that is easy to analyze.

[0755] A "database" is a system that stores and manages information such as inquiry data and solutions.

[0756] An "AI model" is an algorithm that uses technologies such as machine learning and deep learning to analyze data and identify problems and find solutions.

[0757] "Emotion analysis" is a technology that infers and analyzes emotions from user input data.

[0758] "Feedback" refers to the evaluation and reporting of results that users provide regarding the solutions offered.

[0759] "Additional support options" refer to support services provided in addition to basic troubleshooting methods, such as live chat and remote access.

[0760] A "device" refers to a device used by a user, such as a smartphone or computer.

[0761] Modes for carrying out the invention

[0762] This invention is a support system for users to remotely resolve problems, and in particular, a system that recognizes the user's emotions and provides appropriate responses.

[0763] System program

[0764] When a user enters inquiry data using a smartphone or computer terminal, the data is sent to the server. The server first preprocesses the data, removing unnecessary spaces and special characters. Then, it analyzes the user's emotional state using an emotion analysis engine. Next, it uses an AI model and database to analyze the problem and search for appropriate solutions. The searched solutions and the results of the emotion analysis are sent back to the terminal and displayed to the user. The user tries the displayed solutions and provides the results as feedback. Based on this feedback, the server provides additional support options. This entire process enables customized responses tailored to the user's emotional state, improving the efficiency and satisfaction of problem solving.

[0765] Hardware and software used

[0766] The following hardware and software will be used to implement the system.

[0767] Hardware: Smartphones, computer terminals, server PCs, or cloud infrastructure (e.g., Google Cloud Platform, Amazon Web Services)

[0768] Software: Sentiment analysis engine (e.g., Google Cloud Natural Language API), AI model (e.g., OpenAI API)

[0769] Data preprocessing is performed using programming languages ​​such as Python, and a sentiment analysis engine and AI model are combined to provide problem analysis and solutions.

[0770] Specific examples and prompt statements

[0771] 1. Specific example:

[0772] When a user enters "I don't know how to use this product," the server analyzes their emotional state and recognizes "confusion." As a result, solutions such as a "link to an online product demo" or a "video guide" are provided.

[0773] 2. Example of a prompt:

[0774] "A user says, 'I don't know how to use this product.' Please suggest a solution."

[0775] This invention will digitally complement in-store customer support, enabling the provision of quick and appropriate assistance. Furthermore, it is expected to improve user satisfaction through responses based on emotion recognition.

[0776] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0777] Processing steps

[0778] Step 1:

[0779] Users use their smartphones or computer terminals to input problems or questions they need help with. This input data is sent to the server as "inquiry data." An example of input data might be, "I don't know how to use this product."

[0780] Step 2:

[0781] The server preprocesses the received query data. This preprocessing includes data manipulation to remove unnecessary spaces and special characters. This results in data in a format suitable for subsequent processing. The input is the user's query data, and the output is the preprocessed data.

[0782] Step 3:

[0783] The server sends pre-processed data to an emotion analysis engine to analyze the user's emotional state. This analysis can identify whether the user is experiencing emotions such as anxiety, confusion, or frustration. The input is pre-processed data, and the output is the analyzed emotion score and emotional state.

[0784] Step 4:

[0785] The server analyzes the problem using a database and a generative AI model based on the sentiment analysis results and pre-processed query data, and searches for solutions. In this step, it generates and inputs appropriate prompt sentences to the AI ​​model to find the best solution. The input is the sentiment analysis results and pre-processed data, and the output is the searched solution.

[0786] Step 5:

[0787] The server returns the searched solutions and sentiment analysis results to the user's terminal. Care is taken to ensure that the solutions and corresponding responses are clearly displayed to the user. The input consists of the solutions and sentiment analysis results, while the output is the solutions displayed on the user's terminal.

[0788] Step 6:

[0789] The user tries the solutions displayed on their device and provides feedback on the results. For example, they may be presented with options such as "Resolved" or "Not resolved," and the user selects one of these.

[0790] Step 7:

[0791] The server receives feedback data and provides further appropriate support options as needed. For example, if the issue is not resolved or the user's anxiety increases, additional assistance such as live chat or remote support is provided quickly. The input is user feedback, and the output is additional support options.

[0792] This series of processes allows users to receive effective support even when they are not in a physical store, leading to improved customer satisfaction.

[0793] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0794] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0795] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0796] [Third Embodiment]

[0797] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0798] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0799] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0800] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0801] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0802] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0803] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0804] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0805] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0806] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0807] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0808] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0809] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. This system includes a series of means for providing quick and accurate solutions to user problems and inquiries.

[0810] Receive user input

[0811] First, the user accesses the system and enters the problem or question they need support for. For example, they might enter, "My smartphone screen goes dark." This triggers the system to receive the user's problem.

[0812] Server analysis of the problem

[0813] Next, the terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[0814] Providing a response

[0815] Once the server finds a solution, that information is sent back to the device. The device then displays the solution received from the server to the user. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[0816] Follow-up

[0817] The user tries the suggested solution and provides feedback on the results. This feedback is then sent back to the server from the device. If the user reports that the issue is "not resolved," the server offers additional support options, including live chat and remote access.

[0818] Specific example

[0819] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[0820] 1. The user enters "My smartphone won't charge."

[0821] 2. The terminal sends this input to the server.

[0822] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0823] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[0824] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[0825] 6. The server receives this feedback and offers live chat as an additional support option.

[0826] In this way, the system provides a series of means for users to efficiently solve problems from their homes. It is easy to use and provides convenient support for users who live far away or who have physical limitations.

[0827] The following describes the processing flow.

[0828] Step 1:

[0829] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark."

[0830] Step 2:

[0831] The terminal sends user input to the server. This process is carried out via HTTP requests.

[0832] Step 3:

[0833] The server preprocesses the received query data. This preprocessing includes removing unnecessary spaces and special characters.

[0834] Step 4:

[0835] The server analyzes the problem using databases and AI models based on pre-processed data. Based on the analysis, an appropriate solution is found.

[0836] Step 5:

[0837] The server returns the searched solution to the user's terminal. This process also takes place via an HTTP response.

[0838] Step 6:

[0839] The device displays the received solution to the user. For example, it might suggest a specific solution such as "Disable automatic brightness adjustment in settings."

[0840] Step 7:

[0841] The user tries to find a solution. For example, they open the settings menu and disable automatic brightness adjustment.

[0842] Step 8:

[0843] The user provides feedback on the results. They answer "Yes" or "No" to the question, "Has the problem been resolved?".

[0844] Step 9:

[0845] The device sends user feedback to the server. This process is also carried out via HTTP requests.

[0846] Step 10:

[0847] The server receives feedback from the user, and if the feedback is "no," it offers additional support options. For example, live chat or remote access options may be provided.

[0848] Step 11:

[0849] The user selects additional support options. For example, they might decide to use live chat.

[0850] Step 12:

[0851] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[0852] This series of steps allows users to efficiently resolve problems from home. The improved speed and convenience of support significantly enhances the user experience.

[0853] (Example 1)

[0854] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0855] In modern society, there is a growing demand for users to receive technical support remotely from their homes, requiring rapid and accurate problem analysis and solutions. However, traditional systems often rely on manual processes for processing inquiry data and analyzing problems, making them inefficient and hindering user satisfaction. Furthermore, responding quickly to user feedback is difficult. A system is needed to improve this situation and provide remote support efficiently and effectively.

[0856] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0857] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, and means for presenting additional support options based on the user's feedback. This enables users to receive quick and accurate support remotely without having to visit a store. Furthermore, by providing additional support options based on user feedback, higher user satisfaction can be achieved.

[0858] "Inquiry data" refers to information related to problems or questions that users submit to the system.

[0859] "Preprocessing" is the process of removing unnecessary spaces and special characters from received data and preparing it for analysis.

[0860] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to analyze data, identify problems, and search for solutions.

[0861] A "database" is a collection of information that is structured, stored, and made searchable, containing relevant information such as user inquiries and past solutions.

[0862] A "solution" refers to the specific steps or instructions provided to resolve a user's inquiry.

[0863] A "terminal" is an electronic device used by a user to access a support system and receive solutions.

[0864] "Feedback" refers to information that users send back to the system, such as their evaluation of the results of the proposed solutions or any additional requests.

[0865] "Additional support options" refer to further assistance or services provided to users when the initial troubleshooting method fails to resolve their problem.

[0866] "Real-time communication" refers to methods that allow users and support staff to interact instantly, such as live chat and voice calls.

[0867] "Remote operation support" is a support method in which a support staff member remotely accesses a user's electronic device and performs operations on their behalf.

[0868] This invention relates to a system for users to receive remote support from their homes. Embodiments of this system are described in detail below.

[0869] The system's basic configuration includes three main elements: users, terminals, and servers. Users access the system via the internet using electronic devices such as smartphones and computers. Terminals connect to the system through web browsers or dedicated applications, receiving user input and sending it to the server. The server processes the received data, analyzes it using generative AI models, and provides solutions.

[0870] Hardware and software to be used

[0871] 1. User:

[0872] Users will use typical smartphones (e.g., Android devices, iPhones) or computers (e.g., Windows PCs, Macs). These devices require an internet connection.

[0873] 2. Terminal:

[0874] The device uses a web browser (e.g., Google Chrome, Safari) or a dedicated application (e.g., a custom support app). These are responsible for receiving user input and communicating with the server.

[0875] 3. Server:

[0876] The servers run in a cloud environment (e.g., AWS EC2 instances, Microsoft Azure VMs) and process and analyze query data. The software used includes the following:

[0877] Preprocessing: Python libraries (e.g., NLTK, SpaCy)

[0878] Analysis: Generative AI models (e.g., OpenAI GPT-4, Google BERT)

[0879] Database: MySQL, PostgreSQL

[0880] Specific actions

[0881] 1. Receive user input.

[0882] Users access the support portal using their smartphones or computers. For example, a user opens a browser and visits the support portal's URL.

[0883] The user enters their problem or question into the input form. For example, they might enter, "My smartphone screen goes dark."

[0884] 2. Sending input data

[0885] The terminal sends the entered data to the server. This communication uses the HTTPS protocol.

[0886] 3. Data preprocessing

[0887] When the server receives data, it first performs preprocessing. For example, it might use a Python library to remove unnecessary spaces and special characters from the input data.

[0888] 4. AI-based analysis

[0889] The server inputs pre-processed data into a generating AI model to identify problems and search for solutions. It also searches databases to supplement relevant information.

[0890] 5. Providing solutions

[0891] The server sends the identified solution back to the terminal. This information is exchanged in JSON format.

[0892] The device displays the received solution to the user. For example, it may show specific steps such as "Disable automatic brightness adjustment in settings."

[0893] 6. Feedback and additional support

[0894] The user tries the suggested solution and provides feedback on the results. For example, if the problem is not resolved, they can enter "Not resolved" in the form and submit it.

[0895] The device sends this feedback to the server.

[0896] The server receives feedback and provides additional support options as needed, including real-time communication and remote assistance.

[0897] Specific example

[0898] This shows the processing flow when a user enters a problem such as "My smartphone won't charge."

[0899] 1. The user enters "My smartphone won't charge."

[0900] 2. The terminal sends this input to the server.

[0901] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[0902] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[0903] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[0904] 6. The server receives this feedback and provides real-time communication as an additional support option.

[0905] Example of a prompt:

[0906] User: My smartphone won't charge.

[0907] Server: Please check the charger connection.

[0908] User: I tried it, but it didn't solve the problem.

[0909] Server: Please try using a different cable. If that doesn't solve the problem, we will provide additional support via real-time communication.

[0910] This provides users with a series of tools to efficiently solve problems from home, making it easier to use and providing convenient support for users who live far away or have physical limitations.

[0911] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0912] Step 1:

[0913] The user enters information.

[0914] Users access the support portal via smartphones or computers.

[0915] Input: The user accesses the support portal and enters their inquiry. For example, they might enter, "My smartphone screen is dimming."

[0916] Output: User input is sent to the system via an HTML form.

[0917] Step 2:

[0918] The terminal receives input data and sends it to the server.

[0919] The terminal receives user input and sends it to the server.

[0920] Input: Inquiry data entered by the user in an HTML form.

[0921] Output: JSON formatted data sent to the server via the HTTPS protocol.

[0922] In terms of specific operations, a web browser or dedicated application converts the input data into an appropriate format and sends it to the server.

[0923] Step 3:

[0924] The server performs data preprocessing upon receiving data.

[0925] The server preprocesses the data it receives.

[0926] Input: Query data in JSON format sent from the terminal.

[0927] Output: Preprocessed data (data suitable for analysis, with unnecessary spaces and special characters removed).

[0928] Specifically, data cleaning is performed using Python's NLTK and SpaCy.

[0929] Step 4:

[0930] The server generates data and analyzes it using an AI model.

[0931] The server generates pre-processed data, which is then input into an AI model to identify problems and search for solutions.

[0932] Input: Pre-processed query data.

[0933] Output: Solutions identified by a generative AI model (e.g., OpenAI GPT-4).

[0934] Specifically, the server sends API requests to the generated AI model to analyze the problem and search for solutions.

[0935] Step 5:

[0936] The server searches the database to retrieve supplementary information.

[0937] The server searches the database and retrieves information to complement the solutions obtained from the generated AI model.

[0938] Input: Candidate solutions obtained from a generative AI model.

[0939] Output: Detailed information related to the optimal solution.

[0940] The server queries the database (MySQL, PostgreSQL) to retrieve additional relevant information.

[0941] Step 6:

[0942] The server sends the solution back to the terminal.

[0943] The server sends the identified solution back to the terminal.

[0944] Input: Generative AI models and solutions obtained from databases.

[0945] Output: Data in JSON format containing the solution to be displayed to the user.

[0946] Specifically, the server generates JSON data and sends it back to the terminal via the HTTPS protocol.

[0947] Step 7:

[0948] The device displays the solution to the user.

[0949] The terminal displays the solution it received from the server to the user.

[0950] Input: Resolution data in JSON format sent from the server.

[0951] Output: Solutions displayed in a user-friendly format.

[0952] A web browser or dedicated application parses the JSON data and displays the solution on the screen.

[0953] Step 8:

[0954] Users provide feedback

[0955] Users try the solutions and provide feedback on the results.

[0956] Input: Results and feedback on solutions attempted by the user.

[0957] Output: User feedback data is sent to the system.

[0958] Specific actions include entering "Not resolved" in the feedback form.

[0959] Step 9:

[0960] The server receives feedback and provides additional support.

[0961] The server receives user feedback and presents additional support options.

[0962] Input: User feedback data.

[0963] Output: Additional support options (real-time communication and remote operation assistance).

[0964] Specifically, the server will use the AI ​​model again to analyze the situation and provide the most suitable additional support options.

[0965] This allows the system to provide quick and accurate solutions to user inquiries, as well as offer additional support as needed.

[0966] (Application Example 1)

[0967] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0968] Traditional customer support systems often required users to visit physical stores, which was inconvenient, especially for users living far away or those with physical limitations. Even when remote support systems existed, they frequently lacked the necessary information and tools to resolve problems. This resulted in decreased support efficiency and lower customer satisfaction.

[0969] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0970] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, means for presenting additional support options based on the user's feedback, and means for the user to receive remote support regarding physical stores via their smartphone. This makes it possible for users to quickly and accurately receive inquiries and support regarding physical stores remotely, even from home.

[0971] "User-entered inquiry data" refers to information that users enter into the system when seeking support, and includes data that details the question or problem.

[0972] "Preprocessing" is the process of removing unnecessary spaces and special characters from received query data and converting it into a format suitable for data analysis.

[0973] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and predict how to solve a problem based on a specific algorithm.

[0974] A "database" is a collection of information where query data and solutions are stored, and which can be used for searching and analysis.

[0975] "Searching for solutions" is the process of identifying appropriate solutions using databases and generative AI models based on pre-processed data.

[0976] "Returning the solution to the user's device" refers to the process of sending the identified solution to the user's device for display.

[0977] "User feedback" refers to the opinions and results that users provide in response to the solutions offered.

[0978] "Additional support options" are further support measures provided based on user feedback, such as live chat and remote access.

[0979] "Remote support" refers to a service that allows users to receive support via the internet without needing to visit a physical store.

[0980] This invention relates to a system that allows users to receive remote support via their smartphones without having to visit a physical store. This system receives user inquiry data, preprocesses that data, uses a generated AI model and database to analyze the problem and search for solutions, and finally provides the user with a solution.

[0981] Hardware and software to be used

[0982] 1. Hardware

[0983] Smartphone: Used as a terminal for users to enter inquiry data.

[0984] Server: Receives, preprocesses, analyzes, and processes user feedback.

[0985] 2. Software

[0986] Smartphone application: An application (iOS, Android) for entering details of questions or problems and receiving solutions.

[0987] Server-side AI models: Data analysis and prediction of problem-solving methods (TensorFlow, PyTorch).

[0988] Database: Stores and searches user query history and solutions (MySQL, PostgreSQL).

[0989] System operation

[0990] 1. User input of inquiry data

[0991] The user opens the smartphone application and enters inquiry data. For example, they might enter a question such as, "I don't know how to return a product."

[0992] 2. Receiving and preprocessing data by the server

[0993] The smartphone application sends the entered query data to the server. The server first preprocesses the data, removing unnecessary spaces and special characters.

[0994] 3. Analysis using generative AI models

[0995] The pre-processed data is input into a generating AI model, where the problem is analyzed and solutions are searched for.

[0996] 4. Providing solutions

[0997] The server searches the database for the best solution based on the analysis results and sends it back to the user's smartphone application. For example, a solution such as "Please access the returns page, fill out the returns form, and submit it" might be displayed.

[0998] 5. Receiving feedback and providing additional support

[0999] The user tries the suggested solution and provides feedback on the results. The server receives the feedback and provides additional support options such as live chat or remote access as needed.

[1000] Examples of specific cases and prompt statements

[1001] Specific example:

[1002] If a user enters "I don't know how to return the product," the server preprocesses the data and uses a generation AI model to search for a solution such as "Please access the returns page, fill out the return form, and submit it," and displays it on the user's smartphone.

[1003] Example of a prompt:

[1004] "User input: 'I don't know how to return the product' -> Provide a solution"

[1005] In this way, users can receive quick and accurate inquiries and support regarding physical stores remotely, even from the comfort of their homes. This system is particularly useful for users who live far away or who have physical limitations.

[1006] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1007] Step 1:

[1008] The user launches the smartphone application and enters inquiry data. For example, the user enters "I don't know how to return a product" into the text field and presses the submit button. The input data (text) is retrieved by the app and sent to the next step.

[1009] Step 2:

[1010] The device (smartphone) sends user input data to the server. The transmitted data is received by the server and recorded in the log. The input includes text data, and the output is ready for preprocessing.

[1011] Step 3:

[1012] The server preprocesses the query data it receives. This preprocessing includes removing unnecessary spaces and special characters. Specifically, it removes unwanted parts from the text using regular expressions, etc. The input includes the user's raw data, and the output is clean query data.

[1013] Step 4:

[1014] Preprocessed data is input to a generating AI model on the server side. The generating AI model analyzes the preprocessed, clean data and predicts relevant solutions. Specifically, the AI ​​model (such as TensorFlow or PyTorch) analyzes the input data and outputs the corresponding solutions. The input includes preprocessed, clean data, and the output is the solutions.

[1015] Step 5:

[1016] The server references the database to supplement the solution obtained from the generated AI model with detailed data. For example, a solution such as "Access the returns page, fill out the return form, and submit it" might be identified. The input includes the output of the AI ​​model and information from the database, and the output provides a specific solution.

[1017] Step 6:

[1018] The server sends the identified solution back to the user's smartphone. The details of the solution are displayed on the user's device. The input includes data on the specific solution, and the output is the solution displayed on the user's device.

[1019] Step 7:

[1020] The user tries the suggested solution and provides feedback on the results. For example, they might enter feedback such as, "I accessed the returns page, filled out the return form, and submitted it, but it was not completed." The input includes the text of the feedback, and the output is feedback data sent to the server.

[1021] Step 8:

[1022] The server receives and analyzes user feedback. It then presents additional support options as needed. For example, it might generate a link to offer the user a live chat option. The input includes the feedback content, and the output presents additional support options.

[1023] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1024] This invention relates to a system that allows users to receive remote support for their smartphones or computers without having to visit a store. In particular, it provides a configuration that combines an emotion engine that recognizes the user's emotions and responds accordingly.

[1025] Receive user input

[1026] Users access the system and enter problems or questions they need support for. For example, they might enter, "My smartphone screen goes dark." The user's input is also sent to an emotion engine for sentiment analysis.

[1027] Server analysis of the problem

[1028] The terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Simultaneously, the emotion engine analyzes the user's input to determine their emotions. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[1029] Utilizing the Emotion Engine

[1030] The emotion engine recognizes the user's emotional state and automatically adjusts solutions and responses accordingly. For example, if the user is experiencing anger or frustration, solutions will be provided more quickly, and live chat will be prioritized as an additional support option.

[1031] Providing a response

[1032] Once the server finds a solution, that information, along with the results of the emotion engine's analysis, is sent back to the user's device. The device then displays the solution received from the server and appropriate countermeasures based on the user's emotion. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[1033] Follow-up

[1034] The user tries the suggested solution and provides feedback on the results. This feedback is sent back to the server from the device, and the sentiment engine is updated. If the user provides feedback that the problem is "not resolved," the server will quickly provide, for example, live chat or remote access, based on the sentiment engine's analysis.

[1035] Specific example

[1036] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[1037] 1. The user types "My smartphone won't charge," and the emotion engine recognizes "confusion."

[1038] 2. The terminal sends this input to the server.

[1039] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[1040] 4. The server sends the solution and the sentiment engine's analysis results back to the terminal, which then displays them to the user.

[1041] 5. The user tries a solution and provides feedback that it "does not solve the problem," and the emotion engine recognizes "frustration."

[1042] 6. The server receives this feedback and, based on the sentiment engine's analysis results, quickly provides live chat.

[1043] This series of steps allows users to efficiently resolve problems from home and receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[1044] The following describes the processing flow.

[1045] Step 1:

[1046] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark." This input is sent to the emotion engine, which analyzes the user's emotions.

[1047] Step 2:

[1048] The terminal sends user input data to the server along with the emotion engine. The input data includes the user's questions and problem details.

[1049] Step 3:

[1050] The server preprocesses the received input data. This preprocessing includes removing unnecessary spaces and special characters. Simultaneously, sentiment analysis is performed by the sentiment engine.

[1051] Step 4:

[1052] The emotion engine analyzes user input to identify emotions such as "confusion," "irritation," and "anxiety." It then provides the identified emotional state to the server.

[1053] Step 5:

[1054] The server analyzes the problem using databases and AI models based on pre-processed data and the results of the emotion engine's analysis. Based on the analysis, it searches for appropriate solutions.

[1055] Step 6:

[1056] The server returns the searched solution, along with the sentiment engine's analysis results, to the user's device. For example, it may include specific instructions such as "Disable automatic brightness adjustment in settings."

[1057] Step 7:

[1058] The device displays the user with solutions and corresponding responses based on their emotions. The user then tries out the suggested solutions.

[1059] Step 8:

[1060] After trying a solution, the user provides feedback on the result. For example, they might answer "yes" or "no" to the question, "Was the problem solved?"

[1061] Step 9:

[1062] The device sends user feedback to the server. This feedback includes information about the success or failure of the resolution and the user's emotional state.

[1063] Step 10:

[1064] The server receives feedback from the user, and if the feedback is "no," it provides additional support options. In this process, it takes into account the results of the emotion engine's analysis; for example, if it recognizes "frustration," it prioritizes offering live chat.

[1065] Step 11:

[1066] The user selects live chat as an additional support option. The device sends that selection to the server.

[1067] Step 12:

[1068] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[1069] Through this series of steps, users can not only efficiently resolve problems from home but also receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[1070] (Example 2)

[1071] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1072] In systems that provide remote support, there is a problem in that they cannot recognize the user's emotions and respond appropriately accordingly, resulting in a degraded user experience. Furthermore, traditional support systems have the challenge of not being able to provide prompt additional support based on user feedback.

[1073] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1074] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the data, and means for analyzing the user's emotions using an emotion analysis engine. This enables highly accurate problem analysis and solution retrieval tailored to the user's emotional state. The invention also includes means for analyzing problems and retrieving solutions using a database and generative AI models, means for returning solutions and corresponding countermeasures to the user's terminal, and means for providing additional support options based on user feedback and emotion analysis results. This improves the user experience and enables efficient and rapid problem solving.

[1075] A "user" refers to a person who accesses the system and enters inquiry data.

[1076] "Inquiry data" refers to information about problems or questions that users enter that require support.

[1077] A "terminal" refers to an information device used by a user to access a system.

[1078] A "server" refers to a primary computer system that receives query data and processes it through pre-processing, analysis, solution retrieval, sentiment analysis, and feedback.

[1079] "Preprocessing" refers to the process of removing unnecessary spaces and special characters from query data.

[1080] A "sentiment analysis engine" refers to a software component used to analyze a user's emotional state from their inquiry data.

[1081] A "database" refers to a system that stores information for problem analysis and searching for solutions.

[1082] A "generative AI model" refers to a model that uses artificial intelligence technology to generate solutions to problems based on input data.

[1083] "Solution" refers to the instructions or methods provided to resolve a problem entered by the user.

[1084] "Countermeasures" refer to solutions and additional support tailored to the user's emotional state.

[1085] "Feedback" refers to the act of a user submitting the results of trying out a provided solution and their opinions on it.

[1086] "Additional support options" refer to supplementary support methods provided to users, such as live chat and remote access.

[1087] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. In particular, it provides a configuration that combines this system with an emotion analysis engine that recognizes the user's emotions and responds accordingly.

[1088] Hardware and software to be used

[1089] This system uses the following hardware and software:

[1090] Hardware:

[1091] The device the user uses (smartphone, computer, etc.)

[1092] Server (a computer with high-performance processing capabilities)

[1093] software:

[1094] Sentiment analysis engine (e.g., BERT model using HuggingFace's Transformers library)

[1095] Database system (stores information for problem analysis and searching for solutions).

[1096] Generative AI models (e.g., GPT-4, which generate problem-solving methods based on input data)

[1097] Data processing and data calculation

[1098] 1. The user accesses the system and enters inquiry data: The user accesses the system and enters a specific problem or question. For example, they might enter, "My smartphone screen goes dark." This input data is sent to the server via the terminal.

[1099] 2. Server preprocessing: The server preprocesses the received input data. It removes unnecessary spaces and special characters and converts it into a format suitable for analysis.

[1100] 3. Emotion Analysis by Emotion Analysis Engine: The server sends the pre-processed data to the emotion analysis engine, which analyzes the user's emotional state. This determines whether the user is experiencing an emotional state such as "anger" or "confusion."

[1101] 4. Problem analysis and solution search using databases and generative AI models: The server identifies the problem based on the sentiment analysis results and searches for the optimal solution using databases and generative AI models. For example, the prompt might be in the format of, "The user says 'the smartphone screen is dimming,' and their emotion is 'frustrated.' What kind of support is best?"

[1102] 5. Sending solutions and emotional responses to the device: The server sends the found solutions and emotional responses to the device. The device receives this and presents it to the user. An example of a presentation is a specific solution such as "Disable automatic brightness adjustment in settings."

[1103] 6. User Feedback: Users attempt solutions and provide feedback to the server via their device. For example, they might send feedback such as "The problem was not resolved."

[1104] 7. Server-based feedback processing and additional support: The server processes user feedback and performs sentiment analysis again. Additional support options, such as live chat and remote access, are provided as needed.

[1105] Specific example

[1106] Specific examples are given below.

[1107] When a user reports a problem such as "my smartphone won't charge," the server provides a solution through the following process.

[1108] 1. User: Enters "My smartphone won't charge," and the emotion analysis engine recognizes "confusion."

[1109] 2. Terminal: Send this input to the server.

[1110] 3. Server: Preprocesses the input and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[1111] 4. Server: Sends the solution and sentiment analysis results back to the terminal.

[1112] 5. Terminal: Displays the returned information to the user.

[1113] 6. User: Tries a solution, sends feedback that it "does not work," and the sentiment analysis engine recognizes "frustration."

[1114] 7. Server: Upon receiving this feedback, the server will promptly provide live chat based on the sentiment analysis results.

[1115] In this way, the system not only efficiently solves user problems but also enables appropriate responses through emotion recognition.

[1116] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1117] Step 1:

[1118] The user accesses the system and enters inquiry data. The user enters a specific problem or question, for example, "My smartphone screen goes dark." This input data is sent to the server via the terminal. The input is in text format, and the output is also inquiry data in text format.

[1119] Step 2:

[1120] The terminal sends user input data to the server. The terminal packets the input data in the appropriate format and quickly sends it to the server via the internet. The input is the user's text input, and the output is the input data sent to the server.

[1121] Step 3:

[1122] The server preprocesses the input data it receives. This preprocessing includes removing unnecessary spaces and special characters. The server cleans up the data using regular expression libraries and other tools. The input is the raw data sent from the terminal, and the output is the clean, preprocessed input data.

[1123] Step 4:

[1124] The server sends pre-processed data to the sentiment analysis engine, which then analyzes the user's emotional state. The sentiment analysis engine uses a BERT model, for example, the HuggingFace Transformers library. The input is pre-processed text data, and the output is the sentiment analysis result (e.g., "irritated," "confused," etc.).

[1125] Step 5:

[1126] The server analyzes the problem and searches for solutions using a database and a generative AI model based on the sentiment analysis results and preprocessed data. For example, it creates a prompt sentence such as, "The user says 'the smartphone screen is getting dark,' and their emotion is 'frustrated.' What kind of support is best?" and inputs it into a generative AI model (e.g., GPT-4). The input is preprocessed data and sentiment analysis results, and the output is a specific solution (e.g., "Disable automatic brightness adjustment in settings").

[1127] Step 6:

[1128] The server sends a solution and emotional response to the user's device. The device receives this and displays it to the user. Specifically, it displays a solution such as "Disable automatic brightness adjustment in settings" on the screen. The input is the solution and emotional response, and the output is the solution instructions displayed on the device.

[1129] Step 7:

[1130] The user attempts the suggested solution and provides feedback on the results. For example, if the solution doesn't work, the user enters "Not solved" and sends it back to the server from their device. The input is the result of attempting the solution, and the output is the feedback data sent to the server.

[1131] Step 8:

[1132] The server receives user feedback and re-analyzes it using an emotion analysis engine. Based on the feedback, the server provides additional support options (e.g., live chat or remote access). The input is the feedback data and the re-analyzed emotion data, and the output is the provision of additional support options.

[1133] This series of steps allows users to efficiently solve problems from home and receive appropriate support based on sentiment analysis.

[1134] (Application Example 2)

[1135] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1136] Traditional remote support systems often lacked the ability to respond to users' emotions, particularly those experiencing confusion or frustration, and frequently failed to provide appropriate support. This resulted in decreased user problem-solving skills and satisfaction. Furthermore, in-store customer support also faced challenges due to physical limitations that made rapid and effective responses difficult.

[1137] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing, means for analyzing the problem using a database or AI model and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback, means for presenting additional support options based on user feedback, and means for performing sentiment analysis and adjusting countermeasures according to the user's emotional state. This enables customized responses according to the user's emotional state, improving the efficiency and satisfaction of customer support in physical stores.

[1138] definition statement

[1139] "Inquiry data" refers to the information that users enter into the system regarding problems or questions.

[1140] "Preprocessing" refers to the process of removing unnecessary spaces and special characters from input data and converting it into a format that is easy to analyze.

[1141] A "database" is a system that stores and manages information such as inquiry data and solutions.

[1142] An "AI model" is an algorithm that uses technologies such as machine learning and deep learning to analyze data and identify problems and find solutions.

[1143] "Emotion analysis" is a technology that infers and analyzes emotions from user input data.

[1144] "Feedback" refers to the evaluation and reporting of results that users provide regarding the solutions offered.

[1145] "Additional support options" refer to support services provided in addition to basic troubleshooting methods, such as live chat and remote access.

[1146] A "device" refers to a device used by a user, such as a smartphone or computer.

[1147] Modes for carrying out the invention

[1148] This invention is a support system for users to remotely resolve problems, and in particular, a system that recognizes the user's emotions and provides appropriate responses.

[1149] System program

[1150] When a user enters inquiry data using a smartphone or computer terminal, the data is sent to the server. The server first preprocesses the data, removing unnecessary spaces and special characters. Then, it analyzes the user's emotional state using an emotion analysis engine. Next, it uses an AI model and database to analyze the problem and search for appropriate solutions. The searched solutions and the results of the emotion analysis are sent back to the terminal and displayed to the user. The user tries the displayed solutions and provides the results as feedback. Based on this feedback, the server provides additional support options. This entire process enables customized responses tailored to the user's emotional state, improving the efficiency and satisfaction of problem solving.

[1151] Hardware and software used

[1152] The following hardware and software will be used to implement the system.

[1153] Hardware: Smartphones, computer terminals, server PCs, or cloud infrastructure (e.g., Google Cloud Platform, Amazon Web Services)

[1154] Software: Sentiment analysis engine (e.g., Google Cloud Natural Language API), AI model (e.g., OpenAI API)

[1155] Data preprocessing is performed using programming languages ​​such as Python, and a sentiment analysis engine and AI model are combined to provide problem analysis and solutions.

[1156] Specific examples and prompt statements

[1157] 1. Specific example:

[1158] When a user enters "I don't know how to use this product," the server analyzes their emotional state and recognizes "confusion." As a result, solutions such as a "link to an online product demo" or a "video guide" are provided.

[1159] 2. Example of a prompt:

[1160] "A user says, 'I don't know how to use this product.' Please suggest a solution."

[1161] This invention will digitally complement in-store customer support, enabling the provision of quick and appropriate assistance. Furthermore, it is expected to improve user satisfaction through responses based on emotion recognition.

[1162] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1163] Processing steps

[1164] Step 1:

[1165] Users use their smartphones or computer terminals to input problems or questions they need help with. This input data is sent to the server as "inquiry data." An example of input data might be, "I don't know how to use this product."

[1166] Step 2:

[1167] The server preprocesses the received query data. This preprocessing includes data manipulation to remove unnecessary spaces and special characters. This results in data in a format suitable for subsequent processing. The input is the user's query data, and the output is the preprocessed data.

[1168] Step 3:

[1169] The server sends pre-processed data to an emotion analysis engine to analyze the user's emotional state. This analysis can identify whether the user is experiencing emotions such as anxiety, confusion, or frustration. The input is pre-processed data, and the output is the analyzed emotion score and emotional state.

[1170] Step 4:

[1171] The server analyzes the problem using a database and a generative AI model based on the sentiment analysis results and pre-processed query data, and searches for solutions. In this step, it generates and inputs appropriate prompt sentences to the AI ​​model to find the best solution. The input is the sentiment analysis results and pre-processed data, and the output is the retrieved solution.

[1172] Step 5:

[1173] The server returns the searched solutions and sentiment analysis results to the user's terminal. Care is taken to ensure that the solutions and corresponding responses are clearly displayed to the user. The input consists of the solutions and sentiment analysis results, while the output is the solutions displayed on the user's terminal.

[1174] Step 6:

[1175] The user tries the solutions displayed on their device and provides feedback on the results. For example, they may be presented with options such as "Resolved" or "Not resolved," and the user selects one of these.

[1176] Step 7:

[1177] The server receives feedback data and provides further appropriate support options as needed. For example, if the issue is not resolved or the user's anxiety increases, additional assistance such as live chat or remote support is provided quickly. The input is user feedback, and the output is additional support options.

[1178] This series of processes allows users to receive effective support even when they are not in a physical store, leading to improved customer satisfaction.

[1179] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1180] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1181] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1182] [Fourth Embodiment]

[1183] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1184] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1185] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1186] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1187] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1188] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1189] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1190] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1191] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1192] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1193] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1194] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1195] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1196] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. This system includes a series of means for providing quick and accurate solutions to user problems and inquiries.

[1197] Receive user input

[1198] First, the user accesses the system and enters the problem or question they need support for. For example, they might enter, "My smartphone screen goes dark." This triggers the system to receive the user's problem.

[1199] Server analysis of the problem

[1200] Next, the terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[1201] Providing a response

[1202] Once the server finds a solution, that information is sent back to the device. The device then displays the solution received from the server to the user. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[1203] Follow-up

[1204] The user tries the suggested solution and provides feedback on the results. This feedback is then sent back to the server from the device. If the user reports that the issue is "not resolved," the server offers additional support options, including live chat and remote access.

[1205] Specific example

[1206] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[1207] 1. The user enters "My smartphone won't charge."

[1208] 2. The terminal sends this input to the server.

[1209] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[1210] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[1211] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[1212] 6. The server receives this feedback and offers live chat as an additional support option.

[1213] In this way, the system provides a series of means for users to efficiently solve problems from their homes. It is easy to use and provides convenient support for users who live far away or who have physical limitations.

[1214] The following describes the processing flow.

[1215] Step 1:

[1216] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark."

[1217] Step 2:

[1218] The terminal sends user input to the server. This process is carried out via HTTP requests.

[1219] Step 3:

[1220] The server preprocesses the received query data. This preprocessing includes removing unnecessary spaces and special characters.

[1221] Step 4:

[1222] The server analyzes the problem using databases and AI models based on pre-processed data. Based on the analysis, an appropriate solution is found.

[1223] Step 5:

[1224] The server returns the searched solution to the user's terminal. This process also takes place via an HTTP response.

[1225] Step 6:

[1226] The device displays the received solution to the user. For example, it might suggest a specific solution such as "Disable automatic brightness adjustment in settings."

[1227] Step 7:

[1228] The user tries to find a solution. For example, they open the settings menu and disable automatic brightness adjustment.

[1229] Step 8:

[1230] The user provides feedback on the results. They answer "Yes" or "No" to the question, "Has the problem been resolved?".

[1231] Step 9:

[1232] The device sends user feedback to the server. This process is also carried out via HTTP requests.

[1233] Step 10:

[1234] The server receives feedback from the user, and if the feedback is "no," it offers additional support options. For example, live chat or remote access options may be provided.

[1235] Step 11:

[1236] The user selects additional support options. For example, they might decide to use live chat.

[1237] Step 12:

[1238] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[1239] This series of steps allows users to efficiently resolve problems from home. The improved speed and convenience of support significantly enhances the user experience.

[1240] (Example 1)

[1241] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1242] In modern society, there is a growing demand for users to receive technical support remotely from their homes, requiring rapid and accurate problem analysis and solutions. However, traditional systems often rely on manual processes for processing inquiry data and analyzing problems, making them inefficient and hindering user satisfaction. Furthermore, responding quickly to user feedback is difficult. A system is needed to improve this situation and provide remote support efficiently and effectively.

[1243] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1244] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, and means for presenting additional support options based on the user's feedback. This enables users to receive quick and accurate support remotely without having to visit a store. Furthermore, by providing additional support options based on user feedback, higher user satisfaction can be achieved.

[1245] "Inquiry data" refers to information related to problems or questions that users submit to the system.

[1246] "Preprocessing" is the process of removing unnecessary spaces and special characters from received data and preparing it for analysis.

[1247] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to analyze data, identify problems, and search for solutions.

[1248] A "database" is a collection of information that is structured, stored, and made searchable, containing relevant information such as user inquiries and past solutions.

[1249] A "solution" refers to the specific steps or instructions provided to resolve a user's inquiry.

[1250] A "terminal" is an electronic device used by a user to access a support system and receive solutions.

[1251] "Feedback" refers to information that users send back to the system, such as their evaluation of the results of the proposed solutions or any additional requests.

[1252] "Additional support options" refer to further assistance or services provided to users when the initial troubleshooting method fails to resolve their problem.

[1253] "Real-time communication" refers to methods that allow users and support staff to interact instantly, such as live chat and voice calls.

[1254] "Remote operation support" is a support method in which a support staff member remotely accesses a user's electronic device and performs operations on their behalf.

[1255] This invention relates to a system for users to receive remote support from their homes. Embodiments of this system are described in detail below.

[1256] The system's basic configuration includes three main elements: users, terminals, and servers. Users access the system via the internet using electronic devices such as smartphones and computers. Terminals connect to the system through web browsers or dedicated applications, receiving user input and sending it to the server. The server processes the received data, analyzes it using generative AI models, and provides solutions.

[1257] Hardware and software to be used

[1258] 1. User:

[1259] Users will use typical smartphones (e.g., Android devices, iPhones) or computers (e.g., Windows PCs, Macs). These devices require an internet connection.

[1260] 2. Terminal:

[1261] The device uses a web browser (e.g., Google Chrome, Safari) or a dedicated application (e.g., a custom support app). These are responsible for receiving user input and communicating with the server.

[1262] 3. Server:

[1263] The servers run in a cloud environment (e.g., AWS EC2 instances, Microsoft Azure VMs) and process and analyze query data. The software used includes the following:

[1264] Preprocessing: Python libraries (e.g., NLTK, SpaCy)

[1265] Analysis: Generative AI models (e.g., OpenAI GPT-4, Google BERT)

[1266] Database: MySQL, PostgreSQL

[1267] Specific actions

[1268] 1. Receive user input.

[1269] Users access the support portal using their smartphones or computers. For example, a user opens a browser and visits the support portal's URL.

[1270] The user enters their problem or question into the input form. For example, they might enter, "My smartphone screen goes dark."

[1271] 2. Sending input data

[1272] The terminal sends the entered data to the server. This communication uses the HTTPS protocol.

[1273] 3. Data preprocessing

[1274] When the server receives data, it first performs preprocessing. For example, it might use a Python library to remove unnecessary spaces and special characters from the input data.

[1275] 4. AI-based analysis

[1276] The server inputs pre-processed data into a generating AI model to identify problems and search for solutions. It also searches databases to supplement relevant information.

[1277] 5. Providing solutions

[1278] The server sends the identified solution back to the terminal. This information is exchanged in JSON format.

[1279] The device displays the received solution to the user. For example, it may show specific steps such as "Disable automatic brightness adjustment in settings."

[1280] 6. Feedback and additional support

[1281] The user tries the suggested solution and provides feedback on the results. For example, if the problem is not resolved, they can enter "Not resolved" in the form and submit it.

[1282] The device sends this feedback to the server.

[1283] The server receives feedback and provides additional support options as needed, including real-time communication and remote assistance.

[1284] Specific example

[1285] This shows the processing flow when a user enters a problem such as "My smartphone won't charge."

[1286] 1. The user enters "My smartphone won't charge."

[1287] 2. The terminal sends this input to the server.

[1288] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[1289] 4. The server sends the solution back to the terminal, and the terminal displays it to the user.

[1290] 5. The user tries the suggested solution and provides feedback stating that it "does not solve the problem."

[1291] 6. The server receives this feedback and provides real-time communication as an additional support option.

[1292] Example of a prompt:

[1293] User: My smartphone won't charge.

[1294] Server: Please check the charger connection.

[1295] User: I tried it, but it didn't solve the problem.

[1296] Server: Please try using a different cable. If that doesn't solve the problem, we will provide additional support via real-time communication.

[1297] This provides users with a series of tools to efficiently solve problems from home, making it easier to use and providing convenient support for users who live far away or have physical limitations.

[1298] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1299] Step 1:

[1300] The user enters information.

[1301] Users access the support portal via smartphones or computers.

[1302] Input: The user accesses the support portal and enters their inquiry. For example, they might enter, "My smartphone screen is dimming."

[1303] Output: User input is sent to the system via an HTML form.

[1304] Step 2:

[1305] The terminal receives input data and sends it to the server.

[1306] The terminal receives user input and sends it to the server.

[1307] Input: Inquiry data entered by the user in an HTML form.

[1308] Output: JSON formatted data sent to the server via the HTTPS protocol.

[1309] In terms of specific operations, a web browser or dedicated application converts the input data into an appropriate format and sends it to the server.

[1310] Step 3:

[1311] The server performs data preprocessing upon receiving data.

[1312] The server preprocesses the data it receives.

[1313] Input: Query data in JSON format sent from the terminal.

[1314] Output: Preprocessed data (data suitable for analysis, with unnecessary spaces and special characters removed).

[1315] Specifically, data cleaning is performed using Python's NLTK and SpaCy.

[1316] Step 4:

[1317] The server generates data and analyzes it using an AI model.

[1318] The server generates pre-processed data, which is then input into an AI model to identify problems and search for solutions.

[1319] Input: Pre-processed query data.

[1320] Output: Solutions identified by a generative AI model (e.g., OpenAI GPT-4).

[1321] Specifically, the server sends API requests to the generated AI model to analyze the problem and search for solutions.

[1322] Step 5:

[1323] The server searches the database to retrieve supplementary information.

[1324] The server searches the database and retrieves information to complement the solutions obtained from the generated AI model.

[1325] Input: Candidate solutions obtained from a generative AI model.

[1326] Output: Detailed information related to the optimal solution.

[1327] The server queries the database (MySQL, PostgreSQL) to retrieve additional relevant information.

[1328] Step 6:

[1329] The server sends the solution back to the terminal.

[1330] The server sends the identified solution back to the terminal.

[1331] Input: Generative AI models and solutions obtained from databases.

[1332] Output: Data in JSON format containing the solution to be displayed to the user.

[1333] Specifically, the server generates JSON data and sends it back to the terminal via the HTTPS protocol.

[1334] Step 7:

[1335] The device displays the solution to the user.

[1336] The terminal displays the solution it received from the server to the user.

[1337] Input: Resolution data in JSON format sent from the server.

[1338] Output: Solutions displayed in a user-friendly format.

[1339] A web browser or dedicated application parses the JSON data and displays the solution on the screen.

[1340] Step 8:

[1341] Users provide feedback

[1342] Users try the solutions and provide feedback on the results.

[1343] Input: Results and feedback on solutions attempted by the user.

[1344] Output: User feedback data is sent to the system.

[1345] Specific actions include entering "Not resolved" in the feedback form.

[1346] Step 9:

[1347] The server receives feedback and provides additional support.

[1348] The server receives user feedback and presents additional support options.

[1349] Input: User feedback data.

[1350] Output: Additional support options (real-time communication and remote operation assistance).

[1351] Specifically, the server will use the AI ​​model again to analyze the situation and provide the most suitable additional support options.

[1352] This allows the system to provide quick and accurate solutions to user inquiries, as well as offer additional support as needed.

[1353] (Application Example 1)

[1354] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1355] Traditional customer support systems often required users to visit physical stores, which was inconvenient, especially for users living far away or those with physical limitations. Even when remote support systems existed, they frequently lacked the necessary information and tools to resolve problems. This resulted in decreased support efficiency and lower customer satisfaction.

[1356] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1357] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the received inquiry data, means for analyzing the problem using a database and a generative AI model based on the pre-processed data and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback on the solution, means for presenting additional support options based on the user's feedback, and means for the user to receive remote support regarding physical stores via their smartphone. This makes it possible for users to quickly and accurately receive inquiries and support regarding physical stores remotely, even from home.

[1358] "User-entered inquiry data" refers to information that users enter into the system when seeking support, and includes data that details the question or problem.

[1359] "Preprocessing" is the process of removing unnecessary spaces and special characters from received query data and converting it into a format suitable for data analysis.

[1360] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and predict how to solve a problem based on a specific algorithm.

[1361] A "database" is a collection of information where query data and solutions are stored, and which can be used for searching and analysis.

[1362] "Searching for solutions" is the process of identifying appropriate solutions using databases and generative AI models based on pre-processed data.

[1363] "Returning the solution to the user's device" refers to the process of sending the identified solution to the user's device for display.

[1364] "User feedback" refers to the opinions and results that users provide in response to the solutions offered.

[1365] "Additional support options" are further support measures provided based on user feedback, such as live chat and remote access.

[1366] "Remote support" refers to a service that allows users to receive support via the internet without needing to visit a physical store.

[1367] This invention relates to a system that allows users to receive remote support via their smartphones without having to visit a physical store. This system receives user inquiry data, preprocesses that data, uses a generated AI model and database to analyze the problem and search for solutions, and finally provides the user with a solution.

[1368] Hardware and software to be used

[1369] 1. Hardware

[1370] Smartphone: Used as a terminal for users to enter inquiry data.

[1371] Server: Receives, preprocesses, analyzes, and processes user feedback.

[1372] 2. Software

[1373] Smartphone application: An application (iOS, Android) for entering details of questions or problems and receiving solutions.

[1374] Server-side AI models: Data analysis and prediction of problem-solving methods (TensorFlow, PyTorch).

[1375] Database: Stores and searches user query history and solutions (MySQL, PostgreSQL).

[1376] System operation

[1377] 1. User input of inquiry data

[1378] The user opens the smartphone application and enters inquiry data. For example, they might enter a question such as, "I don't know how to return a product."

[1379] 2. Receiving and preprocessing data by the server

[1380] The smartphone application sends the entered query data to the server. The server first preprocesses the data, removing unnecessary spaces and special characters.

[1381] 3. Analysis using generative AI models

[1382] The pre-processed data is input into a generating AI model, where the problem is analyzed and solutions are searched for.

[1383] 4. Providing solutions

[1384] The server searches the database for the best solution based on the analysis results and sends it back to the user's smartphone application. For example, a solution such as "Please access the returns page, fill out the returns form, and submit it" might be displayed.

[1385] 5. Receiving feedback and providing additional support

[1386] The user tries the suggested solution and provides feedback on the results. The server receives the feedback and provides additional support options such as live chat or remote access as needed.

[1387] Examples of specific cases and prompt statements

[1388] Specific example:

[1389] If a user enters "I don't know how to return the product," the server preprocesses the data and uses a generation AI model to search for a solution such as "Please access the returns page, fill out the return form, and submit it," and displays it on the user's smartphone.

[1390] Example of a prompt:

[1391] "User input: 'I don't know how to return the product' -> Provide a solution"

[1392] In this way, users can receive quick and accurate inquiries and support regarding physical stores remotely, even from the comfort of their homes. This system is particularly useful for users who live far away or who have physical limitations.

[1393] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1394] Step 1:

[1395] The user launches the smartphone application and enters inquiry data. For example, the user enters "I don't know how to return a product" into the text field and presses the submit button. The input data (text) is retrieved by the app and sent to the next step.

[1396] Step 2:

[1397] The device (smartphone) sends user input data to the server. The transmitted data is received by the server and recorded in the log. The input includes text data, and the output is ready for preprocessing.

[1398] Step 3:

[1399] The server preprocesses the query data it receives. This preprocessing includes removing unnecessary spaces and special characters. Specifically, it removes unwanted parts from the text using regular expressions, etc. The input includes the user's raw data, and the output is clean query data.

[1400] Step 4:

[1401] Preprocessed data is input to a generating AI model on the server side. The generating AI model analyzes the preprocessed, clean data and predicts relevant solutions. Specifically, the AI ​​model (such as TensorFlow or PyTorch) analyzes the input data and outputs the corresponding solutions. The input includes preprocessed, clean data, and the output is the solutions.

[1402] Step 5:

[1403] The server references the database to supplement the solution obtained from the generated AI model with detailed data. For example, a solution such as "Access the returns page, fill out the return form, and submit it" might be identified. The input includes the output of the AI ​​model and information from the database, and the output provides a specific solution.

[1404] Step 6:

[1405] The server sends the identified solution back to the user's smartphone. The details of the solution are displayed on the user's device. The input includes data on the specific solution, and the output is the solution displayed on the user's device.

[1406] Step 7:

[1407] The user tries the suggested solution and provides feedback on the results. For example, they might enter feedback such as, "I accessed the returns page, filled out the return form, and submitted it, but it was not completed." The input includes the text of the feedback, and the output is feedback data sent to the server.

[1408] Step 8:

[1409] The server receives and analyzes user feedback. It then presents additional support options as needed. For example, it might generate a link to offer the user a live chat option. The input includes the feedback content, and the output presents additional support options.

[1410] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1411] This invention relates to a system that allows users to receive remote support for their smartphones or computers without having to visit a store. In particular, it provides a configuration that combines an emotion engine that recognizes the user's emotions and responds accordingly.

[1412] Receive user input

[1413] Users access the system and enter problems or questions they need support for. For example, they might enter, "My smartphone screen goes dark." The user's input is also sent to an emotion engine for sentiment analysis.

[1414] Server analysis of the problem

[1415] The terminal sends user input to the server, which receives the data. The server first preprocesses the input data. This preprocessing includes removing unnecessary spaces and special characters. Simultaneously, the emotion engine analyzes the user's input to determine their emotions. Once preprocessing is complete, the server uses databases and AI models to analyze the input data, identify problems, and search for solutions.

[1416] Utilizing the Emotion Engine

[1417] The emotion engine recognizes the user's emotional state and automatically adjusts solutions and responses accordingly. For example, if the user is experiencing anger or frustration, solutions will be provided more quickly, and live chat will be prioritized as an additional support option.

[1418] Providing a response

[1419] Once the server finds a solution, that information, along with the results of the emotion engine's analysis, is sent back to the user's device. The device then displays the solution received from the server and appropriate countermeasures based on the user's emotion. For example, it might display a specific solution such as "Disable automatic brightness adjustment in settings."

[1420] Follow-up

[1421] The user tries the suggested solution and provides feedback on the results. This feedback is sent back to the server from the device, and the sentiment engine is updated. If the user provides feedback that the problem is "not resolved," the server will quickly provide, for example, live chat or remote access, based on the sentiment engine's analysis.

[1422] Specific example

[1423] For example, if a user enters a problem such as "My smartphone won't charge," the server will provide a solution through the following process.

[1424] 1. The user types "My smartphone won't charge," and the emotion engine recognizes "confusion."

[1425] 2. The terminal sends this input to the server.

[1426] 3. The server preprocesses the input it receives and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[1427] 4. The server sends the solution and the sentiment engine's analysis results back to the terminal, which then displays them to the user.

[1428] 5. The user tries a solution and provides feedback that it "does not solve the problem," and the emotion engine recognizes "frustration."

[1429] 6. The server receives this feedback and, based on the sentiment engine's analysis results, quickly provides live chat.

[1430] This series of steps allows users to efficiently resolve problems from home and receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[1431] The following describes the processing flow.

[1432] Step 1:

[1433] The user accesses the system and enters a specific problem or inquiry. For example, they might enter, "My smartphone screen goes dark." This input is sent to the emotion engine, which analyzes the user's emotions.

[1434] Step 2:

[1435] The terminal sends user input data to the server along with the emotion engine. The input data includes the user's questions and problem details.

[1436] Step 3:

[1437] The server preprocesses the received input data. This preprocessing includes removing unnecessary spaces and special characters. Simultaneously, sentiment analysis is performed by the sentiment engine.

[1438] Step 4:

[1439] The emotion engine analyzes user input to identify emotions such as "confusion," "irritation," and "anxiety." It then provides the identified emotional state to the server.

[1440] Step 5:

[1441] The server analyzes the problem using databases and AI models based on pre-processed data and the results of the emotion engine's analysis. Based on the analysis, it searches for appropriate solutions.

[1442] Step 6:

[1443] The server returns the searched solution, along with the sentiment engine's analysis results, to the user's device. For example, it may include specific instructions such as "Disable automatic brightness adjustment in settings."

[1444] Step 7:

[1445] The device displays the user with solutions and corresponding responses based on their emotions. The user then tries out the suggested solutions.

[1446] Step 8:

[1447] After trying a solution, the user provides feedback on the result. For example, they might answer "yes" or "no" to the question, "Was the problem solved?"

[1448] Step 9:

[1449] The device sends user feedback to the server. This feedback includes information about the success or failure of the resolution and the user's emotional state.

[1450] Step 10:

[1451] The server receives feedback from the user, and if the feedback is "no," it provides additional support options. In this process, it takes into account the results of the emotion engine's analysis; for example, if it recognizes "frustration," it prioritizes offering live chat.

[1452] Step 11:

[1453] The user selects live chat as an additional support option. The device sends that selection to the server.

[1454] Step 12:

[1455] The server initiates a live chat session, connecting you to a specialist who provides real-time user support.

[1456] Through this series of steps, users can not only efficiently resolve problems from home but also receive appropriate responses based on emotion recognition. This significantly improves the quality of support and the user experience.

[1457] (Example 2)

[1458] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1459] In systems that provide remote support, there is a problem in that they cannot recognize the user's emotions and respond appropriately accordingly, resulting in a degraded user experience. Furthermore, traditional support systems have the challenge of not being able to provide prompt additional support based on user feedback.

[1460] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1461] In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing the data, and means for analyzing the user's emotions using an emotion analysis engine. This enables highly accurate problem analysis and solution retrieval tailored to the user's emotional state. The invention also includes means for analyzing problems and retrieving solutions using a database and generative AI models, means for returning solutions and corresponding countermeasures to the user's terminal, and means for providing additional support options based on user feedback and emotion analysis results. This improves the user experience and enables efficient and rapid problem solving.

[1462] A "user" refers to a person who accesses the system and enters inquiry data.

[1463] "Inquiry data" refers to information about problems or questions that users enter that require support.

[1464] A "terminal" refers to an information device used by a user to access a system.

[1465] A "server" refers to a primary computer system that receives query data and processes it through pre-processing, analysis, solution retrieval, sentiment analysis, and feedback.

[1466] "Preprocessing" refers to the process of removing unnecessary spaces and special characters from query data.

[1467] A "sentiment analysis engine" refers to a software component used to analyze a user's emotional state from their inquiry data.

[1468] A "database" refers to a system that stores information for problem analysis and searching for solutions.

[1469] A "generative AI model" refers to a model that uses artificial intelligence technology to generate solutions to problems based on input data.

[1470] "Solution" refers to the instructions or methods provided to resolve a problem entered by the user.

[1471] "Countermeasures" refer to solutions and additional support tailored to the user's emotional state.

[1472] "Feedback" refers to the act of a user submitting the results of trying out a provided solution and their opinions on it.

[1473] "Additional support options" refer to supplementary support methods provided to users, such as live chat and remote access.

[1474] This invention relates to a system that allows users to receive remote support via smartphone or computer without having to visit a store. In particular, it provides a configuration that combines this system with an emotion analysis engine that recognizes the user's emotions and responds accordingly.

[1475] Hardware and software to be used

[1476] This system uses the following hardware and software:

[1477] Hardware:

[1478] The device the user uses (smartphone, computer, etc.)

[1479] Server (a computer with high-performance processing capabilities)

[1480] software:

[1481] Sentiment analysis engine (e.g., BERT model using HuggingFace's Transformers library)

[1482] Database system (stores information for problem analysis and searching for solutions).

[1483] Generative AI models (e.g., GPT-4, which generate problem-solving methods based on input data)

[1484] Data processing and data calculation

[1485] 1. The user accesses the system and enters inquiry data: The user accesses the system and enters a specific problem or question. For example, they might enter, "My smartphone screen goes dark." This input data is sent to the server via the terminal.

[1486] 2. Server preprocessing: The server preprocesses the received input data. It removes unnecessary spaces and special characters and converts it into a format suitable for analysis.

[1487] 3. Emotion Analysis by Emotion Analysis Engine: The server sends the pre-processed data to the emotion analysis engine, which analyzes the user's emotional state. This determines whether the user is experiencing an emotional state such as "anger" or "confusion."

[1488] 4. Problem analysis and solution search using databases and generative AI models: The server identifies the problem based on the sentiment analysis results and searches for the optimal solution using databases and generative AI models. For example, the prompt might be in the format of, "The user says 'the smartphone screen is dimming,' and their emotion is 'frustrated.' What kind of support is best?"

[1489] 5. Sending solutions and emotional responses to the device: The server sends the found solutions and emotional responses to the device. The device receives this and presents it to the user. An example of a presentation is a specific solution such as "Disable automatic brightness adjustment in settings."

[1490] 6. User Feedback: Users attempt solutions and provide feedback to the server via their device. For example, they might send feedback such as "The problem was not resolved."

[1491] 7. Server-based feedback processing and additional support: The server processes user feedback and performs sentiment analysis again. Additional support options, such as live chat and remote access, are provided as needed.

[1492] Specific example

[1493] Specific examples are given below.

[1494] When a user reports a problem such as "my smartphone won't charge," the server provides a solution through the following process.

[1495] 1. User: Enters "My smartphone won't charge," and the emotion analysis engine recognizes "confusion."

[1496] 2. Terminal: Send this input to the server.

[1497] 3. Server: Preprocesses the input and searches the database to find solutions such as "check the charger connection" or "try a different cable."

[1498] 4. Server: Sends the solution and sentiment analysis results back to the terminal.

[1499] 5. Terminal: Displays the returned information to the user.

[1500] 6. User: Tries a solution, sends feedback that it "does not work," and the sentiment analysis engine recognizes "frustration."

[1501] 7. Server: Upon receiving this feedback, the server will promptly provide live chat based on the sentiment analysis results.

[1502] In this way, the system not only efficiently solves user problems but also enables appropriate responses through emotion recognition.

[1503] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1504] Step 1:

[1505] The user accesses the system and enters inquiry data. The user enters a specific problem or question, for example, "My smartphone screen goes dark." This input data is sent to the server via the terminal. The input is in text format, and the output is also inquiry data in text format.

[1506] Step 2:

[1507] The terminal sends user input data to the server. The terminal packets the input data in the appropriate format and quickly sends it to the server via the internet. The input is the user's text input, and the output is the input data sent to the server.

[1508] Step 3:

[1509] The server preprocesses the input data it receives. This preprocessing includes removing unnecessary spaces and special characters. The server cleans up the data using regular expression libraries and other tools. The input is the raw data sent from the terminal, and the output is the clean, preprocessed input data.

[1510] Step 4:

[1511] The server sends pre-processed data to the sentiment analysis engine, which then analyzes the user's emotional state. The sentiment analysis engine uses a BERT model, for example, the HuggingFace Transformers library. The input is pre-processed text data, and the output is the sentiment analysis result (e.g., "irritated," "confused," etc.).

[1512] Step 5:

[1513] The server analyzes the problem and searches for solutions using a database and a generative AI model based on the sentiment analysis results and preprocessed data. For example, it creates a prompt sentence such as, "The user says 'the smartphone screen is getting dark,' and their emotion is 'frustrated.' What kind of support is best?" and inputs it into a generative AI model (e.g., GPT-4). The input is preprocessed data and sentiment analysis results, and the output is a specific solution (e.g., "Disable automatic brightness adjustment in settings").

[1514] Step 6:

[1515] The server sends a solution and emotional response to the user's device. The device receives this and displays it to the user. Specifically, it displays a solution such as "Disable automatic brightness adjustment in settings" on the screen. The input is the solution and emotional response, and the output is the solution instructions displayed on the device.

[1516] Step 7:

[1517] The user attempts the suggested solution and provides feedback on the results. For example, if the solution doesn't work, the user enters "Not solved" and sends it back to the server from their device. The input is the result of attempting the solution, and the output is the feedback data sent to the server.

[1518] Step 8:

[1519] The server receives user feedback and re-analyzes it using an emotion analysis engine. Based on the feedback, the server provides additional support options (e.g., live chat or remote access). The input is the feedback data and the re-analyzed emotion data, and the output is the provision of additional support options.

[1520] This series of steps allows users to efficiently solve problems from home and receive appropriate support based on sentiment analysis.

[1521] (Application Example 2)

[1522] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1523] Traditional remote support systems often lacked the ability to respond to users' emotions, particularly those experiencing confusion or frustration, and frequently failed to provide appropriate support. This resulted in decreased user problem-solving skills and satisfaction. Furthermore, in-store customer support also faced challenges due to physical limitations that made rapid and effective responses difficult.

[1524] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving inquiry data entered by the user, means for pre-processing, means for analyzing the problem using a database or AI model and searching for a solution, means for returning the solution to the user's terminal, means for receiving user feedback, means for presenting additional support options based on user feedback, and means for performing sentiment analysis and adjusting countermeasures according to the user's emotional state. This enables customized responses according to the user's emotional state, improving the efficiency and satisfaction of customer support in physical stores.

[1525] definition statement

[1526] "Inquiry data" refers to the information that users enter into the system regarding problems or questions.

[1527] "Preprocessing" refers to the process of removing unnecessary spaces and special characters from input data and converting it into a format that is easy to analyze.

[1528] A "database" is a system that stores and manages information such as inquiry data and solutions.

[1529] An "AI model" is an algorithm that uses technologies such as machine learning and deep learning to analyze data and identify problems and find solutions.

[1530] "Emotion analysis" is a technology that infers and analyzes emotions from user input data.

[1531] "Feedback" refers to the evaluation and reporting of results that users provide regarding the solutions offered.

[1532] "Additional support options" refer to support services provided in addition to basic troubleshooting methods, such as live chat and remote access.

[1533] A "device" refers to a device used by a user, such as a smartphone or computer.

[1534] Modes for carrying out the invention

[1535] This invention is a support system for users to remotely resolve problems, and in particular, a system that recognizes the user's emotions and provides appropriate responses.

[1536] System program

[1537] When a user enters inquiry data using a smartphone or computer terminal, the data is sent to the server. The server first preprocesses the data, removing unnecessary spaces and special characters. Then, it analyzes the user's emotional state using an emotion analysis engine. Next, it uses an AI model and database to analyze the problem and search for appropriate solutions. The searched solutions and the results of the emotion analysis are sent back to the terminal and displayed to the user. The user tries the displayed solutions and provides the results as feedback. Based on this feedback, the server provides additional support options. This entire process enables customized responses tailored to the user's emotional state, improving the efficiency and satisfaction of problem solving.

[1538] Hardware and software used

[1539] The following hardware and software will be used to implement the system.

[1540] Hardware: Smartphones, computer terminals, server PCs, or cloud infrastructure (e.g., Google Cloud Platform, Amazon Web Services)

[1541] Software: Sentiment analysis engine (e.g., Google Cloud Natural Language API), AI model (e.g., OpenAI API)

[1542] Data preprocessing is performed using programming languages ​​such as Python, and a sentiment analysis engine and AI model are combined to provide problem analysis and solutions.

[1543] Specific examples and prompt statements

[1544] 1. Specific example:

[1545] When a user enters "I don't know how to use this product," the server analyzes their emotional state and recognizes "confusion." As a result, solutions such as a "link to an online product demo" or a "video guide" are provided.

[1546] 2. Example of a prompt:

[1547] "A user says, 'I don't know how to use this product.' Please suggest a solution."

[1548] This invention will digitally complement in-store customer support, enabling the provision of quick and appropriate assistance. Furthermore, it is expected to improve user satisfaction through responses based on emotion recognition.

[1549] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1550] Processing steps

[1551] Step 1:

[1552] Users use their smartphones or computer terminals to input problems or questions they need help with. This input data is sent to the server as "inquiry data." An example of input data might be, "I don't know how to use this product."

[1553] Step 2:

[1554] The server preprocesses the received query data. This preprocessing includes data manipulation to remove unnecessary spaces and special characters. This results in data in a format suitable for subsequent processing. The input is the user's query data, and the output is the preprocessed data.

[1555] Step 3:

[1556] The server sends pre-processed data to an emotion analysis engine to analyze the user's emotional state. This analysis can identify whether the user is experiencing emotions such as anxiety, confusion, or frustration. The input is pre-processed data, and the output is the analyzed emotion score and emotional state.

[1557] Step 4:

[1558] The server analyzes the problem using a database and a generative AI model based on the sentiment analysis results and pre-processed query data, and searches for solutions. In this step, it generates and inputs appropriate prompt sentences to the AI ​​model to find the best solution. The input is the sentiment analysis results and pre-processed data, and the output is the searched solution.

[1559] Step 5:

[1560] The server returns the searched solutions and sentiment analysis results to the user's terminal. Care is taken to ensure that the solutions and corresponding responses are clearly displayed to the user. The input consists of the solutions and sentiment analysis results, while the output is the solutions displayed on the user's terminal.

[1561] Step 6:

[1562] The user tries the solutions displayed on their device and provides feedback on the results. For example, they may be presented with options such as "Resolved" or "Not resolved," and the user selects one of these.

[1563] Step 7:

[1564] The server receives feedback data and provides further appropriate support options as needed. For example, if the issue is not resolved or the user's anxiety increases, additional assistance such as live chat or remote support is provided quickly. The input is user feedback, and the output is additional support options.

[1565] This series of processes allows users to receive effective support even when they are not in a physical store, leading to improved customer satisfaction.

[1566] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1567] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1568] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1569] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1570] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1571] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1572] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1573] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1574] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1575] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1576] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1577] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1578] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1579] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1580] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1581] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1582] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1583] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1584] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1585] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1586] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1587] The following is further disclosed regarding the embodiments described above.

[1588] (Claim 1)

[1589] A means of receiving inquiry data entered by the user,

[1590] The means for pre-processing the received query data,

[1591] Based on the pre-processed data described above, a means of analyzing the problem using a database or AI model and searching for a solution is provided.

[1592] The above solution is to be sent back to the user's device,

[1593] A means of receiving user feedback on the above solution,

[1594] Based on the user feedback mentioned above, we will provide additional support options.

[1595] A system that includes this.

[1596] (Claim 2)

[1597] The system according to claim 1, which includes instructions to check the charger connection and try other cables as a solution to the above problem.

[1598] (Claim 3)

[1599] The system according to claim 1, further providing live chat and remote access as additional support options.

[1600] "Example 1"

[1601] (Claim 1)

[1602] A means of receiving inquiry data entered by the user,

[1603] The means for pre-processing the received query data,

[1604] Based on the pre-processed data described above, a means of analyzing the problem using a database and a generative AI model, and searching for a solution,

[1605] The above solution is to be sent back to the user's device,

[1606] A means of receiving user feedback on the above solution,

[1607] Based on the user feedback mentioned above, we will provide additional support options.

[1608] A system that includes this.

[1609] (Claim 2)

[1610] The system according to claim 1, wherein the above solution includes instructions to check the connection of electronic devices and to try other peripheral devices.

[1611] (Claim 3)

[1612] The system according to claim 1, further providing real-time communication and remote operation assistance as additional support options.

[1613] "Application Example 1"

[1614] (Claim 1)

[1615] A means of receiving inquiry data entered by the user,

[1616] The means for pre-processing the received query data,

[1617] Based on the pre-processed data described above, a means of analyzing the problem using a database and a generative AI model, and searching for a solution,

[1618] The above solution is to be sent back to the user's device,

[1619] A means of receiving user feedback on the above solution,

[1620] Based on the user feedback mentioned above, we will provide additional support options.

[1621] A means for users to receive remote support regarding physical stores via their smartphones,

[1622] A system that includes this.

[1623] (Claim 2)

[1624] As a solution to the above, the system according to claim 1 includes instructions for remotely handling product inquiries and return procedures.

[1625] (Claim 3)

[1626] The system according to claim 1, further providing live chat and remote access as additional support options.

[1627] "Example 2 of combining an emotion engine"

[1628] (Claim 1)

[1629] A means of receiving inquiry data entered by the user,

[1630] The means for pre-processing the received query data,

[1631] Based on the pre-processed data described above, a means of analyzing the user's emotions using an emotion analysis engine,

[1632] Based on the results of the above sentiment analysis and preprocessed data, a means is provided to analyze the problem using a database and a generative AI model and search for a solution.

[1633] A means of returning the above-mentioned solutions and corresponding responses to the user's emotions to the user's device,

[1634] A means of receiving user feedback on the solutions and countermeasures returned above,

[1635] Based on the user feedback and sentiment analysis results mentioned above, a means of presenting additional support options is provided.

[1636] A system that includes this.

[1637] (Claim 2)

[1638] The system according to claim 1, which includes instructions to check the connection or try a different cable as a solution to the above problem.

[1639] (Claim 3)

[1640] The system according to claim 1, further providing live chat and remote access as additional support options.

[1641] "Application example 2 when combining with an emotional engine"

[1642] (Claim 1)

[1643] A means of receiving inquiry data entered by the user,

[1644] The means for pre-processing the received query data,

[1645] Based on the pre-processed data described above, a means of analyzing the problem using a database or AI model and searching for a solution is provided.

[1646] The above solution is to be sent back to the user's device,

[1647] A means of receiving user feedback on the above solution,

[1648] Based on the user feedback mentioned above, we will provide additional support options.

[1649] A means of performing emotion analysis and adjusting countermeasures according to the user's emotional state,

[1650] A system that includes this.

[1651] (Claim 2)

[1652] The system according to claim 1, which includes, as a solution to the above, instructions to check the charger connection and try other cables, and providing links to online demonstrations or video guides.

[1653] (Claim 3)

[1654] The system according to claim 1, further comprising the provision of live chat and remote access, and customized solution suggestions based on sentiment analysis results, as additional support options. [Explanation of Symbols]

[1655] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving inquiry data entered by the user, The means for pre-processing the received query data, Based on the pre-processed data described above, a means of analyzing the problem using a database or AI model and searching for a solution is provided. The above solution is to be sent back to the user's device, A means of receiving user feedback on the above solution, Based on the user feedback mentioned above, we will provide additional support options. A system that includes this.

2. The system according to claim 1, which includes instructions to check the charger connection and try other cables as a solution to the above problem.

3. The system according to claim 1, further providing live chat and remote access as additional support options.

Images