system

Abstract

We provide the system. [Solution] Means for receiving user information from a user terminal, Means for storing the user information, Means for processing support requests based on the aforementioned user information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, the opportunities for the elderly to use smartphones have increased. Along with this, troubles such as not knowing how to operate and being unable to change settings frequently occur. In such cases, since an urgent response is often required, a system that can provide support quickly and efficiently is necessary. However, in existing support systems, it has been difficult for users to seek support on their own, and there has often been a lack of information for the support team to respond quickly. Therefore, there is a need for a system that allows the elderly to easily receive support and enables the support team to easily obtain appropriate information.

Means for Solving the Problems

[0005] The present invention solves the above-mentioned problems by providing a system that receives user information from a user terminal, stores the user information, and processes support requests based on the user information. Specifically, the present invention provides a system comprising means for receiving user information from a user terminal, means for storing the user information, and means for processing support requests based on the user information. This system further comprises means for receiving the support request from the user terminal, and the means for processing the support request further includes means for sending a notification to an emergency contact based on the user information. With such a configuration, elderly people can easily request support, and the support team can respond quickly based on the appropriate information.

[0006] A "user terminal" is an information processing device that a user can operate.

[0007] "User information" refers to information including user identification information, contact information, and emergency contact information.

[0008] A "support request" is request data that a user sends from their device to seek support.

[0009] "Means of receiving" refers to functions that include communication means for receiving data transmitted from a user terminal.

[0010] "Means of storage" refers to technical means for storing received user information in a storage device or similar.

[0011] "Means of processing" refers to a function that performs appropriate actions (such as sending notifications) using the received and stored data.

[0012] "Emergency contact information" refers to contact details registered by the user for use in emergencies.

[0013] "Means of sending notifications" refers to technologies that include communication methods for sending messages to specific recipients. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[0036] System Overview

[0037] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and to submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[0038] User information registration process

[0039] 1. User: Information Input

[0040] The user launches a dedicated app on their smartphone and accesses the registration screen.

[0041] On the registration screen, you will enter information such as your "User ID," "Name," "Phone Number," and "Emergency Contact Number."

[0042] 2. Terminal: Information transmission

[0043] When the user presses the "Register" button, the device sends this information to the server in JSON format.

[0044] 3. Server: Data reception and storage

[0045] The server parses the received JSON data and saves the user information to the database.

[0046] The saved information is associated using the user ID as the key.

[0047] 4. Server: Completion notification

[0048] The server sends a response back to the terminal indicating that registration was successful, and displays "Registration complete" to the user.

[0049] Support request process

[0050] 1. User: Press the support button.

[0051] If a user needs support, they press the "Support button" in the app.

[0052] 2. Terminal: Sending a request

[0053] The device sends a support request, including the user ID, to the server in JSON format.

[0054] 3. Server: Request received

[0055] The server analyzes the support requests received from the terminal.

[0056] 4. Server: User information verification and notification

[0057] The server retrieves user information from the database based on the user ID and sends a notification to the support staff.

[0058] 5. Support staff: Start of support

[0059] Support staff will contact the user based on notifications received from the server and begin providing support to resolve the issue.

[0060] Specific example

[0061] For example, an elderly person, Mr. A, purchases a new smartphone and, upon first use, launches a dedicated app and enters user information. Mr. A enters information such as "User ID: A123", "Name: Taro Yamada", "Phone number: 090-1234-5678", and "Emergency contact: 090-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in its database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to Mr. A.

[0062] Afterward, if Person A has trouble using their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies Person A's information. The server then notifies the support staff, who call Person A to help resolve the problem.

[0063] In this way, this system provides an environment in which elderly people can use smartphones with peace of mind.

[0064] The following describes the processing flow.

[0065] User information registration process

[0066] Step 1:

[0067] The user launches a dedicated app on their smartphone and accesses the registration screen. The user enters information such as "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[0068] Step 2:

[0069] When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server.

[0070] Step 3:

[0071] The server receives the JSON data sent as an HTTP request.

[0072] Step 4:

[0073] The server parses the received JSON data and extracts user information. For example, it associates data using the user ID as a key.

[0074] Step 5:

[0075] The server stores the extracted user information in a database. Each user's information is uniquely managed using the user ID as the key.

[0076] Step 6:

[0077] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[0078] Step 7:

[0079] The terminal receives a response message from the server and displays "Registration complete."

[0080] Support request process

[0081] Step 1:

[0082] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[0083] Step 2:

[0084] The terminal converts the support request, including the user ID, into JSON format and sends it to the server.

[0085] Step 3:

[0086] The server receives the JSON data of the support request, which was sent as an HTTP request.

[0087] Step 4:

[0088] The server parses the received JSON data and extracts the user ID that sent the request.

[0089] Step 5:

[0090] Based on the extracted user ID, the server retrieves the corresponding user information (e.g., name, phone number, emergency contact) from the database.

[0091] Step 6:

[0092] The server generates notification messages for the support team and emergency contacts based on the acquired user information.

[0093] Step 7:

[0094] The server sends the generated notification message to the support team and emergency contacts.

[0095] Step 8:

[0096] The support team receives notification messages from the server, contacts the user, and begins providing support to resolve the issue.

[0097] Step 9:

[0098] The server generates a response message indicating that the support request was processed successfully and sends it back to the terminal.

[0099] Step 10:

[0100] The terminal receives a response message from the server and displays to the user, "Support request submitted."

[0101] (Example 1)

[0102] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0103] This invention relates to a system that supports elderly people in using smartphones with peace of mind. This user group often struggles with smartphone operation and frequently requires immediate support. However, conventional systems have complex procedures for registering user information and processing support requests, making it difficult to respond quickly, especially in emergencies. Therefore, there is a need for a system that allows for easy registration of user information and rapid notification to support staff.

[0104] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0105] In this invention, the server includes means for receiving user information from a user terminal, means for storing user information, means for processing support requests based on user information, means for the processing means to send notifications to emergency contacts, means for displaying pop-up messages based on user information, means for processing HTTPS requests and responses, and means for notifying support staff of support requests. This makes it possible for users to easily register information, have support requests processed quickly, and receive appropriate support.

[0106] A "user terminal" is a device used by a user to input information and submit support requests.

[0107] "User information" refers to personal information related to the user, such as user ID, name, phone number, and emergency contact information.

[0108] A "server" is a core computing system that stores information received from user terminals and processes support requests.

[0109] "Support staff" are individuals who receive notifications from the server and provide appropriate support to users.

[0110] A "support request" is a request that a user submits when they need support.

[0111] "Means of sending notifications to emergency contacts" refers to a means of automatically sending notifications to designated contacts in the event of an emergency, based on user information.

[0112] A "pop-up message" is a notification or message window that appears on the screen of a user's device.

[0113] "Means for processing HTTPS requests and responses" refers to communication methods for securely exchanging data between a user's terminal and a server.

[0114] "Means of sending notifications" refers to the means by which the server notifies support staff of user information and support requests.

[0115] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[0116] System Overview

[0117] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[0118] Hardware and software to be used

[0119] User terminals utilize iOS or Android® devices. These devices have a dedicated app installed, allowing for user information input and support requests. The server operates on a Linux®-based system, with programs implemented using Python. MySQL® is used for the database.

[0120] 1. Data reception and storage

[0121] Users enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact" using a dedicated smartphone app.

[0122] The terminal sends this information to the server in JSON format. The server receives the HTTPS request, parses the JSON data, and saves the user information to a MySQL database.

[0123] 2. Processing support requests

[0124] When a user needs support, they press the "Support button" in the app. This causes the device to send a support request, including the user ID, to the server.

[0125] The server parses the request and retrieves user information from the database based on the user ID. It then sends a support request notification to the support staff along with the user information.

[0126] Upon receiving this notification, support staff will begin providing specific assistance to the user via phone or messaging app.

[0127] Specific example

[0128] For example, when an elderly user purchases a new smartphone, they launch a dedicated app and enter their user information the first time they use it. The user enters information such as "User ID: user123", "Name: Taro Tanaka", "Phone number: 080-1234-5678", and "Emergency contact: 080-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in a database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to the user.

[0129] Subsequently, if the user encounters difficulties operating their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies the user's information. The server then notifies the support staff, who contact the user to provide assistance in resolving the problem.

[0130] Example of a prompt

[0131] The following are examples of prompt statements to input into a generative AI model:

[0132] Please describe a smartphone support system for the elderly. This system includes the steps from user registration via smartphone to requesting support. Please clearly explain how users input information and receive support. Also, please specify the names of the hardware and software used.

[0133] This system provides an environment where elderly people can use smartphones with peace of mind.

[0134] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0135] Step 1:

[0136] User: Information Entry

[0137] The user launches a dedicated app on their smartphone and accesses the registration screen. There, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[0138] Input: User ID, Name, Phone Number, Emergency Contact Information

[0139] Output: Entered user information

[0140] Specific action: The user enters information into a text field and presses the register button.

[0141] Step 2:

[0142] Terminal: Information transmission

[0143] When a user presses the "Register" button, the device converts the entered information into JSON format and sends it to the server as an HTTPS POST request.

[0144] Input: User information in JSON format

[0145] Output: HTTPS request to the server

[0146] Specific operation: The app serializes the input data into JSON format and sends a POST request to the URL endpoint.

[0147] Step 3:

[0148] Server: Data reception and analysis

[0149] The server receives the HTTPS request, parses the JSON data, and extracts user information.

[0150] Input: User information in JSON format sent via HTTPS request

[0151] Output: Analyzed user information

[0152] Specific operation: The server uses the json.loads function to parse the received JSON data.

[0153] Step 4:

[0154] Server: Database storage

[0155] The server stores the analyzed user information in a MySQL database. The stored information includes the user's name, phone number, and emergency contact information, with the user ID as the key.

[0156] Input: Analyzed user information

[0157] Output: User information stored in the database

[0158] Specific operation: The server generates an SQL INSERT statement and executes it against the MySQL database.

[0159] Step 5:

[0160] Server: Completion notification

[0161] The server generates a response message indicating that registration was successful and sends it back to the terminal as an HTTPS response.

[0162] Input: Success status of the registration process

[0163] Output: Notification message for successful registration

[0164] Specific operation: The server generates a JSON response and sends an HTTP response.

[0165] Step 6:

[0166] Device: Notification display

[0167] The device displays a notification message confirming successful registration as a pop-up to the user.

[0168] Input: Registration success notification message from the server

[0169] Output: Pop-up message to the user

[0170] Specific operation: The app parses the message it receives and displays a pop-up in the UI.

[0171] Step 7:

[0172] User: Press the support button

[0173] If a user needs support, they press the "Support button" in the app.

[0174] Input: Press the support button.

[0175] Output: Submitting a support request

[0176] Specific action: The user taps the support button within the app.

[0177] Step 8:

[0178] Terminal: Sending a request

[0179] The device sends a support request, including the user ID, to the server in JSON format.

[0180] Input: Support request including User ID

[0181] Output: HTTPS request to the server

[0182] Specific actions: The app retrieves the user ID, serializes the support request in JSON format, and sends a POST request.

[0183] Step 9:

[0184] Server: Request reception and analysis

[0185] The server receives the support request and parses the JSON data.

[0186] Input: Support request for JSON format as HTTPS request

[0187] Output: Analyzed support request

[0188] Specific operation: The server uses the json.loads function to parse the received JSON data.

[0189] Step 10:

[0190] Server: User information retrieval

[0191] The server retrieves user information from the database based on the user ID.

[0192] Input: User ID

[0193] Output: User information retrieved from the database

[0194] Specific operation: The server generates an SQL SELECT statement and executes it against the database.

[0195] Step 11:

[0196] Server: Notification to support staff

[0197] The server will notify the support staff via email of the retrieved user information and support request.

[0198] Input: User information, support request

[0199] Output: Notification email to support staff

[0200] Specific operation: The server generates an email using the SMTP library and sends it to the support staff.

[0201] Step 12:

[0202] Support staff: Started responding

[0203] Support staff receive notifications and contact users via phone or messaging apps to provide support.

[0204] Input: Support Request Notification

[0205] Output: Providing support to users

[0206] Specific actions: Support staff will check the email notification and use phone or messaging apps to contact the user.

[0207] (Application Example 1)

[0208] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0209] There is a need to quickly and efficiently resolve the difficulties elderly people face when using online shopping sites. In particular, a system is needed that can provide timely and appropriate support for problems during the purchase process and operation. However, with conventional systems, it is difficult for elderly people to receive prompt assistance when they get lost, and an environment for using online shopping sites safely and with peace of mind is not established.

[0210] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0211] In this invention, the server includes means for receiving user data from a user device, means for storing user data, means for processing support requests based on the user data, means for notifying communication staff of support requests in real time, and means for communication staff to provide two-way communication to the user device. This makes it possible for elderly people to quickly receive real-time support from communication staff if they encounter difficulties using an online shopping site.

[0212] A "user device" is an electronic device that allows a user to input and transmit information, and to send out support requests.

[0213] "User data" refers to a collection of data that includes user identification information, contact information, and other related information.

[0214] A "support request" is request information sent from a user's device to the server when a user requests assistance.

[0215] A "communication staff member" is a person responsible for providing real-time support to users based on their support requests.

[0216] A "server" is a central device that receives and stores information from user devices, processes support requests, and sends notifications to communication staff.

[0217] "Real-time notification" is a process in which communication staff are promptly notified immediately after a support request is made.

[0218] "Two-way communication" refers to a communication method in which both the communication staff and the user device can communicate via voice and video.

[0219] "Storage means" refers to a method or device for storing user data received from users in a database on a server.

[0220] "Processing means" refers to the means of determining an appropriate response based on user data and support requests, and then notifying the user.

[0221] System Overview

[0222] This system is a support system designed to allow elderly people to use online shopping sites with peace of mind. The system consists of user devices, a server, and communication staff. Their respective roles and functions are as follows:

[0223] User devices

[0224] A user device is an electronic device used to input and transmit user information and to send support requests. Specific examples include smartphones and tablets. A dedicated application is installed on the user device, and user data registration and support request submission are performed through this application.

[0225] server

[0226] The server is responsible for processing and storing information received from user devices and sending notifications to communication staff as needed. The main functions of the server are described below.

[0227] Information reception and storage:

[0228] The server receives user data in JSON format and stores it in the database. This stored user data is then used to process subsequent support requests.

[0229] Processing support requests:

[0230] The server receives and analyzes support requests from user devices. Based on the analysis results, it sends notifications to the appropriate communications staff.

[0231] Real-time notifications:

[0232] The server notifies communication staff in real time immediately after a support request is received, enabling rapid assistance.

[0233] Communications staff

[0234] Upon receiving notifications from the server, communication staff will quickly contact the user. Specifically, they will use two-way communication to communicate with the user's device in real time and assist in resolving the issue.

[0235] Hardware and software

[0236] Hardware:

[0237] Smartphones and tablets will be used as user devices. High-performance server machines (e.g., Dell EMC PowerEdge) are recommended for the server.

[0238] software:

[0239] The server-side program will use Flask (a Python framework) and SQLite (a database management system). The user-side application will be designed for iOS or Android.

[0240] Examples of prompt statements

[0241] The following prompt messages illustrate a scenario where a user purchases a new smartphone from an online shopping site and encounters difficulties in using it.

[0242] Example of a prompt:

[0243] "If you encounter any problems while purchasing your smartphone, please press the support button in the app. Our expert staff will provide support in real time."

[0244] Specific example

[0245] For example, suppose an elderly person, Mr. B, purchases a new smartphone from an online shopping site. When Mr. B uses it for the first time, he launches a dedicated app and enters and registers his user information. Once Mr. B sends his user data, the server receives it and stores it in a database. Later, if Mr. B encounters any problems during the purchase process, he presses the "support button." A support request is sent to the server, which receives and analyzes it and notifies the communication staff in real time. The communication staff immediately contacts Mr. B and provides support to resolve the problem through two-way communication. This system allows elderly people to use online shopping sites with peace of mind.

[0246] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0247] Step 1:

[0248] The user launches a dedicated app and enters their user data.

[0249] Input: User ID, Name, Phone Number, Emergency Contact Number

[0250] Specific operation: The user accesses the app screen and enters the required information for each field. Once the input is complete, they press the "Register" button.

[0251] Step 2:

[0252] The device sends user data to the server.

[0253] Input: User data entered by the user

[0254] Output: Data sent to the server in JSON format

[0255] Data processing: The terminal converts the input data into JSON format and sends it.

[0256] Specific operation: The device collects user data and sends it to the server by calling the API endpoint.

[0257] Step 3:

[0258] The server receives user data and stores it in the database.

[0259] Input: User data sent to the server in JSON format

[0260] Output: User data stored in the database

[0261] Data processing: Parse the received JSON data and store it in the SQLite database.

[0262] Specific operation: The server analyzes the data and saves it to the corresponding field.

[0263] Step 4:

[0264] A user submits a support request.

[0265] Input: User ID

[0266] Specific action: The user presses the "Support button" within the app.

[0267] Step 5:

[0268] The device sends a support request to the server.

[0269] Input: Support request including User ID

[0270] Output: Request data sent to the server in JSON format

[0271] Data processing: The terminal converts the user ID into JSON format and sends it.

[0272] Specific operation: The terminal consolidates the support request and calls the API endpoint to send it to the server.

[0273] Step 6:

[0274] The server receives and analyzes the support request.

[0275] Input: Support request sent to the server in JSON format

[0276] Output: User information (registered user data)

[0277] Data calculation: Analyze the received request and retrieve the corresponding user data from the database based on the user ID.

[0278] Specific operation: The server analyzes the request and retrieves user information from the database.

[0279] Step 7:

[0280] The server notifies the communication staff of the support request in real time.

[0281] Input: User information (user ID, name, phone number, emergency contact)

[0282] Output: Notification message sent to the communication staff

[0283] Data calculation: Generate a notification message based on the retrieved user information and send it to the communication staff via a dedicated notification system.

[0284] Specific operation: The server consolidates the user information into a notification message and immediately sends it to the communication staff.

[0285] Step 8:

[0286] The communication staff initiates a two-way call to the user device.

[0287] Input: Received notification message

[0288] Output: Two-way call start

[0289] Specific operation: The communication staff contacts the user based on the received information and provides support through voice calls or video calls as necessary.

[0290] Step 9:

[0291] The user contacts the communication staff and receives support.

[0292] Input: The call function of the terminal

[0293] Output: Problem solved

[0294] Specific operation: The user receives support from the communication staff and maintains the call until the problem is solved.

[0295] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0296] The present invention is a support system with an added function of recognizing the user's emotion and determining the support priority. This system is composed of a user terminal, a server, an emotion engine, and support staff. The user terminal is responsible for inputting, transmitting user information, and collecting emotion data. The server plays a central role in receiving, storing user information, processing, and notifying support requests. The emotion engine analyzes the user's emotion and sets the support priority. The support staff receives notifications from the server and provides appropriate support to the user.

[0297] Process of user information registration

[0298] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[0299] Support request process

[0300] When a user presses the "Support button" on their smartphone app, the device sends a support request, including the user ID, to the server in JSON format. The server parses the received JSON data of the support request. Next, the server retrieves user information from the database based on the user ID and processes the support request appropriately.

[0301] Introducing an emotional engine

[0302] At this point, an emotion engine is added. The emotion engine analyzes the user's voice and text data to recognize the user's emotional state. The recognized emotional state may be, for example, "joy," "sadness," or "anger." The emotion engine sends this emotional state to the server. The server then sets the priority of the support request based on this emotional state. For example, if the user is showing strong anger or sadness, the support request will be given a higher priority.

[0303] Specific examples of emotional data

[0304] For example, before the user presses the support button, the terminal inputs an audio message saying "The smartphone is running slowly and I'm frustrated" into the terminal. This audio message is analyzed by the emotion engine, and the user's emotional state is recognized as "anger". This emotion data is sent to the server, and the server sets a high priority for the support request. As a result, the support staff can quickly contact the user and provide support for problem-solving.

[0305] System Operation

[0306] The user inputs audio or text indicating their emotion into the app, and the emotion engine analyzes this and sends the appropriate emotional state to the server. The server processes the support request based on the emotional state and notifies the support staff to respond promptly. With this system, the user can accurately convey their emotions, and the support staff can quickly take appropriate measures.

[0307] As described above, the present invention provides an environment in which the elderly can use smartphones with confidence by recognizing the user's emotions and accelerating the response.

[0308] The processing flow will be described below.

[0309] Process of User Information Registration

[0310] Step 1:

[0311] The user launches the dedicated app on the smartphone and accesses the registration screen. Here, information such as "User ID", "Name", "Phone Number", and "Emergency Contact" is input.

[0312] Step 2:

[0313] When the user presses the "Register" button, the terminal converts this information into JSON format and sends it to the server.

[0314] Step 3:

[0315] The server receives the JSON data sent as an HTTP request.

[0316] Step 4:

[0317] The server parses the received JSON data and extracts user information. For example, it saves the following items in the database: "User ID," "Name," "Phone Number," and "Emergency Contact."

[0318] Step 5:

[0319] The server stores the extracted user information in a database using the user ID as the key.

[0320] Step 6:

[0321] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[0322] Step 7:

[0323] The terminal receives a response message from the server and displays "Registration complete" to the user.

[0324] Support request process

[0325] Step 1:

[0326] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[0327] Step 2:

[0328] The terminal converts the support request, including the user ID, into JSON format and sends it to the server. Simultaneously, the user's voice or text input is received and this data is also sent.

[0329] Emotional Engine Processing Process

[0330] Step 3:

[0331] The server receives the JSON data of the support request received from the terminal.

[0332] Step 4:

[0333] The server parses the received JSON data and extracts the user ID of the requester, voice data, and text data.

[0334] Step 5:

[0335] The emotion engine analyzes extracted audio and text data to recognize the user's emotional state (e.g., "joy," "sadness," "anger").

[0336] Step 6:

[0337] The emotion engine sends the recognized emotional state to the server.

[0338] Support request prioritization and notification process

[0339] Step 7:

[0340] The server prioritizes support requests based on the emotional state transmitted from the emotion engine. For example, if a user indicates "anger," the request will be given a higher priority.

[0341] Step 8:

[0342] The server retrieves the relevant user information from the database based on the support request with the assigned priority.

[0343] Step 9:

[0344] The server generates a message to notify the support team of the user information and support request.

[0345] Step 10:

[0346] The server sends the generated notification message to the support team and emergency contacts.

[0347] Support staff response process

[0348] Step 11:

[0349] Support staff receive notification messages from the server and contact the user.

[0350] Step 12:

[0351] Support staff will begin assisting users in resolving their problems.

[0352] Specific example

[0353] For example, an elderly person, Mr. A, experiences a problem with his smartphone and presses the support button. Mr. A then uses voice input to say, "My smartphone is so slow, it's frustrating." This voice data is sent from the device to the server, where the emotion engine recognizes it as "anger." Based on this information, the server sets a high priority for the support request and sends a notification to the support team. The support staff immediately contacts Mr. A and begins working to resolve the problem.

[0354] Thus, this system recognizes the user's emotions and enables the provision of prompt and appropriate support.

[0355] (Example 2)

[0356] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0357] Traditional support systems often provide uniform responses without considering the user's emotional state, resulting in a lack of prompt support, particularly for urgent issues. Furthermore, when a user is experiencing emotional distress, there's a lack of effective ways to communicate this information to support staff, raising concerns about a decline in the quality of support. This creates a challenge in providing a safe and secure environment for users, especially the elderly, who may have anxieties about digital devices.

[0358] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0359] In this invention, the server includes means for receiving user data from a user device, means for storing the user data, means for processing support requests based on the user data, and means including an emotion analysis engine for analyzing the user's emotional state and setting the priority of the support requests. This enables prompt and appropriate support in accordance with the user's emotional state, and allows for rapid response to particularly urgent problems. Furthermore, by analyzing the user's emotional information, support staff can take more appropriate action regarding the user's situation. As a result, it becomes possible to provide an environment in which elderly people and others can use digital devices with peace of mind.

[0360] A "user device" is an electronic device operated by a user, and is a device that inputs and retrieves user data.

[0361] "User data" refers to information entered by the user and used by the system, and includes user ID, name, phone number, emergency contact information, voice data, and text data.

[0362] A "request for assistance" refers to the act of a user requesting help or support, and is a request sent from the user's device to the server.

[0363] An "emotion analysis engine" is a software or hardware component that analyzes a user's voice data and text data to recognize the user's emotional state.

[0364] "Priority setting" is the process of determining the priority of responses based on the importance and urgency of the support request.

[0365] A "server" is a central processing unit that receives, stores, and processes user data and support requests, and plays a primary role in coordinating with other components.

[0366] This invention is a support system that adds a function to recognize user emotions and determine support priorities. The system consists of a user device, a server, an emotion analysis engine, and support staff.

[0367] User information registration process

[0368] First, the user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[0369] Support request process

[0370] When a user presses the "Support button" on their smartphone app, the device sends a support request containing the user ID to the server in JSON format. The server parses the received JSON data of the support request, retrieves user information from the database based on the user ID, and processes the support request appropriately.

[0371] Introducing an emotional engine

[0372] An emotion analysis engine is added to the system. Users input information indicating their emotions via voice or text into the terminal. For example, a voice message such as, "My smartphone is slow and I'm getting frustrated." The emotion analysis engine analyzes this input data. Voice data is converted to text using speech recognition technology, and then emotion analysis is performed. The emotion analysis engine sends the recognized emotional state to the server. The server sets the priority of the support request based on this emotional state. For example, if the user is indicating "anger," the support request will be given a high priority. The server notifies support staff of the priority to take prompt action, and the support staff, upon receiving the notification, provide appropriate support to the user.

[0373] Specific examples of system operation

[0374] For example, a user enters a voice message into the app saying, "My smartphone isn't working, please do something about it." The emotion analysis engine analyzes the voice data and recognizes the user's emotional state as "anger." The analysis results are sent to the server, which sets the request as a high priority. Support staff receive a notification from the server and contact the user immediately. The support staff then provides assistance to resolve the problem.

[0375] Examples of prompts for generative AI models

[0376] "Process user inquiries appropriately and analyze their emotional state. For example, if a user's voice message is 'My smartphone isn't working and I'm having trouble,' analyze what emotional state is recognized and send the results to the server."

[0377] "Recognize the emotional state of users from the text and voice data they input and prioritize support requests accordingly. For example, if a user says 'I'm in a lot of trouble,' analyze their emotional state and evaluate the priority."

[0378] This system allows users to accurately communicate their emotions, and enables support staff to respond quickly and appropriately. It also provides an environment where elderly people can use smartphones with peace of mind.

[0379] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0380] User information registration process

[0381] Step 1:

[0382] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information." The entered data is stored in the device's temporary memory.

[0383] Step 2:

[0384] When a user presses the "Register" button, the terminal converts this input information into JSON format. This conversion process organizes the entered string and numerical data into a structured format. The output is JSON data containing user information.

[0385] Step 3:

[0386] The terminal sends the generated JSON data to the server via an HTTP request. This is done through network communication. The input is the generated JSON data, and the output is the completion of sending the request to the server.

[0387] Step 4:

[0388] The server receives the JSON data and uses a JSON parser to analyze (deserialize) the data. This process extracts specific information such as user ID, name, phone number, and emergency contact information. The input is the received JSON data, and the output is the extracted user information.

[0389] Step 5:

[0390] The server saves the analyzed user information to the database. This saving process is performed by executing an insert query to the database. The input is the analyzed user information, and the output is the completion of saving to the database.

[0391] Step 6:

[0392] The server generates a response message indicating that the user information was successfully saved and sends it to the terminal. The input is the success status of the save, and the output is the completion of sending the response message.

[0393] Step 7:

[0394] The terminal receives a response message from the server and displays "Registration complete" on the screen. The input is the response message from the server, and the output is the display of the registration completion message.

[0395] Support request process

[0396] Step 1:

[0397] The user presses the "Support button" on the smartphone app. This action triggers the request creation process to begin.

[0398] Step 2:

[0399] The terminal creates a support request in JSON format, including the user ID. This data contains information indicating the user's current status. Inputs are the user ID and the press of the support button, and output is the created support request JSON data.

[0400] Step 3:

[0401] The terminal sends the generated support request to the server. HTTP requests are used via network communication. The input is the JSON data of the created support request, and the output is the completion of sending the request to the server.

[0402] Step 4:

[0403] The server parses the JSON data of the received support request. It retrieves user information from the database based on the user ID. The input is the JSON data of the support request, and the output is the retrieved user information.

[0404] Step 5:

[0405] The server processes support requests appropriately based on the retrieved user information and request details. The input is the retrieved user information and request details, and the output is the processed support request.

[0406] Introducing an emotional engine

[0407] Step 1:

[0408] The user inputs emotional information into the device via voice or text. For example, a voice message such as, "My smartphone is slow and it's frustrating." The input is the user's voice or text data, and the output is the completion of data input to the device.

[0409] Step 2:

[0410] The emotion analysis engine analyzes this input data. For example, if it's audio data, it uses speech recognition technology to convert it to text, and then performs emotion analysis on that text. The input is audio or text data, and the output is the analyzed emotional state.

[0411] Step 3:

[0412] The emotion analysis engine sends the recognized emotional state to the server. This data includes emotional states such as "joy," "sadness," and "anger." The input is the analyzed emotional state, and the output is the completion of sending the emotional state to the server.

[0413] Step 4:

[0414] The server prioritizes support requests based on the user's emotional state. For example, if a user is expressing "anger," the server will prioritize the support request higher. The input is the emotional state, and the output is the set priority.

[0415] Step 5:

[0416] The server sends notifications to support staff based on the configured priority to encourage a quick response. The input is the configured priority, and the output is the completion of sending the notification to support staff.

[0417] Step 6:

[0418] Support staff receive notifications and provide appropriate support to users. Input is a notification from the server, and output is the commencement of support for the user.

[0419] (Application Example 2)

[0420] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0421] Traditional security service systems have a problem in that they cannot respond quickly and appropriately to high-priority situations because they respond uniformly without considering the user's emotions. In particular, when a user is experiencing strong emotions such as fear, immediate action is required, but delays in making such decisions increase security risks. Furthermore, there was a lack of means to properly analyze and prioritize emotions expressed by users through voice or text. Therefore, it is necessary to provide more advanced support and improve the appropriateness and speed of emergency responses.

[0422] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving user information from a user terminal, means for storing the user information, means for processing support requests based on the user information, means for analyzing voice data or text data to recognize the emotional state, means for setting the priority of support requests based on the emotional state, and means for determining a response according to the priority and sending a notification. This enables emergency responses that take into account the user's emotions, reduces security risks, and enables faster and more appropriate responses.

[0423] A "user terminal" is an electronic device used by a user to input and transmit information.

[0424] "User information" refers to data that includes identifying information about a user.

[0425] A "support request" is data that indicates a user's request for assistance.

[0426] "Voice data" refers to data that includes voice information emitted by the user.

[0427] "Text data" refers to data that includes character information entered by the user.

[0428] "Emotional state" refers to the state of a user's emotions as analyzed from voice and text data.

[0429] "Priority" is an indicator that shows the importance of addressing a support request based on emotional state.

[0430] A "notification" is a message that informs you that a response has been taken in response to your support request.

[0431] A "security service system" is a system that handles requests for assistance and provides responses to ensure the safety of users.

[0432] This invention is a security service system that prioritizes security requests by considering user emotions and responds quickly. The system consists of a user terminal, a server, an emotion engine, and security staff.

[0433] System Configuration

[0434] User terminal

[0435] Users use electronic devices such as smartphones to input data via voice or text. For example, a smartphone app using React Native could be considered. This app receives user input and sends the data to a server.

[0436] server

[0437] The server is located in the cloud, for example, built on AWS® EC2 using Node.js and the Express framework. The server receives and processes user information, support requests, and sentiment data. Specifically, it sends the aforementioned data to the Google® Cloud Natural Language API for sentiment analysis.

[0438] Emotional Engine

[0439] The emotion engine uses the Google Cloud Natural Language API to analyze the user's voice or text data to determine their emotional state (e.g., "fear," "anger," etc.). The results of this analysis are then sent back to the server.

[0440] Security staff

[0441] The server prioritizes support requests based on their emotional state and determines how to respond. Based on priority, it sends a notification to the most appropriate security staff. Upon receiving the notification, the security staff quickly travels to the site and takes the necessary action.

[0442] Explanation of the program's processing details

[0443] The user opens a smartphone app and enters a message via voice or text saying, "I feel something suspicious is happening around my house and it's scary." The app converts this data into JSON format and sends it to a server. The server sends the voice or text data to the Google Cloud Natural Language API, where the emotion engine analyzes it as "fear." Based on this analysis, the server sets the case as high priority and determines that immediate action is required. A notification is then sent to the nearest security staff, who rush to the scene.

[0444] Example of a prompt

[0445] "When a user voice-inputs 'I feel scared because I sense suspicious activity around my house,' use the Google Cloud Natural Language API to analyze the emotion as 'fear.' Based on this result, design a program that sets the server to high priority and immediately notifies security staff."

[0446] This invention enables the reduction of security risks and the realization of faster and more appropriate responses by appropriately analyzing user emotions and performing highly accurate priority evaluations. This allows for the creation of security services that offer high reliability and safety.

[0447] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0448] Step 1:

[0449] The user launches the smartphone app and creates a security request. Specifically, the user inputs a voice or text message into the app saying, "I feel scared because I've noticed suspicious activity around my house." This input data is converted into JSON format. After receiving the input (voice or text) into the app, data converted into JSON format is generated.

[0450] Step 2:

[0451] The device sends the converted JSON data to the server. Specifically, the data is sent as an HTTP request from the app to the server. At this time, the input data is voice or text information including the user's emotions, and the output data is the JSON data sent to the server.

[0452] Step 3:

[0453] The server parses the received JSON data and sends it to the sentiment engine. Specifically, the server makes an API request to the Google Cloud Natural Language API to send data. At this time, the input data is the JSON data received by the server, and the output data is the data sent to the sentiment engine.

[0454] Step 4:

[0455] The emotion engine analyzes the user's emotional state from voice or text data. Using the Google Cloud Natural Language API, emotions such as "fear" are analyzed from the input voice or text. Here, the input is the data sent to the emotion engine, and the output is the analysis result.

[0456] Step 5:

[0457] The server receives the analysis results returned from the emotion engine and sets the priority of the support request. Specifically, a higher priority is set based on the analyzed emotional state of "fear." The input here is the emotion analysis result, and the output is the set priority.

[0458] Step 6:

[0459] The server sends notifications to the nearest security staff based on priority. Specifically, if the priority is high, the server sends an urgent notification to the security staff via the API. The input here is the configured priority, and the output is the notification sent to the security staff.

[0460] Step 7:

[0461] Security staff receive the notification and rush to the scene to respond. Security staff who receive the notification quickly arrive at the scene. In this step, the input is the notification to security staff, and the output is the on-site response.

[0462] In this way, each step works in conjunction, enabling swift and appropriate security responses that take user emotions into consideration.

[0463] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0464] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0465] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0466] [Second Embodiment]

[0467] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0468] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0469] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0470] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0471] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0472] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0473] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0474] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0475] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0476] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0477] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0478] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0479] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[0480] System Overview

[0481] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and to submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[0482] User information registration process

[0483] 1. User: Information Input

[0484] The user launches a dedicated app on their smartphone and accesses the registration screen.

[0485] On the registration screen, you will enter information such as your "User ID," "Name," "Phone Number," and "Emergency Contact Number."

[0486] 2. Terminal: Information transmission

[0487] When the user presses the "Register" button, the device sends this information to the server in JSON format.

[0488] 3. Server: Data reception and storage

[0489] The server parses the received JSON data and saves the user information to the database.

[0490] The saved information is associated using the user ID as the key.

[0491] 4. Server: Completion notification

[0492] The server sends a response back to the terminal indicating that registration was successful, and displays "Registration complete" to the user.

[0493] Support request process

[0494] 1. User: Press the support button.

[0495] If a user needs support, they press the "Support button" in the app.

[0496] 2. Terminal: Sending a request

[0497] The device sends a support request, including the user ID, to the server in JSON format.

[0498] 3. Server: Request received

[0499] The server analyzes the support requests received from the terminal.

[0500] 4. Server: User information verification and notification

[0501] The server retrieves user information from the database based on the user ID and sends a notification to the support staff.

[0502] 5. Support staff: Start of support

[0503] Support staff will contact the user based on notifications received from the server and begin providing support to resolve the issue.

[0504] Specific example

[0505] For example, an elderly person, Mr. A, purchases a new smartphone and, upon first use, launches a dedicated app and enters user information. Mr. A enters information such as "User ID: A123", "Name: Taro Yamada", "Phone number: 090-1234-5678", and "Emergency contact: 090-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in its database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to Mr. A.

[0506] Afterward, if Person A has trouble using their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies Person A's information. The server then notifies the support staff, who call Person A to help resolve the problem.

[0507] In this way, this system provides an environment in which elderly people can use smartphones with peace of mind.

[0508] The following describes the processing flow.

[0509] User information registration process

[0510] Step 1:

[0511] The user launches a dedicated app on their smartphone and accesses the registration screen. The user enters information such as "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[0512] Step 2:

[0513] When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server.

[0514] Step 3:

[0515] The server receives the JSON data sent as an HTTP request.

[0516] Step 4:

[0517] The server parses the received JSON data and extracts user information. For example, it associates data using the user ID as a key.

[0518] Step 5:

[0519] The server stores the extracted user information in a database. Each user's information is uniquely managed using the user ID as the key.

[0520] Step 6:

[0521] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[0522] Step 7:

[0523] The terminal receives a response message from the server and displays "Registration complete."

[0524] Support request process

[0525] Step 1:

[0526] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[0527] Step 2:

[0528] The terminal converts the support request, including the user ID, into JSON format and sends it to the server.

[0529] Step 3:

[0530] The server receives the JSON data of the support request, which was sent as an HTTP request.

[0531] Step 4:

[0532] The server parses the received JSON data and extracts the user ID that sent the request.

[0533] Step 5:

[0534] Based on the extracted user ID, the server retrieves the corresponding user information (e.g., name, phone number, emergency contact) from the database.

[0535] Step 6:

[0536] The server generates notification messages for the support team and emergency contacts based on the acquired user information.

[0537] Step 7:

[0538] The server sends the generated notification message to the support team and emergency contacts.

[0539] Step 8:

[0540] The support team receives notification messages from the server, contacts the user, and begins providing support to resolve the issue.

[0541] Step 9:

[0542] The server generates a response message indicating that the support request was processed successfully and sends it back to the terminal.

[0543] Step 10:

[0544] The terminal receives a response message from the server and displays to the user, "Support request submitted."

[0545] (Example 1)

[0546] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0547] This invention relates to a system that supports elderly people in using smartphones with peace of mind. This user group often struggles with smartphone operation and frequently requires immediate support. However, conventional systems have complex procedures for registering user information and processing support requests, making it difficult to respond quickly, especially in emergencies. Therefore, there is a need for a system that allows for easy registration of user information and rapid notification to support staff.

[0548] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0549] In this invention, the server includes means for receiving user information from a user terminal, means for storing user information, means for processing support requests based on user information, means for the processing means to send notifications to emergency contacts, means for displaying pop-up messages based on user information, means for processing HTTPS requests and responses, and means for notifying support staff of support requests. This makes it possible for users to easily register information, have support requests processed quickly, and receive appropriate support.

[0550] A "user terminal" is a device used by a user to input information and submit support requests.

[0551] "User information" refers to personal information related to the user, such as user ID, name, phone number, and emergency contact information.

[0552] A "server" is a core computing system that stores information received from user terminals and processes support requests.

[0553] "Support staff" are individuals who receive notifications from the server and provide appropriate support to users.

[0554] A "support request" is a request that a user submits when they need support.

[0555] "Means of sending notifications to emergency contacts" refers to a means of automatically sending notifications to designated contacts in the event of an emergency, based on user information.

[0556] A "pop-up message" is a notification or message window that appears on the screen of a user's device.

[0557] "Means for processing HTTPS requests and responses" refers to communication methods for securely exchanging data between a user's terminal and a server.

[0558] "Means of sending notifications" refers to the means by which the server notifies support staff of user information and support requests.

[0559] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[0560] System Overview

[0561] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[0562] Hardware and software to be used

[0563] User terminals are either iOS or Android devices. These devices have a dedicated app installed, allowing users to input user information and submit support requests. The server runs on a Linux-based system, and the program is implemented using Python. MySQL is used as the database.

[0564] 1. Data reception and storage

[0565] Users enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact" using a dedicated smartphone app.

[0566] The terminal sends this information to the server in JSON format. The server receives the HTTPS request, parses the JSON data, and saves the user information to a MySQL database.

[0567] 2. Processing support requests

[0568] When a user needs support, they press the "Support button" in the app. This causes the device to send a support request, including the user ID, to the server.

[0569] The server parses the request and retrieves user information from the database based on the user ID. It then sends a support request notification to the support staff along with the user information.

[0570] Upon receiving this notification, support staff will begin providing specific assistance to the user via phone or messaging app.

[0571] Specific example

[0572] For example, when an elderly user purchases a new smartphone, they launch a dedicated app and enter their user information the first time they use it. The user enters information such as "User ID: user123", "Name: Taro Tanaka", "Phone number: 080-1234-5678", and "Emergency contact: 080-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in a database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to the user.

[0573] Subsequently, if the user encounters difficulties operating their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies the user's information. The server then notifies the support staff, who contact the user to provide assistance in resolving the problem.

[0574] Example of a prompt

[0575] The following are examples of prompt statements to input into a generative AI model:

[0576] Please describe a smartphone support system for the elderly. This system includes the steps from user registration via smartphone to requesting support. Please clearly explain how users input information and receive support. Also, please specify the names of the hardware and software used.

[0577] This system provides an environment where elderly people can use smartphones with peace of mind.

[0578] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0579] Step 1:

[0580] User: Information Entry

[0581] The user launches a dedicated app on their smartphone and accesses the registration screen. There, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[0582] Input: User ID, Name, Phone Number, Emergency Contact Information

[0583] Output: Entered user information

[0584] Specific action: The user enters information into a text field and presses the register button.

[0585] Step 2:

[0586] Terminal: Information transmission

[0587] When a user presses the "Register" button, the device converts the entered information into JSON format and sends it to the server as an HTTPS POST request.

[0588] Input: User information in JSON format

[0589] Output: HTTPS request to the server

[0590] Specific operation: The app serializes the input data into JSON format and sends a POST request to the URL endpoint.

[0591] Step 3:

[0592] Server: Data reception and analysis

[0593] The server receives the HTTPS request, parses the JSON data, and extracts user information.

[0594] Input: User information in JSON format sent via HTTPS request

[0595] Output: Analyzed user information

[0596] Specific operation: The server uses the json.loads function to parse the received JSON data.

[0597] Step 4:

[0598] Server: Database storage

[0599] The server stores the analyzed user information in a MySQL database. The stored information includes the user's name, phone number, and emergency contact information, with the user ID as the key.

[0600] Input: Analyzed user information

[0601] Output: User information stored in the database

[0602] Specific operation: The server generates an SQL INSERT statement and executes it against the MySQL database.

[0603] Step 5:

[0604] Server: Completion notification

[0605] The server generates a response message indicating that registration was successful and sends it back to the terminal as an HTTPS response.

[0606] Input: Success status of the registration process

[0607] Output: Notification message for successful registration

[0608] Specific operation: The server generates a JSON response and sends an HTTP response.

[0609] Step 6:

[0610] Device: Notification display

[0611] The device displays a notification message confirming successful registration as a pop-up to the user.

[0612] Input: Registration success notification message from the server

[0613] Output: Pop-up message to the user

[0614] Specific operation: The app parses the message it receives and displays a pop-up in the UI.

[0615] Step 7:

[0616] User: Press the support button

[0617] If a user needs support, they press the "Support button" in the app.

[0618] Input: Press the support button.

[0619] Output: Submitting a support request

[0620] Specific action: The user taps the support button within the app.

[0621] Step 8:

[0622] Terminal: Sending a request

[0623] The device sends a support request, including the user ID, to the server in JSON format.

[0624] Input: Support request including User ID

[0625] Output: HTTPS request to the server

[0626] Specific actions: The app retrieves the user ID, serializes the support request in JSON format, and sends a POST request.

[0627] Step 9:

[0628] Server: Request reception and analysis

[0629] The server receives the support request and parses the JSON data.

[0630] Input: Support request for JSON format as HTTPS request

[0631] Output: Analyzed support request

[0632] Specific operation: The server uses the json.loads function to parse the received JSON data.

[0633] Step 10:

[0634] Server: User information retrieval

[0635] The server retrieves user information from the database based on the user ID.

[0636] Input: User ID

[0637] Output: User information retrieved from the database

[0638] Specific operation: The server generates an SQL SELECT statement and executes it against the database.

[0639] Step 11:

[0640] Server: Notification to support staff

[0641] The server will notify the support staff via email of the retrieved user information and support request.

[0642] Input: User information, support request

[0643] Output: Notification email to support staff

[0644] Specific operation: The server generates an email using the SMTP library and sends it to the support staff.

[0645] Step 12:

[0646] Support staff: Started responding

[0647] Support staff receive notifications and contact users via phone or messaging apps to provide support.

[0648] Input: Support Request Notification

[0649] Output: Providing support to users

[0650] Specific actions: Support staff will check the email notification and use phone or messaging apps to contact the user.

[0651] (Application Example 1)

[0652] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0653] There is a need to quickly and efficiently resolve the difficulties elderly people face when using online shopping sites. In particular, a system is needed that can provide timely and appropriate support for problems during the purchase process and operation. However, with conventional systems, it is difficult for elderly people to receive prompt assistance when they get lost, and an environment for using online shopping sites safely and with peace of mind is not established.

[0654] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0655] In this invention, the server includes means for receiving user data from a user device, means for storing user data, means for processing support requests based on the user data, means for notifying communication staff of support requests in real time, and means for communication staff to provide two-way communication to the user device. This makes it possible for elderly people to quickly receive real-time support from communication staff if they encounter difficulties using an online shopping site.

[0656] A "user device" is an electronic device that allows a user to input and transmit information, and to send out support requests.

[0657] "User data" refers to a collection of data that includes user identification information, contact information, and other related information.

[0658] A "support request" is request information sent from a user's device to the server when a user requests assistance.

[0659] A "communication staff member" is a person responsible for providing real-time support to users based on their support requests.

[0660] A "server" is a central device that receives and stores information from user devices, processes support requests, and sends notifications to communication staff.

[0661] "Real-time notification" is a process in which communication staff are promptly notified immediately after a support request is made.

[0662] "Two-way communication" refers to a communication method in which both the communication staff and the user device can communicate via voice and video.

[0663] "Storage means" refers to a method or device for storing user data received from users in a database on a server.

[0664] "Processing means" refers to the means of determining an appropriate response based on user data and support requests, and then notifying the user.

[0665] System Overview

[0666] This system is a support system designed to allow elderly people to use online shopping sites with peace of mind. The system consists of user devices, a server, and communication staff. Their respective roles and functions are as follows:

[0667] User devices

[0668] A user device is an electronic device used to input and transmit user information and to send support requests. Specific examples include smartphones and tablets. A dedicated application is installed on the user device, and user data registration and support request submission are performed through this application.

[0669] server

[0670] The server is responsible for processing and storing information received from user devices and sending notifications to communication staff as needed. The main functions of the server are described below.

[0671] Information reception and storage:

[0672] The server receives user data in JSON format and stores it in the database. This stored user data is then used to process subsequent support requests.

[0673] Processing support requests:

[0674] The server receives and analyzes support requests from user devices. Based on the analysis results, it sends notifications to the appropriate communications staff.

[0675] Real-time notifications:

[0676] The server notifies communication staff in real time immediately after a support request is received, enabling rapid assistance.

[0677] Communications staff

[0678] Upon receiving notifications from the server, communication staff will quickly contact the user. Specifically, they will use two-way communication to communicate with the user's device in real time and assist in resolving the issue.

[0679] Hardware and software

[0680] Hardware:

[0681] Smartphones and tablets will be used as user devices. High-performance server machines (e.g., Dell EMC PowerEdge) are recommended for the server.

[0682] software:

[0683] The server-side program will use Flask (a Python framework) and SQLite (a database management system). The user-side application will be designed for iOS or Android.

[0684] Examples of prompt statements

[0685] The following prompt messages illustrate a scenario where a user purchases a new smartphone from an online shopping site and encounters difficulties in using it.

[0686] Example of a prompt:

[0687] "If you encounter any problems while purchasing your smartphone, please press the support button in the app. Our expert staff will provide support in real time."

[0688] Specific example

[0689] For example, suppose an elderly person, Mr. B, purchases a new smartphone from an online shopping site. When Mr. B uses it for the first time, he launches a dedicated app and enters and registers his user information. Once Mr. B sends his user data, the server receives it and stores it in a database. Later, if Mr. B encounters any problems during the purchase process, he presses the "support button." A support request is sent to the server, which receives and analyzes it and notifies the communication staff in real time. The communication staff immediately contacts Mr. B and provides support to resolve the problem through two-way communication. This system allows elderly people to use online shopping sites with peace of mind.

[0690] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0691] Step 1:

[0692] The user launches a dedicated app and enters their user data.

[0693] Input: User ID, Name, Phone Number, Emergency Contact Number

[0694] Specific operation: The user accesses the app screen and enters the required information for each field. Once the input is complete, they press the "Register" button.

[0695] Step 2:

[0696] The device sends user data to the server.

[0697] Input: User data entered by the user

[0698] Output: Data sent to the server in JSON format

[0699] Data processing: The terminal converts the input data into JSON format and sends it.

[0700] Specific operation: The device collects user data and sends it to the server by calling the API endpoint.

[0701] Step 3:

[0702] The server receives user data and stores it in the database.

[0703] Input: User data sent to the server in JSON format

[0704] Output: User data stored in the database

[0705] Data processing: Parse the received JSON data and store it in the SQLite database.

[0706] Specific operation: The server analyzes the data and saves it to the corresponding field.

[0707] Step 4:

[0708] A user submits a support request.

[0709] Input: User ID

[0710] Specific action: The user presses the "Support button" within the app.

[0711] Step 5:

[0712] The device sends a support request to the server.

[0713] Input: Support request including User ID

[0714] Output: Request data sent to the server in JSON format

[0715] Data processing: The terminal converts the user ID into JSON format and sends it.

[0716] Specific operation: The device collects support requests and calls the API endpoint to send them to the server.

[0717] Step 6:

[0718] The server receives and analyzes the support request.

[0719] Input: Support request sent to the server in JSON format

[0720] Output: User information (registered user data)

[0721] Data processing: Analyzes the received request and retrieves the corresponding user data from the database based on the user ID.

[0722] Specific operation: The server parses the request and retrieves user information from the database.

[0723] Step 7:

[0724] The server notifies the communications staff of the support request in real time.

[0725] Input: User information (User ID, Name, Phone number, Emergency contact number)

[0726] Output: Notification message sent to communications staff

[0727] Data processing: Based on acquired user information, a notification message is generated and sent to the communications staff via a dedicated notification system.

[0728] Specific operation: The server compiles user information into a notification message and immediately sends it to the communications staff.

[0729] Step 8:

[0730] The communications staff initiates a two-way call with the user's device.

[0731] Input: Received notification message

[0732] Output: Two-way call start

[0733] Specific actions: Communication staff will contact the user based on the information received and provide support via voice or video call as needed.

[0734] Step 9:

[0735] The user contacts the communications staff and receives support.

[0736] Input: Terminal's calling function

[0737] Output: Problem solved

[0738] Specific action: The user receives assistance from the communications staff and maintains the call until the problem is resolved.

[0739] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0740] This invention relates to a support system that incorporates a function to recognize user emotions and determine support priorities. The system consists of a user terminal, a server, an emotion engine, and support staff. The user terminal is responsible for inputting and transmitting user information and collecting emotion data. The server plays a central role in receiving and storing user information and processing and notifying support requests. The emotion engine analyzes user emotions and sets support priorities. Support staff receive notifications from the server and provide appropriate support to the user.

[0741] User information registration process

[0742] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[0743] Support request process

[0744] When a user presses the "Support button" on their smartphone app, the device sends a support request, including the user ID, to the server in JSON format. The server parses the received JSON data of the support request. Next, the server retrieves user information from the database based on the user ID and processes the support request appropriately.

[0745] Introducing an emotional engine

[0746] At this point, an emotion engine is added. The emotion engine analyzes the user's voice and text data to recognize the user's emotional state. The recognized emotional state may be, for example, "joy," "sadness," or "anger." The emotion engine sends this emotional state to the server. The server then sets the priority of the support request based on this emotional state. For example, if the user is showing strong anger or sadness, the support request will be given a higher priority.

[0747] Specific examples of emotional data

[0748] For example, before a user presses the support button, they might input a voice message into their device saying, "My smartphone is slow and it's frustrating." This voice message is analyzed by an emotion engine, which recognizes the user's emotional state as "anger." This emotion data is sent to a server, which then prioritizes the support request. As a result, support staff can contact the user immediately and provide assistance to resolve the problem.

[0749] System operation

[0750] Users input voice or text expressing their emotions into the app, and an emotion engine analyzes this and sends the appropriate emotional state to the server. The server processes support requests based on the emotional state and notifies support staff for a prompt response. This system allows users to accurately communicate their emotions, and enables support staff to take appropriate action quickly.

[0751] In summary, the present invention provides an environment in which elderly people can use smartphones with peace of mind by recognizing the user's emotions and responding quickly.

[0752] The following describes the processing flow.

[0753] User information registration process

[0754] Step 1:

[0755] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[0756] Step 2:

[0757] When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server.

[0758] Step 3:

[0759] The server receives the JSON data sent as an HTTP request.

[0760] Step 4:

[0761] The server parses the received JSON data and extracts user information. For example, it saves the following items in the database: "User ID," "Name," "Phone Number," and "Emergency Contact."

[0762] Step 5:

[0763] The server stores the extracted user information in a database using the user ID as the key.

[0764] Step 6:

[0765] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[0766] Step 7:

[0767] The terminal receives a response message from the server and displays "Registration complete" to the user.

[0768] Support request process

[0769] Step 1:

[0770] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[0771] Step 2:

[0772] The terminal converts the support request, including the user ID, into JSON format and sends it to the server. Simultaneously, the user's voice or text input is received and this data is also sent.

[0773] Emotional Engine Processing Process

[0774] Step 3:

[0775] The server receives the JSON data of the support request received from the terminal.

[0776] Step 4:

[0777] The server parses the received JSON data and extracts the user ID of the requester, voice data, and text data.

[0778] Step 5:

[0779] The emotion engine analyzes extracted audio and text data to recognize the user's emotional state (e.g., "joy," "sadness," "anger").

[0780] Step 6:

[0781] The emotion engine sends the recognized emotional state to the server.

[0782] Support request prioritization and notification process

[0783] Step 7:

[0784] The server prioritizes support requests based on the emotional state transmitted from the emotion engine. For example, if a user indicates "anger," the request will be given a higher priority.

[0785] Step 8:

[0786] The server retrieves the relevant user information from the database based on the support request with the assigned priority.

[0787] Step 9:

[0788] The server generates a message to notify the support team of the user information and support request.

[0789] Step 10:

[0790] The server sends the generated notification message to the support team and emergency contacts.

[0791] Support staff response process

[0792] Step 11:

[0793] Support staff receive notification messages from the server and contact the user.

[0794] Step 12:

[0795] Support staff will begin assisting users in resolving their problems.

[0796] Specific example

[0797] For example, an elderly person, Mr. A, experiences a problem with his smartphone and presses the support button. Mr. A then uses voice input to say, "My smartphone is so slow, it's frustrating." This voice data is sent from the device to the server, where the emotion engine recognizes it as "anger." Based on this information, the server sets a high priority for the support request and sends a notification to the support team. The support staff immediately contacts Mr. A and begins working to resolve the problem.

[0798] Thus, this system recognizes the user's emotions and enables the provision of prompt and appropriate support.

[0799] (Example 2)

[0800] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0801] Traditional support systems often provide uniform responses without considering the user's emotional state, resulting in a lack of prompt support, particularly for urgent issues. Furthermore, when a user is experiencing emotional distress, there's a lack of effective ways to communicate this information to support staff, raising concerns about a decline in the quality of support. This creates a challenge in providing a safe and secure environment for users, especially the elderly, who may have anxieties about digital devices.

[0802] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0803] In this invention, the server includes means for receiving user data from a user device, means for storing the user data, means for processing support requests based on the user data, and means including an emotion analysis engine for analyzing the user's emotional state and setting the priority of the support requests. This enables prompt and appropriate support in accordance with the user's emotional state, and allows for rapid response to particularly urgent problems. Furthermore, by analyzing the user's emotional information, support staff can take more appropriate action regarding the user's situation. As a result, it becomes possible to provide an environment in which elderly people and others can use digital devices with peace of mind.

[0804] A "user device" is an electronic device operated by a user, and is a device that inputs and retrieves user data.

[0805] "User data" refers to information entered by the user and used by the system, and includes user ID, name, phone number, emergency contact information, voice data, and text data.

[0806] A "request for assistance" refers to the act of a user requesting help or support, and is a request sent from the user's device to the server.

[0807] An "emotion analysis engine" is a software or hardware component that analyzes a user's voice data and text data to recognize the user's emotional state.

[0808] "Priority setting" is the process of determining the priority of responses based on the importance and urgency of the support request.

[0809] A "server" is a central processing unit that receives, stores, and processes user data and support requests, and plays a primary role in coordinating with other components.

[0810] This invention is a support system that adds a function to recognize user emotions and determine support priorities. The system consists of a user device, a server, an emotion analysis engine, and support staff.

[0811] User information registration process

[0812] First, the user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[0813] Support request process

[0814] When a user presses the "Support button" on their smartphone app, the device sends a support request containing the user ID to the server in JSON format. The server parses the received JSON data of the support request, retrieves user information from the database based on the user ID, and processes the support request appropriately.

[0815] Introducing an emotional engine

[0816] An emotion analysis engine is added to the system. Users input information indicating their emotions via voice or text into the terminal. For example, a voice message such as, "My smartphone is slow and I'm getting frustrated." The emotion analysis engine analyzes this input data. Voice data is converted to text using speech recognition technology, and then emotion analysis is performed. The emotion analysis engine sends the recognized emotional state to the server. The server sets the priority of the support request based on this emotional state. For example, if the user is indicating "anger," the support request will be given a high priority. The server notifies support staff of the priority to take prompt action, and the support staff, upon receiving the notification, provide appropriate support to the user.

[0817] Specific examples of system operation

[0818] For example, a user enters a voice message into the app saying, "My smartphone isn't working, please do something about it." The emotion analysis engine analyzes the voice data and recognizes the user's emotional state as "anger." The analysis results are sent to the server, which sets the request as a high priority. Support staff receive a notification from the server and contact the user immediately. The support staff then provides assistance to resolve the problem.

[0819] Examples of prompts for generative AI models

[0820] "Process user inquiries appropriately and analyze their emotional state. For example, if a user's voice message is 'My smartphone isn't working and I'm having trouble,' analyze what emotional state is recognized and send the results to the server."

[0821] "Recognize the emotional state of users from the text and voice data they input and prioritize support requests accordingly. For example, if a user says 'I'm in a lot of trouble,' analyze their emotional state and evaluate the priority."

[0822] This system allows users to accurately communicate their emotions, and enables support staff to respond quickly and appropriately. It also provides an environment where elderly people can use smartphones with peace of mind.

[0823] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0824] User information registration process

[0825] Step 1:

[0826] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information." The entered data is stored in the device's temporary memory.

[0827] Step 2:

[0828] When a user presses the "Register" button, the terminal converts this input information into JSON format. This conversion process organizes the entered string and numerical data into a structured format. The output is JSON data containing user information.

[0829] Step 3:

[0830] The terminal sends the generated JSON data to the server via an HTTP request. This is done through network communication. The input is the generated JSON data, and the output is the completion of sending the request to the server.

[0831] Step 4:

[0832] The server receives the JSON data and uses a JSON parser to analyze (deserialize) the data. This process extracts specific information such as user ID, name, phone number, and emergency contact information. The input is the received JSON data, and the output is the extracted user information.

[0833] Step 5:

[0834] The server saves the analyzed user information to the database. This saving process is performed by executing an insert query to the database. The input is the analyzed user information, and the output is the completion of saving to the database.

[0835] Step 6:

[0836] The server generates a response message indicating that the user information was successfully saved and sends it to the terminal. The input is the success status of the save, and the output is the completion of sending the response message.

[0837] Step 7:

[0838] The terminal receives a response message from the server and displays "Registration complete" on the screen. The input is the response message from the server, and the output is the display of the registration completion message.

[0839] Support request process

[0840] Step 1:

[0841] The user presses the "Support button" on the smartphone app. This action triggers the request creation process to begin.

[0842] Step 2:

[0843] The terminal creates a support request in JSON format, including the user ID. This data contains information indicating the user's current status. Inputs are the user ID and the press of the support button, and output is the created support request JSON data.

[0844] Step 3:

[0845] The terminal sends the generated support request to the server. HTTP requests are used via network communication. The input is the JSON data of the created support request, and the output is the completion of sending the request to the server.

[0846] Step 4:

[0847] The server parses the JSON data of the received support request. It retrieves user information from the database based on the user ID. The input is the JSON data of the support request, and the output is the retrieved user information.

[0848] Step 5:

[0849] The server processes support requests appropriately based on the retrieved user information and request details. The input is the retrieved user information and request details, and the output is the processed support request.

[0850] Introducing an emotional engine

[0851] Step 1:

[0852] The user inputs emotional information into the device via voice or text. For example, a voice message such as, "My smartphone is slow and it's frustrating." The input is the user's voice or text data, and the output is the completion of data input to the device.

[0853] Step 2:

[0854] The emotion analysis engine analyzes this input data. For example, if it's audio data, it uses speech recognition technology to convert it to text, and then performs emotion analysis on that text. The input is audio or text data, and the output is the analyzed emotional state.

[0855] Step 3:

[0856] The emotion analysis engine sends the recognized emotional state to the server. This data includes emotional states such as "joy," "sadness," and "anger." The input is the analyzed emotional state, and the output is the completion of sending the emotional state to the server.

[0857] Step 4:

[0858] The server prioritizes support requests based on the user's emotional state. For example, if a user is expressing "anger," the server will prioritize the support request higher. The input is the emotional state, and the output is the set priority.

[0859] Step 5:

[0860] The server sends notifications to support staff based on the configured priority to encourage a quick response. The input is the configured priority, and the output is the completion of sending the notification to support staff.

[0861] Step 6:

[0862] Support staff receive notifications and provide appropriate support to users. Input is a notification from the server, and output is the commencement of support for the user.

[0863] (Application Example 2)

[0864] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0865] Traditional security service systems have a problem in that they cannot respond quickly and appropriately to high-priority situations because they respond uniformly without considering the user's emotions. In particular, when a user is experiencing strong emotions such as fear, immediate action is required, but delays in making such decisions increase security risks. Furthermore, there was a lack of means to properly analyze and prioritize emotions expressed by users through voice or text. Therefore, it is necessary to provide more advanced support and improve the appropriateness and speed of emergency responses.

[0866] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving user information from a user terminal, means for storing the user information, means for processing support requests based on the user information, means for analyzing voice data or text data to recognize the emotional state, means for setting the priority of support requests based on the emotional state, and means for determining a response according to the priority and sending a notification. This enables emergency responses that take into account the user's emotions, reduces security risks, and enables faster and more appropriate responses.

[0867] A "user terminal" is an electronic device used by a user to input and transmit information.

[0868] "User information" refers to data that includes identifying information about a user.

[0869] A "support request" is data that indicates a user's request for assistance.

[0870] "Voice data" refers to data that includes voice information emitted by the user.

[0871] "Text data" refers to data that includes character information entered by the user.

[0872] "Emotional state" refers to the state of a user's emotions as analyzed from voice and text data.

[0873] "Priority" is an indicator that shows the importance of addressing a support request based on emotional state.

[0874] A "notification" is a message that informs you that a response has been taken in response to your support request.

[0875] A "security service system" is a system that handles requests for assistance and provides responses to ensure the safety of users.

[0876] This invention is a security service system that prioritizes security requests by considering user emotions and responds quickly. The system consists of a user terminal, a server, an emotion engine, and security staff.

[0877] System Configuration

[0878] User terminal

[0879] Users use electronic devices such as smartphones to input data via voice or text. For example, a smartphone app using React Native could be considered. This app receives user input and sends the data to a server.

[0880] server

[0881] The server is located in the cloud, for example, built on AWS EC2 using Node.js and the Express framework. The server receives and processes user information, support requests, and sentiment data. Specifically, it sends the aforementioned data to the Google Cloud Natural Language API for sentiment analysis.

[0882] Emotional Engine

[0883] The emotion engine uses the Google Cloud Natural Language API to analyze the user's voice or text data to determine their emotional state (e.g., "fear," "anger," etc.). The results of this analysis are then sent back to the server.

[0884] Security staff

[0885] The server prioritizes support requests based on their emotional state and determines how to respond. Based on priority, it sends a notification to the most appropriate security staff. Upon receiving the notification, the security staff quickly travels to the site and takes the necessary action.

[0886] Explanation of the program's processing details

[0887] The user opens a smartphone app and enters a message via voice or text saying, "I feel something suspicious is happening around my house and it's scary." The app converts this data into JSON format and sends it to a server. The server sends the voice or text data to the Google Cloud Natural Language API, where the emotion engine analyzes it as "fear." Based on this analysis, the server sets the case as high priority and determines that immediate action is required. A notification is then sent to the nearest security staff, who rush to the scene.

[0888] Example of a prompt

[0889] "When a user voice-inputs 'I feel scared because I sense suspicious activity around my house,' use the Google Cloud Natural Language API to analyze the emotion as 'fear.' Based on this result, design a program that sets the server to high priority and immediately notifies security staff."

[0890] This invention enables the reduction of security risks and the realization of faster and more appropriate responses by appropriately analyzing user emotions and performing highly accurate priority evaluations. This allows for the creation of security services that offer high reliability and safety.

[0891] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0892] Step 1:

[0893] The user launches the smartphone app and creates a security request. Specifically, the user inputs a voice or text message into the app saying, "I feel scared because I've noticed suspicious activity around my house." This input data is converted into JSON format. After receiving the input (voice or text) into the app, data converted into JSON format is generated.

[0894] Step 2:

[0895] The device sends the converted JSON data to the server. Specifically, the data is sent as an HTTP request from the app to the server. At this time, the input data is voice or text information including the user's emotions, and the output data is the JSON data sent to the server.

[0896] Step 3:

[0897] The server parses the received JSON data and sends it to the sentiment engine. Specifically, the server makes an API request to the Google Cloud Natural Language API to send data. At this time, the input data is the JSON data received by the server, and the output data is the data sent to the sentiment engine.

[0898] Step 4:

[0899] The emotion engine analyzes the user's emotional state from voice or text data. Using the Google Cloud Natural Language API, emotions such as "fear" are analyzed from the input voice or text. Here, the input is the data sent to the emotion engine, and the output is the analysis result.

[0900] Step 5:

[0901] The server receives the analysis results returned from the emotion engine and sets the priority of the support request. Specifically, a higher priority is set based on the analyzed emotional state of "fear." The input here is the emotion analysis result, and the output is the set priority.

[0902] Step 6:

[0903] The server sends notifications to the nearest security staff based on priority. Specifically, if the priority is high, the server sends an urgent notification to the security staff via the API. The input here is the configured priority, and the output is the notification sent to the security staff.

[0904] Step 7:

[0905] Security staff receive the notification and rush to the scene to respond. Security staff who receive the notification quickly arrive at the scene. In this step, the input is the notification to security staff, and the output is the on-site response.

[0906] In this way, each step works in conjunction, enabling swift and appropriate security responses that take user emotions into consideration.

[0907] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0908] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0909] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0910] [Third Embodiment]

[0911] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0912] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0913] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0914] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0915] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0916] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0917] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0918] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0919] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0920] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0921] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0922] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0923] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[0924] System Overview

[0925] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and to submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[0926] User information registration process

[0927] 1. User: Information Input

[0928] The user launches a dedicated app on their smartphone and accesses the registration screen.

[0929] On the registration screen, you will enter information such as your "User ID," "Name," "Phone Number," and "Emergency Contact Number."

[0930] 2. Terminal: Information transmission

[0931] When the user presses the "Register" button, the device sends this information to the server in JSON format.

[0932] 3. Server: Data reception and storage

[0933] The server parses the received JSON data and saves the user information to the database.

[0934] The saved information is associated using the user ID as the key.

[0935] 4. Server: Completion notification

[0936] The server sends a response back to the terminal indicating that registration was successful, and displays "Registration complete" to the user.

[0937] Support request process

[0938] 1. User: Press the support button.

[0939] If a user needs support, they press the "Support button" in the app.

[0940] 2. Terminal: Sending a request

[0941] The device sends a support request, including the user ID, to the server in JSON format.

[0942] 3. Server: Request received

[0943] The server analyzes the support requests received from the terminal.

[0944] 4. Server: User information verification and notification

[0945] The server retrieves user information from the database based on the user ID and sends a notification to the support staff.

[0946] 5. Support staff: Start of support

[0947] Support staff will contact the user based on notifications received from the server and begin providing support to resolve the issue.

[0948] Specific example

[0949] For example, an elderly person, Mr. A, purchases a new smartphone and, upon first use, launches a dedicated app and enters user information. Mr. A enters information such as "User ID: A123", "Name: Taro Yamada", "Phone number: 090-1234-5678", and "Emergency contact: 090-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in its database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to Mr. A.

[0950] Afterward, if Person A has trouble using their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies Person A's information. The server then notifies the support staff, who call Person A to help resolve the problem.

[0951] In this way, this system provides an environment in which elderly people can use smartphones with peace of mind.

[0952] The following describes the processing flow.

[0953] User information registration process

[0954] Step 1:

[0955] The user launches a dedicated app on their smartphone and accesses the registration screen. The user enters information such as "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[0956] Step 2:

[0957] When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server.

[0958] Step 3:

[0959] The server receives the JSON data sent as an HTTP request.

[0960] Step 4:

[0961] The server parses the received JSON data and extracts user information. For example, it associates data using the user ID as a key.

[0962] Step 5:

[0963] The server stores the extracted user information in a database. Each user's information is uniquely managed using the user ID as the key.

[0964] Step 6:

[0965] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[0966] Step 7:

[0967] The terminal receives a response message from the server and displays "Registration complete."

[0968] Support request process

[0969] Step 1:

[0970] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[0971] Step 2:

[0972] The terminal converts the support request, including the user ID, into JSON format and sends it to the server.

[0973] Step 3:

[0974] The server receives the JSON data of the support request, which was sent as an HTTP request.

[0975] Step 4:

[0976] The server parses the received JSON data and extracts the user ID that sent the request.

[0977] Step 5:

[0978] Based on the extracted user ID, the server retrieves the corresponding user information (e.g., name, phone number, emergency contact) from the database.

[0979] Step 6:

[0980] The server generates notification messages for the support team and emergency contacts based on the acquired user information.

[0981] Step 7:

[0982] The server sends the generated notification message to the support team and emergency contacts.

[0983] Step 8:

[0984] The support team receives notification messages from the server, contacts the user, and begins providing support to resolve the issue.

[0985] Step 9:

[0986] The server generates a response message indicating that the support request was processed successfully and sends it back to the terminal.

[0987] Step 10:

[0988] The terminal receives a response message from the server and displays to the user, "Support request submitted."

[0989] (Example 1)

[0990] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0991] This invention relates to a system that supports elderly people in using smartphones with peace of mind. This user group often struggles with smartphone operation and frequently requires immediate support. However, conventional systems have complex procedures for registering user information and processing support requests, making it difficult to respond quickly, especially in emergencies. Therefore, there is a need for a system that allows for easy registration of user information and rapid notification to support staff.

[0992] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0993] In this invention, the server includes means for receiving user information from a user terminal, means for storing user information, means for processing support requests based on user information, means for the processing means to send notifications to emergency contacts, means for displaying pop-up messages based on user information, means for processing HTTPS requests and responses, and means for notifying support staff of support requests. This makes it possible for users to easily register information, have support requests processed quickly, and receive appropriate support.

[0994] A "user terminal" is a device used by a user to input information and submit support requests.

[0995] "User information" refers to personal information related to the user, such as user ID, name, phone number, and emergency contact information.

[0996] A "server" is a core computing system that stores information received from user terminals and processes support requests.

[0997] "Support staff" are individuals who receive notifications from the server and provide appropriate support to users.

[0998] A "support request" is a request that a user submits when they need support.

[0999] "Means of sending notifications to emergency contacts" refers to a means of automatically sending notifications to designated contacts in the event of an emergency, based on user information.

[1000] A "pop-up message" is a notification or message window that appears on the screen of a user's device.

[1001] "Means for processing HTTPS requests and responses" refers to communication methods for securely exchanging data between a user's terminal and a server.

[1002] "Means of sending notifications" refers to the means by which the server notifies support staff of user information and support requests.

[1003] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[1004] System Overview

[1005] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[1006] Hardware and software to be used

[1007] User terminals are either iOS or Android devices. These devices have a dedicated app installed, allowing users to input user information and submit support requests. The server runs on a Linux-based system, and the program is implemented using Python. MySQL is used as the database.

[1008] 1. Data reception and storage

[1009] Users enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact" using a dedicated smartphone app.

[1010] The terminal sends this information to the server in JSON format. The server receives the HTTPS request, parses the JSON data, and saves the user information to a MySQL database.

[1011] 2. Processing support requests

[1012] When a user needs support, they press the "Support button" in the app. This causes the device to send a support request, including the user ID, to the server.

[1013] The server parses the request and retrieves user information from the database based on the user ID. It then sends a support request notification to the support staff along with the user information.

[1014] Upon receiving this notification, support staff will begin providing specific assistance to the user via phone or messaging app.

[1015] Specific example

[1016] For example, when an elderly user purchases a new smartphone, they launch a dedicated app and enter their user information the first time they use it. The user enters information such as "User ID: user123", "Name: Taro Tanaka", "Phone number: 080-1234-5678", and "Emergency contact: 080-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in a database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to the user.

[1017] Subsequently, if the user encounters difficulties operating their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies the user's information. The server then notifies the support staff, who contact the user to provide assistance in resolving the problem.

[1018] Example of a prompt

[1019] The following are examples of prompt statements to input into a generative AI model:

[1020] Please describe a smartphone support system for the elderly. This system includes the steps from user registration via smartphone to requesting support. Please clearly explain how users input information and receive support. Also, please specify the names of the hardware and software used.

[1021] This system provides an environment where elderly people can use smartphones with peace of mind.

[1022] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1023] Step 1:

[1024] User: Information Entry

[1025] The user launches a dedicated app on their smartphone and accesses the registration screen. There, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[1026] Input: User ID, Name, Phone Number, Emergency Contact Information

[1027] Output: Entered user information

[1028] Specific action: The user enters information into a text field and presses the register button.

[1029] Step 2:

[1030] Terminal: Information transmission

[1031] When a user presses the "Register" button, the device converts the entered information into JSON format and sends it to the server as an HTTPS POST request.

[1032] Input: User information in JSON format

[1033] Output: HTTPS request to the server

[1034] Specific operation: The app serializes the input data into JSON format and sends a POST request to the URL endpoint.

[1035] Step 3:

[1036] Server: Data reception and analysis

[1037] The server receives the HTTPS request, parses the JSON data, and extracts user information.

[1038] Input: User information in JSON format sent via HTTPS request

[1039] Output: Analyzed user information

[1040] Specific operation: The server uses the json.loads function to parse the received JSON data.

[1041] Step 4:

[1042] Server: Database storage

[1043] The server stores the analyzed user information in a MySQL database. The stored information includes the user's name, phone number, and emergency contact information, with the user ID as the key.

[1044] Input: Analyzed user information

[1045] Output: User information stored in the database

[1046] Specific operation: The server generates an SQL INSERT statement and executes it against the MySQL database.

[1047] Step 5:

[1048] Server: Completion notification

[1049] The server generates a response message indicating that registration was successful and sends it back to the terminal as an HTTPS response.

[1050] Input: Success status of the registration process

[1051] Output: Notification message for successful registration

[1052] Specific operation: The server generates a JSON response and sends an HTTP response.

[1053] Step 6:

[1054] Device: Notification display

[1055] The device displays a notification message confirming successful registration as a pop-up to the user.

[1056] Input: Registration success notification message from the server

[1057] Output: Pop-up message to the user

[1058] Specific operation: The app parses the message it receives and displays a pop-up in the UI.

[1059] Step 7:

[1060] User: Press the support button

[1061] If a user needs support, they press the "Support button" in the app.

[1062] Input: Press the support button.

[1063] Output: Submitting a support request

[1064] Specific action: The user taps the support button within the app.

[1065] Step 8:

[1066] Terminal: Sending a request

[1067] The device sends a support request, including the user ID, to the server in JSON format.

[1068] Input: Support request including User ID

[1069] Output: HTTPS request to the server

[1070] Specific actions: The app retrieves the user ID, serializes the support request in JSON format, and sends a POST request.

[1071] Step 9:

[1072] Server: Request reception and analysis

[1073] The server receives the support request and parses the JSON data.

[1074] Input: Support request for JSON format as HTTPS request

[1075] Output: Analyzed support request

[1076] Specific operation: The server uses the json.loads function to parse the received JSON data.

[1077] Step 10:

[1078] Server: User information retrieval

[1079] The server retrieves user information from the database based on the user ID.

[1080] Input: User ID

[1081] Output: User information retrieved from the database

[1082] Specific operation: The server generates an SQL SELECT statement and executes it against the database.

[1083] Step 11:

[1084] Server: Notification to support staff

[1085] The server will notify the support staff via email of the retrieved user information and support request.

[1086] Input: User information, support request

[1087] Output: Notification email to support staff

[1088] Specific operation: The server generates an email using the SMTP library and sends it to the support staff.

[1089] Step 12:

[1090] Support staff: Started responding

[1091] Support staff receive notifications and contact users via phone or messaging apps to provide support.

[1092] Input: Support Request Notification

[1093] Output: Providing support to users

[1094] Specific actions: Support staff will check the email notification and use phone or messaging apps to contact the user.

[1095] (Application Example 1)

[1096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1097] There is a need to quickly and efficiently resolve the difficulties elderly people face when using online shopping sites. In particular, a system is needed that can provide timely and appropriate support for problems during the purchase process and operation. However, with conventional systems, it is difficult for elderly people to receive prompt assistance when they get lost, and an environment for using online shopping sites safely and with peace of mind is not established.

[1098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1099] In this invention, the server includes means for receiving user data from a user device, means for storing user data, means for processing support requests based on the user data, means for notifying communication staff of support requests in real time, and means for communication staff to provide two-way communication to the user device. This makes it possible for elderly people to quickly receive real-time support from communication staff if they encounter difficulties using an online shopping site.

[1100] A "user device" is an electronic device that allows a user to input and transmit information, and to send out support requests.

[1101] "User data" refers to a collection of data that includes user identification information, contact information, and other related information.

[1102] A "support request" is request information sent from a user's device to the server when a user requests assistance.

[1103] A "communication staff member" is a person responsible for providing real-time support to users based on their support requests.

[1104] A "server" is a central device that receives and stores information from user devices, processes support requests, and sends notifications to communication staff.

[1105] "Real-time notification" is a process in which communication staff are promptly notified immediately after a support request is made.

[1106] "Two-way communication" refers to a communication method in which both the communication staff and the user device can communicate via voice and video.

[1107] "Storage means" refers to a method or device for storing user data received from users in a database on a server.

[1108] "Processing means" refers to the means of determining an appropriate response based on user data and support requests, and then notifying the user.

[1109] System Overview

[1110] This system is a support system designed to allow elderly people to use online shopping sites with peace of mind. The system consists of user devices, a server, and communication staff. Their respective roles and functions are as follows:

[1111] User devices

[1112] A user device is an electronic device used to input and transmit user information and to send support requests. Specific examples include smartphones and tablets. A dedicated application is installed on the user device, and user data registration and support request submission are performed through this application.

[1113] server

[1114] The server is responsible for processing and storing information received from user devices and sending notifications to communication staff as needed. The main functions of the server are described below.

[1115] Information reception and storage:

[1116] The server receives user data in JSON format and stores it in the database. This stored user data is then used to process subsequent support requests.

[1117] Processing support requests:

[1118] The server receives and analyzes support requests from user devices. Based on the analysis results, it sends notifications to the appropriate communications staff.

[1119] Real-time notifications:

[1120] The server notifies communication staff in real time immediately after a support request is received, enabling rapid assistance.

[1121] Communications staff

[1122] Upon receiving notifications from the server, communication staff will quickly contact the user. Specifically, they will use two-way communication to communicate with the user's device in real time and assist in resolving the issue.

[1123] Hardware and software

[1124] Hardware:

[1125] Smartphones and tablets will be used as user devices. High-performance server machines (e.g., Dell EMC PowerEdge) are recommended for the server.

[1126] software:

[1127] The server-side program will use Flask (a Python framework) and SQLite (a database management system). The user-side application will be designed for iOS or Android.

[1128] Examples of prompt statements

[1129] The following prompt messages illustrate a scenario where a user purchases a new smartphone from an online shopping site and encounters difficulties in using it.

[1130] Example of a prompt:

[1131] "If you encounter any problems while purchasing your smartphone, please press the support button in the app. Our expert staff will provide support in real time."

[1132] Specific example

[1133] For example, suppose an elderly person, Mr. B, purchases a new smartphone from an online shopping site. When Mr. B uses it for the first time, he launches a dedicated app and enters and registers his user information. Once Mr. B sends his user data, the server receives it and stores it in a database. Later, if Mr. B encounters any problems during the purchase process, he presses the "support button." A support request is sent to the server, which receives and analyzes it and notifies the communication staff in real time. The communication staff immediately contacts Mr. B and provides support to resolve the problem through two-way communication. This system allows elderly people to use online shopping sites with peace of mind.

[1134] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1135] Step 1:

[1136] The user launches a dedicated app and enters their user data.

[1137] Input: User ID, Name, Phone Number, Emergency Contact Number

[1138] Specific operation: The user accesses the app screen and enters the required information for each field. Once the input is complete, they press the "Register" button.

[1139] Step 2:

[1140] The device sends user data to the server.

[1141] Input: User data entered by the user

[1142] Output: Data sent to the server in JSON format

[1143] Data processing: The terminal converts the input data into JSON format and sends it.

[1144] Specific operation: The device collects user data and sends it to the server by calling the API endpoint.

[1145] Step 3:

[1146] The server receives user data and stores it in the database.

[1147] Input: User data sent to the server in JSON format

[1148] Output: User data stored in the database

[1149] Data processing: Parse the received JSON data and store it in the SQLite database.

[1150] Specific operation: The server analyzes the data and saves it to the corresponding field.

[1151] Step 4:

[1152] A user submits a support request.

[1153] Input: User ID

[1154] Specific action: The user presses the "Support button" within the app.

[1155] Step 5:

[1156] The device sends a support request to the server.

[1157] Input: Support request including User ID

[1158] Output: Request data sent to the server in JSON format

[1159] Data processing: The terminal converts the user ID into JSON format and sends it.

[1160] Specific operation: The device collects support requests and calls the API endpoint to send them to the server.

[1161] Step 6:

[1162] The server receives and analyzes the support request.

[1163] Input: Support request sent to the server in JSON format

[1164] Output: User information (registered user data)

[1165] Data processing: Analyzes the received request and retrieves the corresponding user data from the database based on the user ID.

[1166] Specific operation: The server parses the request and retrieves user information from the database.

[1167] Step 7:

[1168] The server notifies the communications staff of the support request in real time.

[1169] Input: User information (User ID, Name, Phone number, Emergency contact number)

[1170] Output: Notification message sent to communications staff

[1171] Data processing: Based on acquired user information, a notification message is generated and sent to the communications staff via a dedicated notification system.

[1172] Specific operation: The server compiles user information into a notification message and immediately sends it to the communications staff.

[1173] Step 8:

[1174] The communications staff initiates a two-way call with the user's device.

[1175] Input: Received notification message

[1176] Output: Two-way call start

[1177] Specific actions: Communication staff will contact the user based on the information received and provide support via voice or video call as needed.

[1178] Step 9:

[1179] The user contacts the communications staff and receives support.

[1180] Input: Terminal's calling function

[1181] Output: Problem solved

[1182] Specific action: The user receives assistance from the communications staff and maintains the call until the problem is resolved.

[1183] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1184] This invention relates to a support system that incorporates a function to recognize user emotions and determine support priorities. The system consists of a user terminal, a server, an emotion engine, and support staff. The user terminal is responsible for inputting and transmitting user information and collecting emotion data. The server plays a central role in receiving and storing user information and processing and notifying support requests. The emotion engine analyzes user emotions and sets support priorities. Support staff receive notifications from the server and provide appropriate support to the user.

[1185] User information registration process

[1186] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[1187] Support request process

[1188] When a user presses the "Support button" on their smartphone app, the device sends a support request, including the user ID, to the server in JSON format. The server parses the received JSON data of the support request. Next, the server retrieves user information from the database based on the user ID and processes the support request appropriately.

[1189] Introducing an emotional engine

[1190] At this point, an emotion engine is added. The emotion engine analyzes the user's voice and text data to recognize the user's emotional state. The recognized emotional state may be, for example, "joy," "sadness," or "anger." The emotion engine sends this emotional state to the server. The server then sets the priority of the support request based on this emotional state. For example, if the user is showing strong anger or sadness, the support request will be given a higher priority.

[1191] Specific examples of emotional data

[1192] For example, before a user presses the support button, they might input a voice message into their device saying, "My smartphone is slow and it's frustrating." This voice message is analyzed by an emotion engine, which recognizes the user's emotional state as "anger." This emotion data is sent to a server, which then prioritizes the support request. As a result, support staff can contact the user immediately and provide assistance to resolve the problem.

[1193] System operation

[1194] Users input voice or text expressing their emotions into the app, and an emotion engine analyzes this and sends the appropriate emotional state to the server. The server processes support requests based on the emotional state and notifies support staff for a prompt response. This system allows users to accurately communicate their emotions, and enables support staff to take appropriate action quickly.

[1195] In summary, the present invention provides an environment in which elderly people can use smartphones with peace of mind by recognizing the user's emotions and responding quickly.

[1196] The following describes the processing flow.

[1197] User information registration process

[1198] Step 1:

[1199] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[1200] Step 2:

[1201] When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server.

[1202] Step 3:

[1203] The server receives the JSON data sent as an HTTP request.

[1204] Step 4:

[1205] The server parses the received JSON data and extracts user information. For example, it saves the following items in the database: "User ID," "Name," "Phone Number," and "Emergency Contact."

[1206] Step 5:

[1207] The server stores the extracted user information in a database using the user ID as the key.

[1208] Step 6:

[1209] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[1210] Step 7:

[1211] The terminal receives a response message from the server and displays "Registration complete" to the user.

[1212] Support request process

[1213] Step 1:

[1214] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[1215] Step 2:

[1216] The terminal converts the support request, including the user ID, into JSON format and sends it to the server. Simultaneously, the user's voice or text input is received and this data is also sent.

[1217] Emotional Engine Processing Process

[1218] Step 3:

[1219] The server receives the JSON data of the support request received from the terminal.

[1220] Step 4:

[1221] The server parses the received JSON data and extracts the user ID of the requester, voice data, and text data.

[1222] Step 5:

[1223] The emotion engine analyzes extracted audio and text data to recognize the user's emotional state (e.g., "joy," "sadness," "anger").

[1224] Step 6:

[1225] The emotion engine sends the recognized emotional state to the server.

[1226] Support request prioritization and notification process

[1227] Step 7:

[1228] The server prioritizes support requests based on the emotional state transmitted from the emotion engine. For example, if a user indicates "anger," the request will be given a higher priority.

[1229] Step 8:

[1230] The server retrieves the relevant user information from the database based on the support request with the assigned priority.

[1231] Step 9:

[1232] The server generates a message to notify the support team of the user information and support request.

[1233] Step 10:

[1234] The server sends the generated notification message to the support team and emergency contacts.

[1235] Support staff response process

[1236] Step 11:

[1237] Support staff receive notification messages from the server and contact the user.

[1238] Step 12:

[1239] Support staff will begin assisting users in resolving their problems.

[1240] Specific example

[1241] For example, an elderly person, Mr. A, experiences a problem with his smartphone and presses the support button. Mr. A then uses voice input to say, "My smartphone is so slow, it's frustrating." This voice data is sent from the device to the server, where the emotion engine recognizes it as "anger." Based on this information, the server sets a high priority for the support request and sends a notification to the support team. The support staff immediately contacts Mr. A and begins working to resolve the problem.

[1242] Thus, this system recognizes the user's emotions and enables the provision of prompt and appropriate support.

[1243] (Example 2)

[1244] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1245] Traditional support systems often provide uniform responses without considering the user's emotional state, resulting in a lack of prompt support, particularly for urgent issues. Furthermore, when a user is experiencing emotional distress, there's a lack of effective ways to communicate this information to support staff, raising concerns about a decline in the quality of support. This creates a challenge in providing a safe and secure environment for users, especially the elderly, who may have anxieties about digital devices.

[1246] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1247] In this invention, the server includes means for receiving user data from a user device, means for storing the user data, means for processing support requests based on the user data, and means including an emotion analysis engine for analyzing the user's emotional state and setting the priority of the support requests. This enables prompt and appropriate support in accordance with the user's emotional state, and allows for rapid response to particularly urgent problems. Furthermore, by analyzing the user's emotional information, support staff can take more appropriate action regarding the user's situation. As a result, it becomes possible to provide an environment in which elderly people and others can use digital devices with peace of mind.

[1248] A "user device" is an electronic device operated by a user, and is a device that inputs and retrieves user data.

[1249] "User data" refers to information entered by the user and used by the system, and includes user ID, name, phone number, emergency contact information, voice data, and text data.

[1250] A "request for assistance" refers to the act of a user requesting help or support, and is a request sent from the user's device to the server.

[1251] An "emotion analysis engine" is a software or hardware component that analyzes a user's voice data and text data to recognize the user's emotional state.

[1252] "Priority setting" is the process of determining the priority of responses based on the importance and urgency of the support request.

[1253] A "server" is a central processing unit that receives, stores, and processes user data and support requests, and plays a primary role in coordinating with other components.

[1254] This invention is a support system that adds a function to recognize user emotions and determine support priorities. The system consists of a user device, a server, an emotion analysis engine, and support staff.

[1255] User information registration process

[1256] First, the user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[1257] Support request process

[1258] When a user presses the "Support button" on their smartphone app, the device sends a support request containing the user ID to the server in JSON format. The server parses the received JSON data of the support request, retrieves user information from the database based on the user ID, and processes the support request appropriately.

[1259] Introducing an emotional engine

[1260] An emotion analysis engine is added to the system. Users input information indicating their emotions via voice or text into the terminal. For example, a voice message such as, "My smartphone is slow and I'm getting frustrated." The emotion analysis engine analyzes this input data. Voice data is converted to text using speech recognition technology, and then emotion analysis is performed. The emotion analysis engine sends the recognized emotional state to the server. The server sets the priority of the support request based on this emotional state. For example, if the user is indicating "anger," the support request will be given a high priority. The server notifies support staff of the priority to take prompt action, and the support staff, upon receiving the notification, provide appropriate support to the user.

[1261] Specific examples of system operation

[1262] For example, a user enters a voice message into the app saying, "My smartphone isn't working, please do something about it." The emotion analysis engine analyzes the voice data and recognizes the user's emotional state as "anger." The analysis results are sent to the server, which sets the request as a high priority. Support staff receive a notification from the server and contact the user immediately. The support staff then provides assistance to resolve the problem.

[1263] Examples of prompts for generative AI models

[1264] "Process user inquiries appropriately and analyze their emotional state. For example, if a user's voice message is 'My smartphone isn't working and I'm having trouble,' analyze what emotional state is recognized and send the results to the server."

[1265] "Recognize the emotional state of users from the text and voice data they input and prioritize support requests accordingly. For example, if a user says 'I'm in a lot of trouble,' analyze their emotional state and evaluate the priority."

[1266] This system allows users to accurately communicate their emotions, and enables support staff to respond quickly and appropriately. It also provides an environment where elderly people can use smartphones with peace of mind.

[1267] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1268] User information registration process

[1269] Step 1:

[1270] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information." The entered data is stored in the device's temporary memory.

[1271] Step 2:

[1272] When a user presses the "Register" button, the terminal converts this input information into JSON format. This conversion process organizes the entered string and numerical data into a structured format. The output is JSON data containing user information.

[1273] Step 3:

[1274] The terminal sends the generated JSON data to the server via an HTTP request. This is done through network communication. The input is the generated JSON data, and the output is the completion of sending the request to the server.

[1275] Step 4:

[1276] The server receives the JSON data and uses a JSON parser to analyze (deserialize) the data. This process extracts specific information such as user ID, name, phone number, and emergency contact information. The input is the received JSON data, and the output is the extracted user information.

[1277] Step 5:

[1278] The server saves the analyzed user information to the database. This saving process is performed by executing an insert query to the database. The input is the analyzed user information, and the output is the completion of saving to the database.

[1279] Step 6:

[1280] The server generates a response message indicating that the user information was successfully saved and sends it to the terminal. The input is the success status of the save, and the output is the completion of sending the response message.

[1281] Step 7:

[1282] The terminal receives a response message from the server and displays "Registration complete" on the screen. The input is the response message from the server, and the output is the display of the registration completion message.

[1283] Support request process

[1284] Step 1:

[1285] The user presses the "Support button" on the smartphone app. This action triggers the request creation process to begin.

[1286] Step 2:

[1287] The terminal creates a support request in JSON format, including the user ID. This data contains information indicating the user's current status. Inputs are the user ID and the press of the support button, and output is the created support request JSON data.

[1288] Step 3:

[1289] The terminal sends the generated support request to the server. HTTP requests are used via network communication. The input is the JSON data of the created support request, and the output is the completion of sending the request to the server.

[1290] Step 4:

[1291] The server parses the JSON data of the received support request. It retrieves user information from the database based on the user ID. The input is the JSON data of the support request, and the output is the retrieved user information.

[1292] Step 5:

[1293] The server processes support requests appropriately based on the retrieved user information and request details. The input is the retrieved user information and request details, and the output is the processed support request.

[1294] Introducing an emotional engine

[1295] Step 1:

[1296] The user inputs emotional information into the device via voice or text. For example, a voice message such as, "My smartphone is slow and it's frustrating." The input is the user's voice or text data, and the output is the completion of data input to the device.

[1297] Step 2:

[1298] The emotion analysis engine analyzes this input data. For example, if it's audio data, it uses speech recognition technology to convert it to text, and then performs emotion analysis on that text. The input is audio or text data, and the output is the analyzed emotional state.

[1299] Step 3:

[1300] The emotion analysis engine sends the recognized emotional state to the server. This data includes emotional states such as "joy," "sadness," and "anger." The input is the analyzed emotional state, and the output is the completion of sending the emotional state to the server.

[1301] Step 4:

[1302] The server prioritizes support requests based on the user's emotional state. For example, if a user is expressing "anger," the server will prioritize the support request higher. The input is the emotional state, and the output is the set priority.

[1303] Step 5:

[1304] The server sends notifications to support staff based on the configured priority to encourage a quick response. The input is the configured priority, and the output is the completion of sending the notification to support staff.

[1305] Step 6:

[1306] Support staff receive notifications and provide appropriate support to users. Input is a notification from the server, and output is the commencement of support for the user.

[1307] (Application Example 2)

[1308] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1309] Traditional security service systems have a problem in that they cannot respond quickly and appropriately to high-priority situations because they respond uniformly without considering the user's emotions. In particular, when a user is experiencing strong emotions such as fear, immediate action is required, but delays in making such decisions increase security risks. Furthermore, there was a lack of means to properly analyze and prioritize emotions expressed by users through voice or text. Therefore, it is necessary to provide more advanced support and improve the appropriateness and speed of emergency responses.

[1310] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving user information from a user terminal, means for storing the user information, means for processing support requests based on the user information, means for analyzing voice data or text data to recognize the emotional state, means for setting the priority of support requests based on the emotional state, and means for determining a response according to the priority and sending a notification. This enables emergency responses that take into account the user's emotions, reduces security risks, and enables faster and more appropriate responses.

[1311] A "user terminal" is an electronic device used by a user to input and transmit information.

[1312] "User information" refers to data that includes identifying information about a user.

[1313] A "support request" is data that indicates a user's request for assistance.

[1314] "Voice data" refers to data that includes voice information emitted by the user.

[1315] "Text data" refers to data that includes character information entered by the user.

[1316] "Emotional state" refers to the state of a user's emotions as analyzed from voice and text data.

[1317] "Priority" is an indicator that shows the importance of addressing a support request based on emotional state.

[1318] A "notification" is a message that informs you that a response has been taken in response to your support request.

[1319] A "security service system" is a system that handles requests for assistance and provides responses to ensure the safety of users.

[1320] This invention is a security service system that prioritizes security requests by considering user emotions and responds quickly. The system consists of a user terminal, a server, an emotion engine, and security staff.

[1321] System Configuration

[1322] User terminal

[1323] Users use electronic devices such as smartphones to input data via voice or text. For example, a smartphone app using React Native could be considered. This app receives user input and sends the data to a server.

[1324] server

[1325] The server is located in the cloud, for example, built on AWS EC2 using Node.js and the Express framework. The server receives and processes user information, support requests, and sentiment data. Specifically, it sends the aforementioned data to the Google Cloud Natural Language API for sentiment analysis.

[1326] Emotional Engine

[1327] The emotion engine uses the Google Cloud Natural Language API to analyze the user's voice or text data to determine their emotional state (e.g., "fear," "anger," etc.). The results of this analysis are then sent back to the server.

[1328] Security staff

[1329] The server prioritizes support requests based on their emotional state and determines how to respond. Based on priority, it sends a notification to the most appropriate security staff. Upon receiving the notification, the security staff quickly travels to the site and takes the necessary action.

[1330] Explanation of the program's processing details

[1331] The user opens a smartphone app and enters a message via voice or text saying, "I feel something suspicious is happening around my house and it's scary." The app converts this data into JSON format and sends it to a server. The server sends the voice or text data to the Google Cloud Natural Language API, where the emotion engine analyzes it as "fear." Based on this analysis, the server sets the case as high priority and determines that immediate action is required. A notification is then sent to the nearest security staff, who rush to the scene.

[1332] Example of a prompt

[1333] "When a user voice-inputs 'I feel scared because I sense suspicious activity around my house,' use the Google Cloud Natural Language API to analyze the emotion as 'fear.' Based on this result, design a program that sets the server to high priority and immediately notifies security staff."

[1334] This invention enables the reduction of security risks and the realization of faster and more appropriate responses by appropriately analyzing user emotions and performing highly accurate priority evaluations. This allows for the creation of security services that offer high reliability and safety.

[1335] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1336] Step 1:

[1337] The user launches the smartphone app and creates a security request. Specifically, the user inputs a voice or text message into the app saying, "I feel scared because I've noticed suspicious activity around my house." This input data is converted into JSON format. After receiving the input (voice or text) into the app, data converted into JSON format is generated.

[1338] Step 2:

[1339] The device sends the converted JSON data to the server. Specifically, the data is sent as an HTTP request from the app to the server. At this time, the input data is voice or text information including the user's emotions, and the output data is the JSON data sent to the server.

[1340] Step 3:

[1341] The server parses the received JSON data and sends it to the sentiment engine. Specifically, the server makes an API request to the Google Cloud Natural Language API to send data. At this time, the input data is the JSON data received by the server, and the output data is the data sent to the sentiment engine.

[1342] Step 4:

[1343] The emotion engine analyzes the user's emotional state from voice or text data. Using the Google Cloud Natural Language API, emotions such as "fear" are analyzed from the input voice or text. Here, the input is the data sent to the emotion engine, and the output is the analysis result.

[1344] Step 5:

[1345] The server receives the analysis results returned from the emotion engine and sets the priority of the support request. Specifically, a higher priority is set based on the analyzed emotional state of "fear." The input here is the emotion analysis result, and the output is the set priority.

[1346] Step 6:

[1347] The server sends notifications to the nearest security staff based on priority. Specifically, if the priority is high, the server sends an urgent notification to the security staff via the API. The input here is the configured priority, and the output is the notification sent to the security staff.

[1348] Step 7:

[1349] Security staff receive the notification and rush to the scene to respond. Security staff who receive the notification quickly arrive at the scene. In this step, the input is the notification to security staff, and the output is the on-site response.

[1350] In this way, each step works in conjunction, enabling swift and appropriate security responses that take user emotions into consideration.

[1351] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1352] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1353] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1354] [Fourth Embodiment]

[1355] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1356] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1357] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1358] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1359] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1360] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1361] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1362] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1363] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1364] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1365] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1366] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1367] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1368] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[1369] System Overview

[1370] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and to submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[1371] User information registration process

[1372] 1. User: Information Input

[1373] The user launches a dedicated app on their smartphone and accesses the registration screen.

[1374] On the registration screen, you will enter information such as your "User ID," "Name," "Phone Number," and "Emergency Contact Number."

[1375] 2. Terminal: Information transmission

[1376] When the user presses the "Register" button, the device sends this information to the server in JSON format.

[1377] 3. Server: Data reception and storage

[1378] The server parses the received JSON data and saves the user information to the database.

[1379] The saved information is associated using the user ID as the key.

[1380] 4. Server: Completion notification

[1381] The server sends a response back to the terminal indicating that registration was successful, and displays "Registration complete" to the user.

[1382] Support request process

[1383] 1. User: Press the support button.

[1384] If a user needs support, they press the "Support button" in the app.

[1385] 2. Terminal: Sending a request

[1386] The device sends a support request, including the user ID, to the server in JSON format.

[1387] 3. Server: Request received

[1388] The server analyzes the support requests received from the terminal.

[1389] 4. Server: User information verification and notification

[1390] The server retrieves user information from the database based on the user ID and sends a notification to the support staff.

[1391] 5. Support staff: Start of support

[1392] Support staff will contact the user based on notifications received from the server and begin providing support to resolve the issue.

[1393] Specific example

[1394] For example, an elderly person, Mr. A, purchases a new smartphone and, upon first use, launches a dedicated app and enters user information. Mr. A enters information such as "User ID: A123", "Name: Taro Yamada", "Phone number: 090-1234-5678", and "Emergency contact: 090-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in its database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to Mr. A.

[1395] Afterward, if Person A has trouble using their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies Person A's information. The server then notifies the support staff, who call Person A to help resolve the problem.

[1396] In this way, this system provides an environment in which elderly people can use smartphones with peace of mind.

[1397] The following describes the processing flow.

[1398] User information registration process

[1399] Step 1:

[1400] The user launches a dedicated app on their smartphone and accesses the registration screen. The user enters information such as "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[1401] Step 2:

[1402] When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server.

[1403] Step 3:

[1404] The server receives the JSON data sent as an HTTP request.

[1405] Step 4:

[1406] The server parses the received JSON data and extracts user information. For example, it associates data using the user ID as a key.

[1407] Step 5:

[1408] The server stores the extracted user information in a database. Each user's information is uniquely managed using the user ID as the key.

[1409] Step 6:

[1410] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[1411] Step 7:

[1412] The terminal receives a response message from the server and displays "Registration complete."

[1413] Support request process

[1414] Step 1:

[1415] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[1416] Step 2:

[1417] The terminal converts the support request, including the user ID, into JSON format and sends it to the server.

[1418] Step 3:

[1419] The server receives the JSON data of the support request, which was sent as an HTTP request.

[1420] Step 4:

[1421] The server parses the received JSON data and extracts the user ID that sent the request.

[1422] Step 5:

[1423] Based on the extracted user ID, the server retrieves the corresponding user information (e.g., name, phone number, emergency contact) from the database.

[1424] Step 6:

[1425] The server generates notification messages for the support team and emergency contacts based on the acquired user information.

[1426] Step 7:

[1427] The server sends the generated notification message to the support team and emergency contacts.

[1428] Step 8:

[1429] The support team receives notification messages from the server, contacts the user, and begins providing support to resolve the issue.

[1430] Step 9:

[1431] The server generates a response message indicating that the support request was processed successfully and sends it back to the terminal.

[1432] Step 10:

[1433] The terminal receives a response message from the server and displays to the user, "Support request submitted."

[1434] (Example 1)

[1435] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1436] This invention relates to a system that supports elderly people in using smartphones with peace of mind. This user group often struggles with smartphone operation and frequently requires immediate support. However, conventional systems have complex procedures for registering user information and processing support requests, making it difficult to respond quickly, especially in emergencies. Therefore, there is a need for a system that allows for easy registration of user information and rapid notification to support staff.

[1437] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1438] In this invention, the server includes means for receiving user information from a user terminal, means for storing user information, means for processing support requests based on user information, means for the processing means to send notifications to emergency contacts, means for displaying pop-up messages based on user information, means for processing HTTPS requests and responses, and means for notifying support staff of support requests. This makes it possible for users to easily register information, have support requests processed quickly, and receive appropriate support.

[1439] A "user terminal" is a device used by a user to input information and submit support requests.

[1440] "User information" refers to personal information related to the user, such as user ID, name, phone number, and emergency contact information.

[1441] A "server" is a core computing system that stores information received from user terminals and processes support requests.

[1442] "Support staff" are individuals who receive notifications from the server and provide appropriate support to users.

[1443] A "support request" is a request that a user submits when they need support.

[1444] "Means of sending notifications to emergency contacts" refers to a means of automatically sending notifications to designated contacts in the event of an emergency, based on user information.

[1445] A "pop-up message" is a notification or message window that appears on the screen of a user's device.

[1446] "Means for processing HTTPS requests and responses" refers to communication methods for securely exchanging data between a user's terminal and a server.

[1447] "Means of sending notifications" refers to the means by which the server notifies support staff of user information and support requests.

[1448] This invention is a support system designed to enable elderly people to use smartphones with peace of mind. The specific method for implementing this system is described below.

[1449] System Overview

[1450] This system consists of user terminals, a server, and support staff. User terminals are devices used to input and transmit user information and submit support requests. The server plays a central role in receiving, storing, and processing user information and support requests. Support staff receive notifications from the server and provide appropriate support to users.

[1451] Hardware and software to be used

[1452] User terminals are either iOS or Android devices. These devices have a dedicated app installed, allowing users to input user information and submit support requests. The server runs on a Linux-based system, and the program is implemented using Python. MySQL is used as the database.

[1453] 1. Data reception and storage

[1454] Users enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact" using a dedicated smartphone app.

[1455] The terminal sends this information to the server in JSON format. The server receives the HTTPS request, parses the JSON data, and saves the user information to a MySQL database.

[1456] 2. Processing support requests

[1457] When a user needs support, they press the "Support button" in the app. This causes the device to send a support request, including the user ID, to the server.

[1458] The server parses the request and retrieves user information from the database based on the user ID. It then sends a support request notification to the support staff along with the user information.

[1459] Upon receiving this notification, support staff will begin providing specific assistance to the user via phone or messaging app.

[1460] Specific example

[1461] For example, when an elderly user purchases a new smartphone, they launch a dedicated app and enter their user information the first time they use it. The user enters information such as "User ID: user123", "Name: Taro Tanaka", "Phone number: 080-1234-5678", and "Emergency contact: 080-8765-4321", and presses the "Register" button. The device sends this information to the server, which receives it and stores it in a database. The server sends a notification of successful registration back to the device, and the device displays "Registration complete" to the user.

[1462] Subsequently, if the user encounters difficulties operating their smartphone, they press the "Support button" in the app. The device sends a support request to the server, which receives it and verifies the user's information. The server then notifies the support staff, who contact the user to provide assistance in resolving the problem.

[1463] Example of a prompt

[1464] The following are examples of prompt statements to input into a generative AI model:

[1465] Please describe a smartphone support system for the elderly. This system includes the steps from user registration via smartphone to requesting support. Please clearly explain how users input information and receive support. Also, please specify the names of the hardware and software used.

[1466] This system provides an environment where elderly people can use smartphones with peace of mind.

[1467] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1468] Step 1:

[1469] User: Information Entry

[1470] The user launches a dedicated app on their smartphone and accesses the registration screen. There, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[1471] Input: User ID, Name, Phone Number, Emergency Contact Information

[1472] Output: Entered user information

[1473] Specific action: The user enters information into a text field and presses the register button.

[1474] Step 2:

[1475] Terminal: Information transmission

[1476] When a user presses the "Register" button, the device converts the entered information into JSON format and sends it to the server as an HTTPS POST request.

[1477] Input: User information in JSON format

[1478] Output: HTTPS request to the server

[1479] Specific operation: The app serializes the input data into JSON format and sends a POST request to the URL endpoint.

[1480] Step 3:

[1481] Server: Data reception and analysis

[1482] The server receives the HTTPS request, parses the JSON data, and extracts user information.

[1483] Input: User information in JSON format sent via HTTPS request

[1484] Output: Analyzed user information

[1485] Specific operation: The server uses the json.loads function to parse the received JSON data.

[1486] Step 4:

[1487] Server: Database storage

[1488] The server stores the analyzed user information in a MySQL database. The stored information includes the user's name, phone number, and emergency contact information, with the user ID as the key.

[1489] Input: Analyzed user information

[1490] Output: User information stored in the database

[1491] Specific operation: The server generates an SQL INSERT statement and executes it against the MySQL database.

[1492] Step 5:

[1493] Server: Completion notification

[1494] The server generates a response message indicating that registration was successful and sends it back to the terminal as an HTTPS response.

[1495] Input: Success status of the registration process

[1496] Output: Notification message for successful registration

[1497] Specific operation: The server generates a JSON response and sends an HTTP response.

[1498] Step 6:

[1499] Device: Notification display

[1500] The device displays a notification message confirming successful registration as a pop-up to the user.

[1501] Input: Registration success notification message from the server

[1502] Output: Pop-up message to the user

[1503] Specific operation: The app parses the message it receives and displays a pop-up in the UI.

[1504] Step 7:

[1505] User: Press the support button

[1506] If a user needs support, they press the "Support button" in the app.

[1507] Input: Press the support button.

[1508] Output: Submitting a support request

[1509] Specific action: The user taps the support button within the app.

[1510] Step 8:

[1511] Terminal: Sending a request

[1512] The device sends a support request, including the user ID, to the server in JSON format.

[1513] Input: Support request including User ID

[1514] Output: HTTPS request to the server

[1515] Specific actions: The app retrieves the user ID, serializes the support request in JSON format, and sends a POST request.

[1516] Step 9:

[1517] Server: Request reception and analysis

[1518] The server receives the support request and parses the JSON data.

[1519] Input: Support request for JSON format as HTTPS request

[1520] Output: Analyzed support request

[1521] Specific operation: The server uses the json.loads function to parse the received JSON data.

[1522] Step 10:

[1523] Server: User information retrieval

[1524] The server retrieves user information from the database based on the user ID.

[1525] Input: User ID

[1526] Output: User information retrieved from the database

[1527] Specific operation: The server generates an SQL SELECT statement and executes it against the database.

[1528] Step 11:

[1529] Server: Notification to support staff

[1530] The server will notify the support staff via email of the retrieved user information and support request.

[1531] Input: User information, support request

[1532] Output: Notification email to support staff

[1533] Specific operation: The server generates an email using the SMTP library and sends it to the support staff.

[1534] Step 12:

[1535] Support staff: Started responding

[1536] Support staff receive notifications and contact users via phone or messaging apps to provide support.

[1537] Input: Support Request Notification

[1538] Output: Providing support to users

[1539] Specific actions: Support staff will check the email notification and use phone or messaging apps to contact the user.

[1540] (Application Example 1)

[1541] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1542] There is a need to quickly and efficiently resolve the difficulties elderly people face when using online shopping sites. In particular, a system is needed that can provide timely and appropriate support for problems during the purchase process and operation. However, with conventional systems, it is difficult for elderly people to receive prompt assistance when they get lost, and an environment for using online shopping sites safely and with peace of mind is not established.

[1543] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1544] In this invention, the server includes means for receiving user data from a user device, means for storing user data, means for processing support requests based on the user data, means for notifying communication staff of support requests in real time, and means for communication staff to provide two-way communication to the user device. This makes it possible for elderly people to quickly receive real-time support from communication staff if they encounter difficulties using an online shopping site.

[1545] A "user device" is an electronic device that allows a user to input and transmit information, and to send out support requests.

[1546] "User data" refers to a collection of data that includes user identification information, contact information, and other related information.

[1547] A "support request" is request information sent from a user's device to the server when a user requests assistance.

[1548] A "communication staff member" is a person responsible for providing real-time support to users based on their support requests.

[1549] A "server" is a central device that receives and stores information from user devices, processes support requests, and sends notifications to communication staff.

[1550] "Real-time notification" is a process in which communication staff are promptly notified immediately after a support request is made.

[1551] "Two-way communication" refers to a communication method in which both the communication staff and the user device can communicate via voice and video.

[1552] "Storage means" refers to a method or device for storing user data received from users in a database on a server.

[1553] "Processing means" refers to the means of determining an appropriate response based on user data and support requests, and then notifying the user.

[1554] System Overview

[1555] This system is a support system designed to allow elderly people to use online shopping sites with peace of mind. The system consists of user devices, a server, and communication staff. Their respective roles and functions are as follows:

[1556] User devices

[1557] A user device is an electronic device used to input and transmit user information and to send support requests. Specific examples include smartphones and tablets. A dedicated application is installed on the user device, and user data registration and support request submission are performed through this application.

[1558] server

[1559] The server is responsible for processing and storing information received from user devices and sending notifications to communication staff as needed. The main functions of the server are described below.

[1560] Information reception and storage:

[1561] The server receives user data in JSON format and stores it in the database. This stored user data is then used to process subsequent support requests.

[1562] Processing support requests:

[1563] The server receives and analyzes support requests from user devices. Based on the analysis results, it sends notifications to the appropriate communications staff.

[1564] Real-time notifications:

[1565] The server notifies communication staff in real time immediately after a support request is received, enabling rapid assistance.

[1566] Communications staff

[1567] Upon receiving notifications from the server, communication staff will quickly contact the user. Specifically, they will use two-way communication to communicate with the user's device in real time and assist in resolving the issue.

[1568] Hardware and software

[1569] Hardware:

[1570] Smartphones and tablets will be used as user devices. High-performance server machines (e.g., Dell EMC PowerEdge) are recommended for the server.

[1571] software:

[1572] The server-side program will use Flask (a Python framework) and SQLite (a database management system). The user-side application will be designed for iOS or Android.

[1573] Examples of prompt statements

[1574] The following prompt messages illustrate a scenario where a user purchases a new smartphone from an online shopping site and encounters difficulties in using it.

[1575] Example of a prompt:

[1576] "If you encounter any problems while purchasing your smartphone, please press the support button in the app. Our expert staff will provide support in real time."

[1577] Specific example

[1578] For example, suppose an elderly person, Mr. B, purchases a new smartphone from an online shopping site. When Mr. B uses it for the first time, he launches a dedicated app and enters and registers his user information. Once Mr. B sends his user data, the server receives it and stores it in a database. Later, if Mr. B encounters any problems during the purchase process, he presses the "support button." A support request is sent to the server, which receives and analyzes it and notifies the communication staff in real time. The communication staff immediately contacts Mr. B and provides support to resolve the problem through two-way communication. This system allows elderly people to use online shopping sites with peace of mind.

[1579] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1580] Step 1:

[1581] The user launches a dedicated app and enters their user data.

[1582] Input: User ID, Name, Phone Number, Emergency Contact Number

[1583] Specific operation: The user accesses the app screen and enters the required information for each field. Once the input is complete, they press the "Register" button.

[1584] Step 2:

[1585] The device sends user data to the server.

[1586] Input: User data entered by the user

[1587] Output: Data sent to the server in JSON format

[1588] Data processing: The terminal converts the input data into JSON format and sends it.

[1589] Specific operation: The device collects user data and sends it to the server by calling the API endpoint.

[1590] Step 3:

[1591] The server receives user data and stores it in the database.

[1592] Input: User data sent to the server in JSON format

[1593] Output: User data stored in the database

[1594] Data processing: Parse the received JSON data and store it in the SQLite database.

[1595] Specific operation: The server analyzes the data and saves it to the corresponding field.

[1596] Step 4:

[1597] A user submits a support request.

[1598] Input: User ID

[1599] Specific action: The user presses the "Support button" within the app.

[1600] Step 5:

[1601] The device sends a support request to the server.

[1602] Input: Support request including User ID

[1603] Output: Request data sent to the server in JSON format

[1604] Data processing: The terminal converts the user ID into JSON format and sends it.

[1605] Specific operation: The device collects support requests and calls the API endpoint to send them to the server.

[1606] Step 6:

[1607] The server receives and analyzes the support request.

[1608] Input: Support request sent to the server in JSON format

[1609] Output: User information (registered user data)

[1610] Data processing: Analyzes the received request and retrieves the corresponding user data from the database based on the user ID.

[1611] Specific operation: The server parses the request and retrieves user information from the database.

[1612] Step 7:

[1613] The server notifies the communications staff of the support request in real time.

[1614] Input: User information (User ID, Name, Phone number, Emergency contact number)

[1615] Output: Notification message sent to communications staff

[1616] Data processing: Based on acquired user information, a notification message is generated and sent to the communications staff via a dedicated notification system.

[1617] Specific operation: The server compiles user information into a notification message and immediately sends it to the communications staff.

[1618] Step 8:

[1619] The communications staff initiates a two-way call with the user's device.

[1620] Input: Received notification message

[1621] Output: Two-way call start

[1622] Specific actions: Communication staff will contact the user based on the information received and provide support via voice or video call as needed.

[1623] Step 9:

[1624] The user contacts the communications staff and receives support.

[1625] Input: Terminal's calling function

[1626] Output: Problem solved

[1627] Specific action: The user receives assistance from the communications staff and maintains the call until the problem is resolved.

[1628] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1629] This invention relates to a support system that incorporates a function to recognize user emotions and determine support priorities. The system consists of a user terminal, a server, an emotion engine, and support staff. The user terminal is responsible for inputting and transmitting user information and collecting emotion data. The server plays a central role in receiving and storing user information and processing and notifying support requests. The emotion engine analyzes user emotions and sets support priorities. Support staff receive notifications from the server and provide appropriate support to the user.

[1630] User information registration process

[1631] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[1632] Support request process

[1633] When a user presses the "Support button" on their smartphone app, the device sends a support request, including the user ID, to the server in JSON format. The server parses the received JSON data of the support request. Next, the server retrieves user information from the database based on the user ID and processes the support request appropriately.

[1634] Introducing an emotional engine

[1635] At this point, an emotion engine is added. The emotion engine analyzes the user's voice and text data to recognize the user's emotional state. The recognized emotional state may be, for example, "joy," "sadness," or "anger." The emotion engine sends this emotional state to the server. The server then sets the priority of the support request based on this emotional state. For example, if the user is showing strong anger or sadness, the support request will be given a higher priority.

[1636] Specific examples of emotional data

[1637] For example, before a user presses the support button, they might input a voice message into their device saying, "My smartphone is slow and it's frustrating." This voice message is analyzed by an emotion engine, which recognizes the user's emotional state as "anger." This emotion data is sent to a server, which then prioritizes the support request. As a result, support staff can contact the user immediately and provide assistance to resolve the problem.

[1638] System operation

[1639] Users input voice or text expressing their emotions into the app, and an emotion engine analyzes this and sends the appropriate emotional state to the server. The server processes support requests based on the emotional state and notifies support staff for a prompt response. This system allows users to accurately communicate their emotions, and enables support staff to take appropriate action quickly.

[1640] In summary, the present invention provides an environment in which elderly people can use smartphones with peace of mind by recognizing the user's emotions and responding quickly.

[1641] The following describes the processing flow.

[1642] User information registration process

[1643] Step 1:

[1644] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information."

[1645] Step 2:

[1646] When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server.

[1647] Step 3:

[1648] The server receives the JSON data sent as an HTTP request.

[1649] Step 4:

[1650] The server parses the received JSON data and extracts user information. For example, it saves the following items in the database: "User ID," "Name," "Phone Number," and "Emergency Contact."

[1651] Step 5:

[1652] The server stores the extracted user information in a database using the user ID as the key.

[1653] Step 6:

[1654] The server generates a response message indicating that registration was successful and sends it back to the terminal.

[1655] Step 7:

[1656] The terminal receives a response message from the server and displays "Registration complete" to the user.

[1657] Support request process

[1658] Step 1:

[1659] When a user presses the "Support button" on the smartphone app, it sends a message indicating that they need support.

[1660] Step 2:

[1661] The terminal converts the support request, including the user ID, into JSON format and sends it to the server. Simultaneously, the user's voice or text input is received and this data is also sent.

[1662] Emotional Engine Processing Process

[1663] Step 3:

[1664] The server receives the JSON data of the support request received from the terminal.

[1665] Step 4:

[1666] The server parses the received JSON data and extracts the user ID of the requester, voice data, and text data.

[1667] Step 5:

[1668] The emotion engine analyzes extracted audio and text data to recognize the user's emotional state (e.g., "joy," "sadness," "anger").

[1669] Step 6:

[1670] The emotion engine sends the recognized emotional state to the server.

[1671] Support request prioritization and notification process

[1672] Step 7:

[1673] The server prioritizes support requests based on the emotional state transmitted from the emotion engine. For example, if a user indicates "anger," the request will be given a higher priority.

[1674] Step 8:

[1675] The server retrieves the relevant user information from the database based on the support request with the assigned priority.

[1676] Step 9:

[1677] The server generates a message to notify the support team of the user information and support request.

[1678] Step 10:

[1679] The server sends the generated notification message to the support team and emergency contacts.

[1680] Support staff response process

[1681] Step 11:

[1682] Support staff receive notification messages from the server and contact the user.

[1683] Step 12:

[1684] Support staff will begin assisting users in resolving their problems.

[1685] Specific example

[1686] For example, an elderly person, Mr. A, experiences a problem with his smartphone and presses the support button. Mr. A then uses voice input to say, "My smartphone is so slow, it's frustrating." This voice data is sent from the device to the server, where the emotion engine recognizes it as "anger." Based on this information, the server sets a high priority for the support request and sends a notification to the support team. The support staff immediately contacts Mr. A and begins working to resolve the problem.

[1687] Thus, this system recognizes the user's emotions and enables the provision of prompt and appropriate support.

[1688] (Example 2)

[1689] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1690] Traditional support systems often provide uniform responses without considering the user's emotional state, resulting in a lack of prompt support, particularly for urgent issues. Furthermore, when a user is experiencing emotional distress, there's a lack of effective ways to communicate this information to support staff, raising concerns about a decline in the quality of support. This creates a challenge in providing a safe and secure environment for users, especially the elderly, who may have anxieties about digital devices.

[1691] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1692] In this invention, the server includes means for receiving user data from a user device, means for storing the user data, means for processing support requests based on the user data, and means including an emotion analysis engine for analyzing the user's emotional state and setting the priority of the support requests. This enables prompt and appropriate support in accordance with the user's emotional state, and allows for rapid response to particularly urgent problems. Furthermore, by analyzing the user's emotional information, support staff can take more appropriate action regarding the user's situation. As a result, it becomes possible to provide an environment in which elderly people and others can use digital devices with peace of mind.

[1693] A "user device" is an electronic device operated by a user, and is a device that inputs and retrieves user data.

[1694] "User data" refers to information entered by the user and used by the system, and includes user ID, name, phone number, emergency contact information, voice data, and text data.

[1695] A "request for assistance" refers to the act of a user requesting help or support, and is a request sent from the user's device to the server.

[1696] An "emotion analysis engine" is a software or hardware component that analyzes a user's voice data and text data to recognize the user's emotional state.

[1697] "Priority setting" is the process of determining the priority of responses based on the importance and urgency of the support request.

[1698] A "server" is a central processing unit that receives, stores, and processes user data and support requests, and plays a primary role in coordinating with other components.

[1699] This invention is a support system that adds a function to recognize user emotions and determine support priorities. The system consists of a user device, a server, an emotion analysis engine, and support staff.

[1700] User information registration process

[1701] First, the user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as "User ID," "Name," "Phone Number," and "Emergency Contact." When the user presses the "Register" button, the device converts this information into JSON format and sends it to the server. The server parses the received JSON data and saves the user information to its database. The server returns a response message to the device indicating successful registration, and the device displays "Registration Complete."

[1702] Support request process

[1703] When a user presses the "Support button" on their smartphone app, the device sends a support request containing the user ID to the server in JSON format. The server parses the received JSON data of the support request, retrieves user information from the database based on the user ID, and processes the support request appropriately.

[1704] Introducing an emotional engine

[1705] An emotion analysis engine is added to the system. Users input information indicating their emotions via voice or text into the terminal. For example, a voice message such as, "My smartphone is slow and I'm getting frustrated." The emotion analysis engine analyzes this input data. Voice data is converted to text using speech recognition technology, and then emotion analysis is performed. The emotion analysis engine sends the recognized emotional state to the server. The server sets the priority of the support request based on this emotional state. For example, if the user is indicating "anger," the support request will be given a high priority. The server notifies support staff of the priority to take prompt action, and the support staff, upon receiving the notification, provide appropriate support to the user.

[1706] Specific examples of system operation

[1707] For example, a user enters a voice message into the app saying, "My smartphone isn't working, please do something about it." The emotion analysis engine analyzes the voice data and recognizes the user's emotional state as "anger." The analysis results are sent to the server, which sets the request as a high priority. Support staff receive a notification from the server and contact the user immediately. The support staff then provides assistance to resolve the problem.

[1708] Examples of prompts for generative AI models

[1709] "Process user inquiries appropriately and analyze their emotional state. For example, if a user's voice message is 'My smartphone isn't working and I'm having trouble,' analyze what emotional state is recognized and send the results to the server."

[1710] "Recognize the emotional state of users from the text and voice data they input and prioritize support requests accordingly. For example, if a user says 'I'm in a lot of trouble,' analyze their emotional state and evaluate the priority."

[1711] This system allows users to accurately communicate their emotions, and enables support staff to respond quickly and appropriately. It also provides an environment where elderly people can use smartphones with peace of mind.

[1712] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1713] User information registration process

[1714] Step 1:

[1715] The user launches a dedicated app on their smartphone and accesses the registration screen. Here, they enter information such as their "User ID," "Name," "Phone Number," and "Emergency Contact Information." The entered data is stored in the device's temporary memory.

[1716] Step 2:

[1717] When a user presses the "Register" button, the terminal converts this input information into JSON format. This conversion process organizes the entered string and numerical data into a structured format. The output is JSON data containing user information.

[1718] Step 3:

[1719] The terminal sends the generated JSON data to the server via an HTTP request. This is done through network communication. The input is the generated JSON data, and the output is the completion of sending the request to the server.

[1720] Step 4:

[1721] The server receives the JSON data and uses a JSON parser to analyze (deserialize) the data. This process extracts specific information such as user ID, name, phone number, and emergency contact information. The input is the received JSON data, and the output is the extracted user information.

[1722] Step 5:

[1723] The server saves the analyzed user information to the database. This saving process is performed by executing an insert query to the database. The input is the analyzed user information, and the output is the completion of saving to the database.

[1724] Step 6:

[1725] The server generates a response message indicating that the user information was successfully saved and sends it to the terminal. The input is the success status of the save, and the output is the completion of sending the response message.

[1726] Step 7:

[1727] The terminal receives a response message from the server and displays "Registration complete" on the screen. The input is the response message from the server, and the output is the display of the registration completion message.

[1728] Support request process

[1729] Step 1:

[1730] The user presses the "Support button" on the smartphone app. This action triggers the request creation process to begin.

[1731] Step 2:

[1732] The terminal creates a support request in JSON format, including the user ID. This data contains information indicating the user's current status. Inputs are the user ID and the press of the support button, and output is the created support request JSON data.

[1733] Step 3:

[1734] The terminal sends the generated support request to the server. HTTP requests are used via network communication. The input is the JSON data of the created support request, and the output is the completion of sending the request to the server.

[1735] Step 4:

[1736] The server parses the JSON data of the received support request. It retrieves user information from the database based on the user ID. The input is the JSON data of the support request, and the output is the retrieved user information.

[1737] Step 5:

[1738] The server processes support requests appropriately based on the retrieved user information and request details. The input is the retrieved user information and request details, and the output is the processed support request.

[1739] Introducing an emotional engine

[1740] Step 1:

[1741] The user inputs emotional information into the device via voice or text. For example, a voice message such as, "My smartphone is slow and it's frustrating." The input is the user's voice or text data, and the output is the completion of data input to the device.

[1742] Step 2:

[1743] The emotion analysis engine analyzes this input data. For example, if it's audio data, it uses speech recognition technology to convert it to text, and then performs emotion analysis on that text. The input is audio or text data, and the output is the analyzed emotional state.

[1744] Step 3:

[1745] The emotion analysis engine sends the recognized emotional state to the server. This data includes emotional states such as "joy," "sadness," and "anger." The input is the analyzed emotional state, and the output is the completion of sending the emotional state to the server.

[1746] Step 4:

[1747] The server prioritizes support requests based on the user's emotional state. For example, if a user is expressing "anger," the server will prioritize the support request higher. The input is the emotional state, and the output is the set priority.

[1748] Step 5:

[1749] The server sends notifications to support staff based on the configured priority to encourage a quick response. The input is the configured priority, and the output is the completion of sending the notification to support staff.

[1750] Step 6:

[1751] Support staff receive notifications and provide appropriate support to users. Input is a notification from the server, and output is the commencement of support for the user.

[1752] (Application Example 2)

[1753] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1754] Traditional security service systems have a problem in that they cannot respond quickly and appropriately to high-priority situations because they respond uniformly without considering the user's emotions. In particular, when a user is experiencing strong emotions such as fear, immediate action is required, but delays in making such decisions increase security risks. Furthermore, there was a lack of means to properly analyze and prioritize emotions expressed by users through voice or text. Therefore, it is necessary to provide more advanced support and improve the appropriateness and speed of emergency responses.

[1755] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving user information from a user terminal, means for storing the user information, means for processing support requests based on the user information, means for analyzing voice data or text data to recognize the emotional state, means for setting the priority of support requests based on the emotional state, and means for determining a response according to the priority and sending a notification. This enables emergency responses that take into account the user's emotions, reduces security risks, and enables faster and more appropriate responses.

[1756] A "user terminal" is an electronic device used by a user to input and transmit information.

[1757] "User information" refers to data that includes identifying information about a user.

[1758] A "support request" is data that indicates a user's request for assistance.

[1759] "Voice data" refers to data that includes voice information emitted by the user.

[1760] "Text data" refers to data that includes character information entered by the user.

[1761] "Emotional state" refers to the state of a user's emotions as analyzed from voice and text data.

[1762] "Priority" is an indicator that shows the importance of addressing a support request based on emotional state.

[1763] A "notification" is a message that informs you that a response has been taken in response to your support request.

[1764] A "security service system" is a system that handles requests for assistance and provides responses to ensure the safety of users.

[1765] This invention is a security service system that prioritizes security requests by considering user emotions and responds quickly. The system consists of a user terminal, a server, an emotion engine, and security staff.

[1766] System Configuration

[1767] User terminal

[1768] Users use electronic devices such as smartphones to input data via voice or text. For example, a smartphone app using React Native could be considered. This app receives user input and sends the data to a server.

[1769] server

[1770] The server is located in the cloud, for example, built on AWS EC2 using Node.js and the Express framework. The server receives and processes user information, support requests, and sentiment data. Specifically, it sends the aforementioned data to the Google Cloud Natural Language API for sentiment analysis.

[1771] Emotional Engine

[1772] The emotion engine uses the Google Cloud Natural Language API to analyze the user's voice or text data to determine their emotional state (e.g., "fear," "anger," etc.). The results of this analysis are then sent back to the server.

[1773] Security staff

[1774] The server prioritizes support requests based on their emotional state and determines how to respond. Based on priority, it sends a notification to the most appropriate security staff. Upon receiving the notification, the security staff quickly travels to the site and takes the necessary action.

[1775] Explanation of the program's processing details

[1776] The user opens a smartphone app and enters a message via voice or text saying, "I feel something suspicious is happening around my house and it's scary." The app converts this data into JSON format and sends it to a server. The server sends the voice or text data to the Google Cloud Natural Language API, where the emotion engine analyzes it as "fear." Based on this analysis, the server sets the case as high priority and determines that immediate action is required. A notification is then sent to the nearest security staff, who rush to the scene.

[1777] Example of a prompt

[1778] "When a user voice-inputs 'I feel scared because I sense suspicious activity around my house,' use the Google Cloud Natural Language API to analyze the emotion as 'fear.' Based on this result, design a program that sets the server to high priority and immediately notifies security staff."

[1779] This invention enables the reduction of security risks and the realization of faster and more appropriate responses by appropriately analyzing user emotions and performing highly accurate priority evaluations. This allows for the creation of security services that offer high reliability and safety.

[1780] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1781] Step 1:

[1782] The user launches the smartphone app and creates a security request. Specifically, the user inputs a voice or text message into the app saying, "I feel scared because I've noticed suspicious activity around my house." This input data is converted into JSON format. After receiving the input (voice or text) into the app, data converted into JSON format is generated.

[1783] Step 2:

[1784] The device sends the converted JSON data to the server. Specifically, the data is sent as an HTTP request from the app to the server. At this time, the input data is voice or text information including the user's emotions, and the output data is the JSON data sent to the server.

[1785] Step 3:

[1786] The server parses the received JSON data and sends it to the sentiment engine. Specifically, the server makes an API request to the Google Cloud Natural Language API to send data. At this time, the input data is the JSON data received by the server, and the output data is the data sent to the sentiment engine.

[1787] Step 4:

[1788] The emotion engine analyzes the user's emotional state from voice or text data. Using the Google Cloud Natural Language API, emotions such as "fear" are analyzed from the input voice or text. Here, the input is the data sent to the emotion engine, and the output is the analysis result.

[1789] Step 5:

[1790] The server receives the analysis results returned from the emotion engine and sets the priority of the support request. Specifically, a higher priority is set based on the analyzed emotional state of "fear." The input here is the emotion analysis result, and the output is the set priority.

[1791] Step 6:

[1792] The server sends notifications to the nearest security staff based on priority. Specifically, if the priority is high, the server sends an urgent notification to the security staff via the API. The input here is the configured priority, and the output is the notification sent to the security staff.

[1793] Step 7:

[1794] Security staff receive the notification and rush to the scene to respond. Security staff who receive the notification quickly arrive at the scene. In this step, the input is the notification to security staff, and the output is the on-site response.

[1795] In this way, each step works in conjunction, enabling swift and appropriate security responses that take user emotions into consideration.

[1796] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1797] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1798] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1799] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1800] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1801] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1802] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1803] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1804] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1805] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1806] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1807] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1808] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1809] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1810] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1811] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1812] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1813] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1814] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1815] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1816] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1817] The following is further disclosed regarding the embodiments described above.

[1818] (Claim 1)

[1819] A means of receiving user information from a user terminal,

[1820] Means for storing the user information,

[1821] Means for processing support requests based on the aforementioned user information,

[1822] A system that includes this.

[1823] (Claim 2)

[1824] The system according to claim 1, further comprising means for receiving the support request from the user terminal.

[1825] (Claim 3)

[1826] The system according to claim 1, wherein the means for processing the support request includes means for sending a notification to an emergency contact based on the user information.

[1827] "Example 1"

[1828] (Claim 1)

[1829] A means of receiving user information from a user terminal,

[1830] Means for storing the user information,

[1831] Means for processing support requests based on the aforementioned user information,

[1832] The processing means includes means for sending a notification to an emergency contact,

[1833] Means for displaying a pop-up message based on the user information,

[1834] Means for processing the HTTPS request and response,

[1835] A means of notifying support staff of the aforementioned support request,

[1836] A system that includes this.

[1837] (Claim 2)

[1838] The system according to claim 1, further comprising means for receiving the support request from the user terminal.

[1839] (Claim 3)

[1840] The system according to claim 1, wherein the means for processing the support request includes means for sending a notification to support staff based on the user information.

[1841] "Application Example 1"

[1842] (Claim 1)

[1843] A means of receiving user data from a user device,

[1844] Means for storing the aforementioned user data,

[1845] Means for processing support requests based on the aforementioned user data,

[1846] A means of notifying the aforementioned support request to the communications staff in real time,

[1847] A means by which communication staff can provide two-way communication to user devices,

[1848] A system that includes this.

[1849] (Claim 2)

[1850] The system according to claim 1, further comprising means for receiving the support request from the user device.

[1851] (Claim 3)

[1852] The system according to claim 1, wherein the means for processing the support request includes means for sending a notification to an emergency contact based on the user data.

[1853] "Example 2 of combining an emotion engine"

[1854] (Claim 1)

[1855] Means for receiving user data from user equipment,

[1856] Means for storing the user data,

[1857] A means for processing support requests based on the aforementioned user data,

[1858] A means including an emotion analysis engine for analyzing the user's emotional state and setting the priority of the aforementioned support request,

[1859] A system that includes this.

[1860] (Claim 2)

[1861] The system according to claim 1, further comprising means for receiving the support request from the user device.

[1862] (Claim 3)

[1863] The system according to claim 1, wherein the means for processing the support request includes means for sending a notification to a contact based on the user data.

[1864] "Application example 2 when combining with an emotional engine"

[1865] (Claim 1)

[1866] A means of receiving user information from a user terminal,

[1867] Means for storing the user information,

[1868] Means for processing support requests based on the aforementioned user information,

[1869] A means of recognizing emotional states by analyzing audio data or text data,

[1870] A means for setting the priority of support requests based on the aforementioned emotional state,

[1871] A means for determining the appropriate response and sending a notification according to the aforementioned priority,

[1872] A system that includes this.

[1873] (Claim 2)

[1874] The system according to claim 1, further comprising means for receiving the support request from the user terminal.

[1875] (Claim 3)

[1876] The system according to claim 1, wherein the means for processing the support request includes means for sending a notification to an emergency contact based on the user information. [Explanation of Symbols]

[1877] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving user information from a user terminal, Means for storing the aforementioned user information, Means for processing support requests based on the aforementioned user information, A system that includes this.

2. The system according to claim 1, further comprising means for receiving the support request from the user terminal.

3. The system according to claim 1, wherein the means for processing the support request includes means for sending a notification to an emergency contact based on the user information.

Images