System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a voice input; means for converting the received voice into text; means for analyzing the converted text to identify a request content; means for searching for a volunteer who can provide requested support based on the identified request content; means for transmitting a notification of the request to the volunteer; means for receiving a response from the volunteer; means for recording a history and an evaluation of the request; means for allowing the elderly person with dementia to make a voice request using a simple operation button; and means for generating and transmitting a response to the request.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, as the aging population progresses, the elderly are increasingly facing various challenges in their daily lives. In particular, due to the trend toward nuclear families and the weakening of local communities, an increasing number of elderly people are unable to receive appropriate support. Another problem is that many elderly people are unfamiliar with using smartphones and computers, which creates a barrier to accessing cutting-edge services. Furthermore, it can be difficult to communicate with elderly people with dementia, which is a factor that reduces the quality of support. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for accepting voice input, a means for converting the received voice into text, and a means for analyzing the converted text to identify the request content. It also includes a means for searching for volunteers who can provide the requested support based on the identified request content, a means for sending a request notification to the volunteer, and a means for accepting responses from the volunteers. It also includes a means for recording the request history and evaluation, supporting the provision of reliable services. Furthermore, it proposes a system that allows elderly people with dementia to make voice requests using simple operation buttons and includes a means for generating and sending responses in response to the requests. By utilizing generative artificial intelligence technology, it achieves a simple and powerful user interface that can be easily used even by elderly people who are not familiar with operating smartphones or PCs.

[0006] "Voice input" is a means by which seniors and users can communicate information and requests to a system through voice.

[0007] "Text conversion" is the process of analyzing audio data and converting it into text data.

[0008] "Request content" refers to the specific requirements for services and support that a user wants the system to provide.

[0009] "Volunteers" are local residents whose role is to provide support to elderly people upon request.

[0010] "Notifications" are messages that inform volunteers of request details and information from the system.

[0011] A "response" is an action or message that a volunteer sends in response to a notification from the system.

[0012] "History" is data that records past requests and volunteer responses.

[0013] "Evaluation" refers to feedback given by seniors and their families about the volunteer's services after the request has been carried out.

[0014] The "trust score" is an indicator of a volunteer's trustworthiness calculated based on their past history and evaluations.

[0015] "Generative AI technology" is a technology that automatically processes knowledge within a system, such as analyzing voice and text, identifying requests, and generating responses.

[0016] An "easy operation button" is an interface device designed to be easily operated by the elderly.

[0017] "Response generation" is the process by which a system automatically creates an appropriate reply or message in response to a user request.

[0018] The "System" is a comprehensive platform that includes functions such as voice input, text conversion, request identification, volunteer search, notification, and response, history and evaluation recording, and response generation. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. This system has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteers, thereby resolving the issues that elderly people face.

[0041] Program processing overview

[0042] 1. Elderly request registration

[0043] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[0044] Device: Uses generative AI to convert speech to text. The speech data is converted into text data: "I want you to take me shopping."

[0045] Server: Receives the text data and analyzes the request. As a result of the analysis, it determines that "shopping support" is required.

[0046] Device: Obtains the elderly person's GPS location information and sends it to the server.

[0047] 2. Volunteer matching

[0048] Server: Based on the elderly person's location information, create a list of nearby volunteers. For example, confirm that "Volunteer A" is nearby.

[0049] Server: Check the skill sets and available time of the volunteers and select the most suitable volunteer. Confirm that "Volunteer A" is available to provide shopping support.

[0050] Server: Sends a request notification to the volunteer. It notifies the volunteer that "Elderly B wants shopping assistance."

[0051] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[0052] 3. Support implementation and history management

[0053] User (volunteer): Visits elderly people and provides shopping support.

[0054] Terminal: Reports to the server that the volunteer has completed their support.

[0055] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[0056] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[0057] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[0058] 4. Communication with elderly people with dementia

[0059] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[0060] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[0061] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[0062] Specific examples

[0063] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[0064] 2. Terminal: The generation AI converts the speech into text and sends it to the server.

[0065] 3. Server: Analyzes the text and determines if shopping assistance is required.

[0066] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[0067] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[0068] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[0069] 7. Terminal: Reports to the server that volunteer A has completed the support.

[0070] 8. Server: Records support history and ratings, and updates the trust score.

[0071] 9. Terminal: Older people and their families evaluate the quality of support.

[0072] In this way, the present invention makes it possible to quickly and effectively match local volunteers with elderly people, thereby supporting the elderly in living independently.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[0076] Action: The device captures audio data and begins recording.

[0077] Step 2:

[0078] On-device: Uses generative AI to convert recorded audio into text in real time.

[0079] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[0080] Step 3:

[0081] Terminal: Sends text data to the server.

[0082] What it does: Uploads text data to a server over the internet.

[0083] Step 4:

[0084] Server: Receives and analyzes the text data to identify the request content.

[0085] How it works: Uses text analytics algorithms to identify that "shopping help" is needed.

[0086] Step 5:

[0087] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[0088] What it does: Uses the GPS module to capture current location information and upload it to a server.

[0089] Step 6:

[0090] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[0091] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[0092] Step 7:

[0093] Server: Check each volunteer's skill set and availability and select the best volunteer.

[0094] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[0095] Step 8:

[0096] Server: Sends notification of the request to selected volunteers.

[0097] What it does: Sends a push notification to the volunteer's device, displaying the request details.

[0098] Step 9:

[0099] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[0100] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[0101] Step 10:

[0102] User (volunteer): Visits elderly people and provides them with the necessary support.

[0103] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[0104] Step 11:

[0105] Terminal: Report to the server that the volunteer has discontinued support.

[0106] Operation: Press the support completion button and send a completion report to the server.

[0107] Step 12:

[0108] Server: Records support details and ratings in a database.

[0109] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[0110] Step 13:

[0111] Terminal: The elderly person or their family member enters their evaluation of the support and sends it to the server.

[0112] Action: The evaluation form is filled out on the device and uploaded to the server.

[0113] Step 14:

[0114] Server: Updates the trust database of volunteers based on the evaluation data.

[0115] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[0116] Step 15:

[0117] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[0118] What it does: Captures voice input from elderly people with dementia.

[0119] Step 16:

[0120] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[0121] What it does: Analyzes audio data and generates the conclusion "Needs meal assistance."

[0122] Step 17:

[0123] Server: Sends a response message to the terminal.

[0124] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[0125] Example 1

[0126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] There is a problem that it is difficult for elderly people to receive prompt and appropriate support from local volunteers when they face difficulties in their daily lives. Furthermore, in the case of elderly people with dementia, it becomes even more difficult to provide appropriate support due to the increased difficulty of responding to emergencies and communicating. Therefore, a system that provides an effective solution to these issues is needed.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0129] In this invention, the server includes means for the elderly to input voice, means for converting the voice input into text data, means for analyzing the text data to identify the request content, means for searching for an appropriate volunteer based on the identified request content, means for sending a request notification to the volunteer, means for accepting responses from the volunteer, means for recording the request history and evaluation, means for allowing the elderly to make voice requests using simple operation buttons, and means for generating and sending a response in accordance with the request. This enables the elderly to receive prompt and appropriate support from local volunteers for difficulties they face in their daily lives.

[0130] "Elderly" refers to individuals in an age group who require some assistance in daily living.

[0131] "Voice input" refers to a method in which elderly people speak into a dedicated device to capture voice information as digital data.

[0132] "Text data" refers to digital data converted from voice input into text information.

[0133] "Request content" refers to the specific request for assistance or support expressed by the elderly person through voice input.

[0134] "Volunteers" refer to local supporters whose purpose is to provide assistance to the elderly.

[0135] "Notification" refers to a message sent from the server to the volunteer informing them of the request.

[0136] "Responding" refers to the act of a volunteer responding to the server after receiving a request notification to let the server know whether or not they can respond.

[0137] "History" refers to the digital data stored record of past requests and their responses.

[0138] "Evaluation" refers to feedback given by seniors and their families regarding the volunteer's response to their request.

[0139] "Generative AI model" refers to the machine learning techniques used to convert voice data into text data and analyze that text data.

[0140] "Trustworthiness score" refers to an index that quantifies the reliability of a volunteer based on past evaluation data.

[0141] "Easy-to-operate buttons" refer to input devices designed to be easy to use even for elderly people who are unfamiliar with operating them.

[0142] "Response message" refers to the reply automatically generated by the server in response to a request entered by the elderly person.

[0143] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. The system of the present invention has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteer. This system allows elderly people to more easily solve problems in their daily lives.

[0144] Hardware and Software Configuration

[0145] The implementation of this system uses the following main hardware and software:

[0146] Hardware

[0147] 1. Dedicated device: A device equipped with a microphone and buttons for voice input by the elderly. This device is equipped with a microphone to capture voice data and a GPS module (e.g., U-Blox NEO-6M GPS module) to obtain location information.

[0148] 2. Smartphone: A device that allows volunteers to receive and respond to notifications. A dedicated application (e.g., ActionBound app) is installed.

[0149] software

[0150] 1. Generative AI model: A machine learning technology that converts voice data into text and then analyzes that text. The speech-to-text conversion is performed using the Google® Cloud Speech-to-Text API.

[0151] 2. Natural language processing engine: Use a natural language processing engine such as IBM Watson(R) NLU to analyze the request content from text data.

[0152] 3. Server system: This is the central system that receives requests from seniors, analyzes them, and notifies and matches them with volunteers.

[0153] Program processing

[0154] The program of this system operates according to the following processing flow.

[0155] Voice input for the elderly

[0156] The device starts when the elderly person presses a button on the dedicated device to input voice commands, for example, "Please take me shopping."

[0157] Speech to text

[0158] When the device receives voice input, it converts the data into text using a generative AI model (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[0159] Text data analysis

[0160] The server receives the text data and analyzes the request using a natural language processing engine (IBM Watson NLU). As a result of the analysis, it determines that "shopping support" is required.

[0161] Acquiring location information of elderly people

[0162] The device acquires the elderly person's GPS location information and sends it to the server, where the GPS module installed in the device measures their current location.

[0163] Volunteer Selection and Notification

[0164] The server uses the elderly person's location information to create a list of nearby volunteers, checks their skill sets and availability, and selects the most suitable volunteer. For example, it verifies that "Volunteer A" is nearby and available to assist with shopping. The server then sends a request notification to the volunteer.

[0165] Volunteer Response

[0166] The device helps the volunteer receive the notification and respond to the server whether they are available. The volunteer responds "I'm available" using the ActionBound app on their smartphone.

[0167] Support and history management

[0168] The user (volunteer) visits the elderly person and provides shopping support. The terminal reports to the server that the volunteer has completed the support. The server records the date, time, and content of the support, as well as the volunteer information, in a database. It also recalculates the reliability score and provides an interface for the elderly person and their family to evaluate the support.

[0169] Support for elderly people with dementia

[0170] The device allows elderly people with dementia to make voice requests using simple operation buttons, such as saying "I'm hungry." The server analyzes the voice in real time and generates an appropriate response message. The device then displays a message such as "We will contact a nearby volunteer."

[0171] Specific examples

[0172] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[0173] 2. Device: The generating AI (Google Cloud Speech-to-Text) converts the speech into text and sends it to the server.

[0174] 3. Server: Analyzes the text using IBM Watson NLU and determines if shopping assistance is required.

[0175] 4. Terminal: Sends the elderly person's location information (U-Blox NEO-6M GPS module) to the server.

[0176] 5. Server: Using the elderly person's location information, send a notification to nearby "Volunteer A."

[0177] 6. Device: Volunteer A receives a notification in the ActionBound app and replies that they are available.

[0178] 7. User (Volunteer A): A volunteer visits an elderly person and provides shopping assistance.

[0179] 8. Device: Volunteer A uses the ActionBound app to report completion of support to the server.

[0180] 9. Server: Records support history and ratings in a database and updates the trust score.

[0181] 10. Device: Elderly people and their families evaluate the quality of support through a smartphone rating app.

[0182] In this way, this system can effectively match local volunteers with elderly people and support their independent living.

[0183] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0184] Step 1:

[0185] Device: The elderly person presses a button on the dedicated device to input voice. For example, the elderly person may say, "I want you to take me shopping."

[0186] Input: Voice input for the elderly

[0187] Output: Audio data

[0188] Step 2:

[0189] Device: The voice input is converted into text using a generative AI (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[0190] Input: Audio data

[0191] Data processing: converting voice to text

[0192] Output: Text data

[0193] Step 3:

[0194] Terminal: The converted text data is sent to the server via an HTTP request.

[0195] Input: Text data

[0196] Output: Request sent to server

[0197] Step 4:

[0198] Server: Analyzes the received text data using IBM Watson NLU to identify the request. From the analysis results, it determines that "shopping support" is required.

[0199] Input: Text data

[0200] Data Calculation: Analyzing request content using natural language processing

[0201] Output: Identifying the request

[0202] Step 5:

[0203] Device: Acquires the elderly person's GPS location information and sends it to the server. The device is equipped with a GPS module (U-Blox NEO-6M GPS module).

[0204] Input: Location request

[0205] Data processing: Measure your current location and convert it into data

[0206] Output: Location data

[0207] Step 6:

[0208] Server: Creates a list of nearby volunteers based on the elderly person's location information.

[0209] Input: Location data

[0210] Data calculation: Generating a volunteer list through database searches

[0211] Output: Volunteer list

[0212] Step 7:

[0213] Server: Checks the skill sets and availability of volunteers and selects the most suitable volunteer. For example, identifies that "Volunteer A" is available to assist with shopping.

[0214] Input: Volunteer List

[0215] Data crunching: Checking volunteer skill sets and availability

[0216] Output: Selected volunteers

[0217] Step 8:

[0218] Server: Sends a request notification to the selected volunteer. Sends a notification saying, "Elderly B wants shopping assistance."

[0219] Input: Selected volunteer, request details

[0220] Output: Notification to volunteer

[0221] Step 9:

[0222] Terminal: After receiving the notification, the volunteer replies to the server whether they are available or not. Specifically, the volunteer uses a dedicated smartphone application (ActionBound app) to select "available" or "unavailable" and respond.

[0223] Input: Volunteer response

[0224] Output: Sending response data

[0225] Step 10:

[0226] User (volunteer): A volunteer visits an elderly person and provides designated support, such as taking them shopping.

[0227] Input: Elderly Request

[0228] Output: Providing support

[0229] Step 11:

[0230] Device: The volunteer reports to the server that they have completed their support. The volunteer uses the ActionBound app on their smartphone to press the "Support Completed" button.

[0231] Input: Support Completion Information

[0232] Output: Report to server

[0233] Step 12:

[0234] Server: Records support history and ratings in a database and updates the reliability score. Records the date and time of support, content, volunteer information, etc., and recalculates the reliability score.

[0235] Input: Support completion information, evaluation data

[0236] Data calculation: Recording support history and recalculating confidence scores

[0237] Output: Updated database

[0238] Step 13:

[0239] Device: Elderly people and their families use a smartphone rating app to rate the quality of support and send the results to the server.

[0240] Input: Evaluation information

[0241] Output: Send to server

[0242] Step 14:

[0243] Server: When an elderly person with dementia makes a voice request using the simple operation buttons, the voice is analyzed in real time and a response message is generated.

[0244] Input: Voice request

[0245] Data calculation: Real-time voice analysis, response message generation

[0246] Output: Response message

[0247] Step 15:

[0248] Server: Sends a response message to the terminal and displays it on the terminal. It displays a message such as "We will contact a nearby volunteer."

[0249] Input: Response message

[0250] Output: Display to terminal

[0251] (Application example 1)

[0252] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0253] The problem to be solved by this invention is to provide a system that provides prompt and appropriate assistance to seniors to eliminate the difficulties they face when searching for items in brick-and-mortar stores. In particular, this system allows seniors with dementia and those who require mobility in their daily lives to easily receive the necessary support through voice requests. Another problem is to ensure the reliability of the supporter and achieve effective support.

[0254] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0255] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the requested content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response in response to the request, and means for matching requests from elderly people who need item search or guidance with supporters in a physical store based on location information, thereby enabling items needed by elderly people to be searched for and guided to in a physical store quickly.

[0256] "Means for accepting voice input" refers to a device or apparatus that allows the elderly person to input requests by voice.

[0257] "Means for converting received voice into text" refers to technology or a system that converts voice input into text data.

[0258] "Means for analyzing textual content to identify the request content" refers to technologies and systems that understand textual data and grasp specific needs and requests.

[0259] A "means for searching for supporters who can provide the requested support" is a system for searching for people or organizations that can provide appropriate support based on the specified request content.

[0260] The "means for sending a request notification to the supporter" refers to a communication technology or system for informing the supporter of the request content.

[0261] "Means for receiving responses from supporters" refers to technologies and systems for receiving responses from supporters, such as whether they can respond or not.

[0262] The "means for recording request history and evaluation" is a system for storing the support provided and its evaluation in a database.

[0263] "Means that allow elderly people with dementia to make voice requests using simple operation buttons" refers to devices and technologies that provide an interface that can be easily operated by elderly people with dementia.

[0264] The "means for generating and transmitting a response in response to a request" is a system for generating an appropriate response to the request content and conveying it to the elderly person.

[0265] "Means for matching elderly people who need to search for items or receive guidance with helpers in physical stores based on location information" refers to technology or a system that uses the elderly person's location information to find helpers in physical stores who can provide appropriate assistance.

[0266] This invention is a system that solves the difficulties that elderly people face when searching for items in physical stores and provides them with prompt and appropriate assistance. This system is configured as follows.

[0267] First, the elderly person uses a device such as smart glasses or a smartphone to input voice commands. This device is equipped with a means to accept voice commands. For example, the elderly person can input a request by saying, "Please help me find milk."

[0268] The device converts speech into text data using generative AI technology. This activates a means for converting received speech into text. For example, OpenAI's GPT-3 (registered trademark) 5 is used as the generative AI model.

[0269] Next, a means for analyzing the textual content and identifying the content of the request operates on the server. This analysis means identifies that a "product guide" is required from the text data "Please help me find milk."

[0270] After the request content is specified, a means for searching for an assistant who can provide the requested support based on the specified request content operates on the server. This means matches the optimal assistant based on the location information, skill set, available time, etc. of the assistant in the physical store. For example, an assistant who is nearby is selected.

[0271] The server then runs a mechanism to notify the supporter of the request, and the supporter then responds to the server using a mechanism to respond to the notification, indicating whether they are able to respond. For example, notifications are sent using Firebase Cloud Messaging (FCM).

[0272] If the support person is available, they will visit the elderly person and assist them with product information and search. Once the support is complete, a system runs on the server to record the request history and evaluation. This records the support content and evaluation in a database (e.g., MySQL (registered trademark), Amazon RDS).

[0273] To accommodate elderly people with dementia, the terminals are equipped with easy-to-operate buttons and interfaces, allowing elderly people with dementia to make voice requests by simply operating buttons.

[0274] A means for generating and transmitting a response according to the request operates on the server. This means creates an appropriate response to the request content and displays it on the terminal. For example, a message such as "We will contact a nearby supporter" is displayed.

[0275] For example, an elderly person might say to the smart glasses, "Please help me find some milk." This speech is converted to text using a generative AI model, and the request is analyzed. An appropriate helper in the physical store is selected and notified. The helper visits the elderly person, provides them with the item, and their evaluation after completing the assistance is recorded in a database.

[0276] Example prompt sentence:

[0277] "An elderly person has made a voice request using smart glasses. Please analyze the following request to identify the appropriate action:

[0278] Voice Request: "Help me find some milk."

[0279] Analysis result: The request for this article is identified as "Product Information."

[0280] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0281] Step 1:

[0282] Elderly people input information by voice into a device (smart glasses or smartphone).

[0283] Input: Senior's voice request (e.g., "Please help me find milk.")

[0284] Output: Audio data

[0285] Specific operation: The device accepts voice input through the microphone and stores it as digital voice data.

[0286] Step 2:

[0287] The device converts the received voice data into text data using generative AI technology.

[0288] Input: Audio data

[0289] Output: Text data (e.g. "Please help me find milk")

[0290] How it works: The device uses a speech recognition library (e.g., Google Speech-to-Text API) to convert voice data into text, using a generative AI model (e.g., OpenAI GPT-3.5).

[0291] Step 3:

[0292] The server analyzes the text data and identifies the request content.

[0293] Input: Text data

[0294] Output: Request details (e.g. "Product information required")

[0295] How it works: The server uses natural language processing technology to analyze the text data and identify what the elderly person wants. This process uses a generative AI model.

[0296] Step 4:

[0297] The server obtains the elderly person's location information and searches for a supporter.

[0298] Input: Request details, location information of the elderly person

[0299] Output: List of supporters

[0300] Specific operation: Based on the GPS location information sent from the device, the server searches the supporter database and lists supporters who meet the requirements, using Google Maps API and iBeacon technology.

[0301] Step 5:

[0302] The server sends a notification of the request to the identified supporter.

[0303] Input: List of supporters, request details

[0304] Output: Notification message

[0305] Specific operation: The server sends a notification message to nearby supporters using Firebase Cloud Messaging (FCM).

[0306] Step 6:

[0307] The supporter receives the notification and responds to the server whether or not they can respond.

[0308] Input: Notification message

[0309] Output: Response regarding availability

[0310] Specific operation: The supporter checks the notification on a device such as a smartphone or tablet, selects whether they can respond, and responds to the server.

[0311] Step 7:

[0312] The corresponding support person provides assistance to the elderly person.

[0313] Input: Elderly person's location information, request details

[0314] Output: The results of the assistance provided

[0315] Specific actions: The supporter visits the elderly person and assists them in guiding them to items and searching for them. In this case, the support is provided face-to-face with the elderly person.

[0316] Step 8:

[0317] The server records the history of assistance and the elderly person's evaluation.

[0318] Input: Support details, evaluation data

[0319] Output: Updated database

[0320] Specific operation: After the supporter completes the support, he / she reports to the server via the terminal, and the elderly person and their family evaluate the quality of the support on the terminal. The evaluation results and support history are stored in a database.

[0321] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0322] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions.

[0323] Program processing overview

[0324] 1. Request registration and emotion recognition for elderly people

[0325] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[0326] On the device: It uses generative AI to convert speech to text, while simultaneously using an emotion engine to recognize the elderly person's emotions from their speech.

[0327] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "shopping support" is needed and recognizes that the elderly person is feeling "anxiety."

[0328] Device: Obtains the elderly person's GPS location information and sends it to the server.

[0329] 2. Volunteer matching and emotion-based prioritization

[0330] Server: Based on the elderly person's location information, it lists nearby available volunteers. For example, it checks whether "Volunteer A" is nearby.

[0331] Server: Check each volunteer's skill set and available time, and select the most suitable volunteer. Confirm that "Volunteer A" is available to assist with shopping.

[0332] Server: Adjust the priority of requests based on the perceived emotion. For example, requests from elderly people who are anxious will be treated as high priority.

[0333] Server: Sends a request notification to the volunteer. For example, it may say, "Elderly B would like shopping assistance. Elderly B is currently feeling anxious."

[0334] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[0335] 3. Support implementation and history management

[0336] User (volunteer): Visits elderly people and provides shopping support.

[0337] Terminal: Report to the server that the volunteer has discontinued support.

[0338] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[0339] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[0340] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[0341] Specific examples

[0342] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[0343] 2. Device: Generative AI and emotion engine converts speech into text and emotion data to identify "shopping assistance" and "anxiety."

[0344] 3. Server: Analyzes text and emotion data and determines that shopping support is needed and is a high priority.

[0345] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[0346] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[0347] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[0348] 7. Terminal: Reports to the server that volunteer A has completed the support.

[0349] 8. Server: Records support history and ratings, and updates the trust score.

[0350] 9. Terminal: Older people and their families evaluate the quality of support.

[0351] Communication with elderly people with dementia

[0352] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[0353] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[0354] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[0355] This invention provides a system that quickly and effectively matches local volunteers with elderly people and also responds appropriately based on the user's emotions, thereby supporting the independent living of elderly people and improving the quality of emotional support.

[0356] The processing flow will be explained below.

[0357] Step 1:

[0358] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[0359] Action: The device captures audio data and begins recording.

[0360] Step 2:

[0361] On-device: Uses generative AI to convert recorded audio into text in real time.

[0362] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[0363] Step 3:

[0364] Terminal: Recognize the emotions of elderly people from their voice using an emotion engine.

[0365] How it works: Analyzes audio data and generates emotion data (e.g., "anxiety") along with text data.

[0366] Step 4:

[0367] Terminal: Sends text data and emotion data to the server.

[0368] How it works: The generated text data and emotion data are uploaded to a server via the Internet.

[0369] Step 5:

[0370] Server: Receives text data and emotion data, analyzes and identifies the request content.

[0371] How it works: It uses text analytics algorithms to identify the need for "shopping assistance" and that the senior is feeling "anxious."

[0372] Step 6:

[0373] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[0374] What it does: Uses the GPS module to capture current location information and upload it to a server.

[0375] Step 7:

[0376] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[0377] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[0378] Step 8:

[0379] Server: Check each volunteer's skill set and availability and select the best volunteer.

[0380] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[0381] Step 9:

[0382] Server: Adjust the priority of requests based on perceived sentiment.

[0383] Behavior: If the emotional data is determined to be "anxiety," it is processed as high priority.

[0384] Step 10:

[0385] Server: Sends notification of the request to selected volunteers.

[0386] What it does: Sends a push notification to the volunteer's device with the message, "Elderly B wants shopping assistance. Elderly B is currently feeling anxious."

[0387] Step 11:

[0388] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[0389] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[0390] Step 12:

[0391] User (volunteer): Visits elderly people and provides them with the necessary support.

[0392] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[0393] Step 13:

[0394] Terminal: Report to the server that the volunteer has discontinued support.

[0395] Operation: Press the support completion button and send a completion report to the server.

[0396] Step 14:

[0397] Server: Records support details and ratings in a database.

[0398] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[0399] Step 15:

[0400] Device: Elderly people and their families rate the quality of support and send the results to the server.

[0401] Action: The evaluation form is filled out on the device and uploaded to the server.

[0402] Step 16:

[0403] Server: Updates the trust database of volunteers based on the evaluation data.

[0404] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[0405] Step 17:

[0406] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[0407] What it does: Captures voice input from elderly people with dementia.

[0408] Step 18:

[0409] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[0410] What it does: Analyzes audio data and generates a conclusion that determines "eating assistance is needed."

[0411] Step 19:

[0412] Server: Sends a response message to the terminal.

[0413] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[0414] Example 2

[0415] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0416] Responding appropriately and promptly to the difficulties faced by elderly people, especially those with dementia, in their daily lives is an important issue in modern society. However, current support systems lack sufficient functionality to effectively analyze user requests and quickly match them with the most suitable volunteers. They also lack the functionality to adjust response priorities while taking into account the emotional state of the elderly. This can result in delays in actual assistance or in the elderly not receiving sufficient support.

[0417] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0418] In this invention, the server includes a means for recognizing the user's emotion from voice, a means for acquiring the elderly person's location information, and a means for sending a request notification to a volunteer, thereby enabling prompt and appropriate volunteer matching that takes into account the elderly person's emotional state.

[0419] "Voice input" refers to the act of elderly people making requests using voice through a dedicated device.

[0420] "Text conversion" is the process of analyzing audio data and converting it into text information.

[0421] "Request content" refers to the specific content of the request conveyed by the elderly person through voice input.

[0422] "Emotion recognition" is a technology that identifies the emotional state of elderly people from voice data.

[0423] "Location information" is data that identifies the current location of an elderly person using GPS functionality.

[0424] A "volunteer" is someone who provides assistance in response to requests from elderly people.

[0425] A "notification" is a message sent by the server to a volunteer to inform them of the contents of the request.

[0426] "Responding" is the act of a volunteer replying to the server about whether they are available to respond to a request.

[0427] "History" is data that records the implementation details and evaluations of past requests.

[0428] The "easy operation button" is an input device designed to be easily operated by elderly people with dementia.

[0429] "Generative artificial intelligence technology" is a technology that uses generative AI models to analyze voice data and recognize emotions.

[0430] A "trust score" is an indicator of a volunteer's trustworthiness calculated based on their past evaluation data.

[0431] This invention is a system for responding quickly and appropriately to the difficulties that elderly people face in their daily lives, with the aim of providing support that takes into account the emotional state of the elderly. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, the system also has the ability to adjust the priority of requests and notification content using emotion recognition technology.

[0432] Hardware and Software Configuration

[0433] Devices: Older adults use dedicated devices (e.g., smart speakers with voice assistants, tablets).

[0434] Analysis software: Generative AI (e.g., Google Cloud Speech-to-Text) is used to convert voice data into text. Emotion engine (e.g., Microsoft® Azure® Emotion API) is used to recognize emotions.

[0435] Server: Receives text and emotion data, analyzes requests, searches for volunteers, sends notifications, receives responses, and records history and ratings.

[0436] GPS function: Use a smartphone or tablet with GPS functionality to track the senior's current location.

[0437] Specific examples of program processing

[0438] 1. Request registration and emotion recognition for elderly people

[0439] - Device: The elderly person presses a button on the device and voice-inputs, "I want you to take me shopping."

[0440] - Device: Uses generative AI to convert speech into text data, while simultaneously using an emotion engine to recognize emotions. Identifies that "shopping assistance" is needed and that the elderly person is feeling "anxious."

[0441] - Server: Receives text data and emotion data and analyzes the request content.

[0442] - Device: The elderly person's GPS location information is also sent to the server.

[0443] 2. Volunteer matching and emotion-based prioritization

[0444] - Server: Based on the received location information, it creates a list of nearby volunteers and checks each volunteer's skill set and available time. For example, it checks whether "Volunteer A" is available to provide shopping assistance.

[0445] - Server: Prioritize requests based on emotion data. Requests from elderly people who are anxious are given high priority.

[0446] - Server: Sends a request notification to the volunteer. The notification reads, "Elderly B would like shopping support. Elderly B is currently feeling anxious."

[0447] - Device: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" responds that they are available.

[0448] 3. Support implementation and history management

[0449] - User (volunteer): Visits elderly people and provides shopping support.

[0450] - Terminal: Report to the server that the volunteer has ended support.

[0451] - Server: Records support details and their evaluations in a database, such as support date and time, content, volunteer information, etc.

[0452] - Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[0453] - Server: Updates the trust database of volunteers based on the evaluation results, recalculates the trust score, and displays it to other seniors.

[0454] Examples of prompt statements

[0455] An elderly person speaks into the terminal, "I want you to take me shopping."

[0456] This system allows elderly people to receive daily support with peace of mind, enables volunteers to respond quickly and appropriately, and improves the quality of support by taking into account emotional states.

[0457] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0458] Detailed flow of program processing

[0459] 1. Request registration and emotion recognition for elderly people

[0460] Step 1: Getting voice input

[0461] Device: The elderly person presses a button on the dedicated device to input voice commands, such as "I want you to take me shopping."

[0462] Specific operation: After the elderly person presses the button, the microphone is activated and captures voice data.

[0463] Step 2: Speech-to-text and emotion recognition

[0464] Terminal: The voice data is converted into text using generative AI, and emotion data is extracted using an emotion engine. The input is voice data, and the output is text data ("Please take me shopping") and emotion data ("anxiety").

[0465] Specific operation: Send voice data to a generative AI (e.g., Google Cloud Speech-to-Text) to obtain text data, and simultaneously extract emotion data using an emotion engine (e.g., Microsoft Azure Emotion API).

[0466] Step 3: Sending text and emotion data

[0467] Terminal: Sends text data and emotion data to the server. The input is text data and emotion data, and the output is the server's confirmation of receipt.

[0468] Specific operation: The device sends data to the server's API endpoint, and the server receives the data.

[0469] Step 4: Obtaining location information

[0470] Terminal: Acquires the elderly person's GPS location information and sends it to the server. The input is GPS data, and the output is sending location information to the server.

[0471] Specific operation: The device uses the built-in GPS function to obtain its current location and sends the location information to the server.

[0472] 2. Volunteer matching and emotion-based prioritization

[0473] Step 5: Create a list of neighborhood volunteers

[0474] Server: Based on the received location information, it creates a list of available volunteers in the vicinity. The input is the location information, and the output is the volunteer list.

[0475] What happens: The server searches the database and lists volunteers within the specified radius.

[0476] Step 6: Identify volunteer skillsets and availability

[0477] Server: Identifies the best volunteers by checking the skill sets and availability of each volunteer. Input is the list of volunteers, output is the best volunteers.

[0478] What happens: The server looks up the volunteer profile information and identifies volunteers who match the requirements.

[0479] Step 7: Prioritize requests based on sentiment data

[0480] Server: Adjusts the priority of requests based on the recognized emotion. The input is emotion data, and the output is the adjusted priority.

[0481] What it does: A priority adjustment algorithm analyzes sentiment data and adjusts the request's queue position.

[0482] Step 8: Sending a Request Notification

[0483] Server: Sends request notifications to volunteers. Inputs are the best volunteer and the request content, and output is the notification message.

[0484] Specific operation: The server sends a push notification to the volunteer's device, displaying the request details.

[0485] Step 9: Receiving volunteer responses

[0486] Terminal: The volunteer receives the notification and replies to the server whether they are available. The input is the volunteer's response, and the output is the response sent to the server.

[0487] What happens: The volunteer taps the notification message to send a reply message from the application.

[0488] 3. Support implementation and history management

[0489] Step 10: Providing support

[0490] User (volunteer): Visits elderly people and provides shopping support. The input is the elderly person's request, and the output is the implementation of the support.

[0491] Specific actions: Volunteers visit elderly people's homes and provide shopping assistance.

[0492] Step 11: Report support complete

[0493] Terminal: Reports to the server that the volunteer has ended support. Input is the completion status, and output is the report to the server.

[0494] Specific action: The volunteer operates the application on the device to update the status of support completion.

[0495] Step 12: Record support and evaluation

[0496] Server: Records support details and evaluations in a database. The input is support details and evaluation data, and the output is the recorded data.

[0497] Specific behavior: The server receives the evaluation result and adds a record to the database.

[0498] Step 13: Evaluate the quality of support

[0499] Terminal: Elderly people and their families evaluate the quality of support and send the results to the server. The input is the evaluation result, and the output is the data sent to the server.

[0500] Specific operation: Elderly people and their families fill out an evaluation form and send the results to the server.

[0501] Step 14: Update the Reliability Database

[0502] Server: Updates the trust database of volunteers based on the evaluation results. The input is the evaluation results, and the output is the updated trust scores.

[0503] Specific behavior: The server recalculates the trust score based on the new reputation score and updates the database.

[0504] (Application example 2)

[0505] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0506] In the daily lives of elderly people, a system that can respond quickly and appropriately is necessary, especially in emergencies or situations where they feel anxious. Conventional systems can convert voice-input requests into text and match them with supporters, but no systems exist that have security features or priority adjustment that take user emotions into account. Therefore, there is a need for a method to adjust the priority of requests based on emotions and ensure emergency responses according to specific situations.

[0507] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0508] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the request content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response according to the request, means for recognizing emotions and adjusting the priority of the request, and means having a security function for immediately notifying after detecting the situation. This makes it possible to take emotions into consideration when an elderly person makes a voice request and quickly respond to requests with high urgency.

[0509] "Means for accepting voice input" refers to a device or interface that receives voice commands issued by the senior.

[0510] "Means of converting received voice into text" refers to the technology or software that converts voice data into text data.

[0511] "Means for analyzing the textual content and identifying the request content" refers to algorithms or programs that analyze textual data and identify the elderly person's requests from its content.

[0512] "Means for searching for supporters who can provide the requested support based on the specified request content" refers to a function for searching a database for supporters who match the request.

[0513] "Means for sending a request notification to a supporter" refers to a communication technology for sending request information to a specified supporter.

[0514] "Means for accepting responses from supporters" refers to the function of receiving confirmation of receipt and replies from supporters.

[0515] "Means for recording request history and evaluation" refers to a system that stores and manages history and evaluation information regarding requests and support for older adults.

[0516] "A means for enabling elderly people with dementia to make voice requests using simple operation buttons" refers to an interface that allows elderly people with dementia to input voice commands with simple operations.

[0517] "Means for generating and sending a response in response to a request" refers to the function of generating an appropriate response based on the content of the request and sending it to the elderly person.

[0518] "Means for recognizing emotions and adjusting the priority of requests" refers to an algorithm that analyzes the emotional state of the elderly person and determines the priority of processing requests accordingly.

[0519] "Having security functions and means to immediately notify after detecting a situation" refers to the function to detect abnormal or emergency situations and immediately notify the relevant security services of that information.

[0520] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with appropriate supporters (volunteers, etc.), this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions. The system also has security functions and is equipped with a means to immediately notify users when a suspicious situation is detected.

[0521] 1. Voice Input and Emotion Recognition

[0522] Device: The elderly person speaks into a dedicated device (e.g., smartphone, robot) and says, for example, "Someone is coming into the house."

[0523] Device: Uses generative AI (e.g., Google Speech-to-Text API) to convert speech to text, while simultaneously using an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the elderly person's emotions from their speech.

[0524] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "suspicious person detection" is required and that the elderly person is feeling "fear."

[0525] 2. Matching supporters and adjusting priorities based on emotions

[0526] Server: Adjust the priority of requests based on the recognized emotion. For example, requests from elderly people who feel "fear" will be given top priority.

[0527] Server: Based on the elderly person's location information and request details, the server creates a list of nearby available supporters. For example, it checks whether "Supporter A" is nearby.

[0528] Server: Sends a request notification to the supporter. For example, it may say, "Elderly B wants to detect suspicious people. Elderly B is currently feeling scared."

[0529] Terminal: The supporter receives the notification and replies to the server whether they are available. For example, "Supporter A" responds that they are available.

[0530] 3. Support implementation and history management

[0531] User (supporter): Visits the elderly person's home and provides support. For example, checking the house for suspicious people.

[0532] Terminal: Report to the server that the supporter has ended support.

[0533] Server: Records the support content and its evaluation in a database, including the date and time of the support, the content, and information about the supporter.

[0534] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. For example, they may rate "Supporter A" five stars.

[0535] Server: Updates the database of supporter reliability based on the evaluation results. Recalculates the reliability score and displays it to other seniors.

[0536] 4. Implementation of security functions

[0537] Server: Detects suspicious situations from voice input and immediately notifies the security service.

[0538] Security Services: Review surveillance footage at the senior's home in real time and dispatch emergency services if necessary.

[0539] Specific examples

[0540] 1. An elderly person tells a robot installed in their living room, "It feels like someone has come into my house."

[0541] 2. The robot converts the voice into text and emotional data (e.g., fear), analyzes it, and determines whether suspicious person detection is necessary.

[0542] 3. The server processes the request with high priority and sends a notification to the security service.

[0543] 4. Security services will review surveillance footage and respond to emergencies as necessary.

[0544] Example of input prompt for generative AI model

[0545] Input prompt: Describe a suspicious person detection system for when an elderly person says "someone has entered the house" and then feels scared. Please be specific and include the voice analysis, emotion engine, and security notification flow.

[0546] Example output: When an elderly person tells the robot, "Someone has come into the house," a speech analysis engine (e.g., Google Speech-to-Text API) converts the speech into text, and an emotion engine (e.g., IBM Watson Tone Analyzer) detects and analyzes the elderly person's fear. The analysis information is notified to the security service with high priority, which checks surveillance camera footage and takes any necessary emergency action.

[0547] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0548] Step 1:

[0549] The device accepts voice input from the elderly person. For example, the elderly person may say, "Someone has come into the house." The input voice is converted into digital voice data via the device's microphone.

[0550] Step 2:

[0551] The speech received by the device is converted into text data using a generation AI (e.g., Google Speech-to-Text API). The input speech data is processed by a speech recognition model via the API, and text data is generated as a result. The output is the text data "Someone has entered the house."

[0552] Step 3:

[0553] The device passes the text data to an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to generate emotion data. The input text data is processed by the emotion analysis model, and the output emotion data is "fear."

[0554] Step 4:

[0555] The server receives the text data and emotion data and analyzes the request content. Using the text data "Someone entered the house" and the emotion data "fear" as input, the text analysis algorithm identifies the request content "suspicious person detected." The request content is identified as the output.

[0556] Step 5:

[0557] The server adjusts the priority of the request based on the request content and emotion data. Using the "suspicious person detected" content and the emotion data of "fear" as input, the priority adjustment algorithm gives the request the highest priority. The output is set as a "high priority" request.

[0558] Step 6:

[0559] The server searches for nearby available supporters based on the elderly person's location information and the request details. Using the elderly person's GPS location information and the request details for "suspicious person detection" as input, a database search algorithm is used to identify an appropriate supporter. The output is "Supporter A."

[0560] Step 7:

[0561] The server sends a request notification to the supporter. Using the identified supporter "Supporter A" and the request content "Suspicious person detected" as input, the request information is sent through the notification system to the supporter. The notification is delivered to the supporter as output.

[0562] Step 8:

[0563] The terminal receives a response from the supporter. Using the supporter's receipt notification as input, the supporter responds that they are available. The response "available" is obtained as output.

[0564] Step 9:

[0565] The user (supporter) visits the elderly person's home and provides support, specifically by checking the house for any suspicious individuals.

[0566] Step 10:

[0567] The device reports to the server that the supporter has ended support. The supporter's termination notification is used as input and reported to the server. The "support end" report is recorded as output.

[0568] Step 11:

[0569] The server records the support content and its evaluation in a database. The support date and time, content, supporter information, and evaluation are used as input and saved in the database. The support history is updated as output.

[0570] Step 12:

[0571] The terminal allows elderly people and their families to evaluate the quality of support and sends the results to the server. The evaluation information is used as input and sent to the server. The evaluation results are recorded in the database as output.

[0572] Step 13:

[0573] The server updates the trust database of supporters based on the evaluation results. The trust database is updated using an algorithm that uses the evaluation data as input and calculates the trust score. The updated trust score is obtained as output.

[0574] Step 14:

[0575] The server detects suspicious situations from voice input and immediately notifies the security service. Using the elderly person's voice request and emotion data as input, the server identifies suspicious situations using an anomaly detection algorithm and sends a notification to the security service. An emergency notification is sent as output.

[0576] Step 15:

[0577] Security services receive the notification, review the surveillance footage at the senior's home, and dispatch emergency services if necessary. The notification information is used as input to review the situation on the surveillance video system. The output is the initiation of an emergency response.

[0578] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0579] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0580] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0581] [Second embodiment]

[0582] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0583] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0584] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0585] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0586] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0587] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0588] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0589] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0590] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0593] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0594] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. This system has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteers, thereby resolving the issues that elderly people face.

[0595] Program processing overview

[0596] 1. Elderly request registration

[0597] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[0598] Device: Uses generative AI to convert speech to text. The speech data is converted into text data: "I want you to take me shopping."

[0599] Server: Receives the text data and analyzes the request. As a result of the analysis, it determines that "shopping support" is required.

[0600] Device: Obtains the elderly person's GPS location information and sends it to the server.

[0601] 2. Volunteer matching

[0602] Server: Based on the elderly person's location information, create a list of nearby volunteers. For example, confirm that "Volunteer A" is nearby.

[0603] Server: Check the skill sets and available time of the volunteers and select the most suitable volunteer. Confirm that "Volunteer A" is available to provide shopping support.

[0604] Server: Sends a request notification to the volunteer. It notifies the volunteer that "Elderly B wants shopping assistance."

[0605] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[0606] 3. Support implementation and history management

[0607] User (volunteer): Visits elderly people and provides shopping support.

[0608] Terminal: Reports to the server that the volunteer has completed their support.

[0609] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[0610] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[0611] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[0612] 4. Communication with elderly people with dementia

[0613] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[0614] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[0615] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[0616] Specific examples

[0617] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[0618] 2. Terminal: The generation AI converts the speech into text and sends it to the server.

[0619] 3. Server: Analyzes the text and determines if shopping assistance is required.

[0620] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[0621] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[0622] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[0623] 7. Terminal: Reports to the server that volunteer A has completed the support.

[0624] 8. Server: Records support history and ratings, and updates the trust score.

[0625] 9. Terminal: Older people and their families evaluate the quality of support.

[0626] In this way, the present invention makes it possible to quickly and effectively match local volunteers with elderly people, thereby supporting the elderly in living independently.

[0627] The processing flow will be explained below.

[0628] Step 1:

[0629] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[0630] Action: The device captures audio data and begins recording.

[0631] Step 2:

[0632] On-device: Uses generative AI to convert recorded audio into text in real time.

[0633] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[0634] Step 3:

[0635] Terminal: Sends text data to the server.

[0636] What it does: Uploads text data to a server over the internet.

[0637] Step 4:

[0638] Server: Receives and analyzes the text data to identify the request content.

[0639] How it works: Uses text analytics algorithms to identify that "shopping help" is needed.

[0640] Step 5:

[0641] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[0642] What it does: Uses the GPS module to capture current location information and upload it to a server.

[0643] Step 6:

[0644] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[0645] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[0646] Step 7:

[0647] Server: Check each volunteer's skill set and availability and select the best volunteer.

[0648] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[0649] Step 8:

[0650] Server: Sends notification of the request to selected volunteers.

[0651] What it does: Sends a push notification to the volunteer's device, displaying the request details.

[0652] Step 9:

[0653] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[0654] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[0655] Step 10:

[0656] User (volunteer): Visits elderly people and provides them with the necessary support.

[0657] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[0658] Step 11:

[0659] Terminal: Report to the server that the volunteer has discontinued support.

[0660] Operation: Press the support completion button and send a completion report to the server.

[0661] Step 12:

[0662] Server: Records support details and ratings in a database.

[0663] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[0664] Step 13:

[0665] Terminal: The elderly person or their family member enters their evaluation of the support and sends it to the server.

[0666] Action: The evaluation form is filled out on the device and uploaded to the server.

[0667] Step 14:

[0668] Server: Updates the trust database of volunteers based on the evaluation data.

[0669] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[0670] Step 15:

[0671] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[0672] What it does: Captures voice input from elderly people with dementia.

[0673] Step 16:

[0674] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[0675] What it does: Analyzes audio data and generates the conclusion "Needs meal assistance."

[0676] Step 17:

[0677] Server: Sends a response message to the terminal.

[0678] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[0679] Example 1

[0680] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0681] There is a problem that it is difficult for elderly people to receive prompt and appropriate support from local volunteers when they face difficulties in their daily lives. Furthermore, in the case of elderly people with dementia, it becomes even more difficult to provide appropriate support due to the increased difficulty of responding to emergencies and communicating. Therefore, a system that provides an effective solution to these issues is needed.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0683] In this invention, the server includes means for the elderly to input voice, means for converting the voice input into text data, means for analyzing the text data to identify the request content, means for searching for an appropriate volunteer based on the identified request content, means for sending a request notification to the volunteer, means for accepting responses from the volunteer, means for recording the request history and evaluation, means for allowing the elderly to make voice requests using simple operation buttons, and means for generating and sending a response in accordance with the request. This enables the elderly to receive prompt and appropriate support from local volunteers for difficulties they face in their daily lives.

[0684] "Elderly" refers to individuals in an age group who require some assistance in daily living.

[0685] "Voice input" refers to a method in which elderly people speak into a dedicated device to capture voice information as digital data.

[0686] "Text data" refers to digital data converted from voice input into text information.

[0687] "Request content" refers to the specific request for assistance or support expressed by the elderly person through voice input.

[0688] "Volunteers" refer to local supporters whose purpose is to provide assistance to the elderly.

[0689] "Notification" refers to a message sent from the server to the volunteer informing them of the request.

[0690] "Responding" refers to the act of a volunteer responding to the server after receiving a request notification to let the server know whether or not they can respond.

[0691] "History" refers to the digital data stored record of past requests and their responses.

[0692] "Evaluation" refers to feedback given by seniors and their families regarding the volunteer's response to their request.

[0693] "Generative AI model" refers to the machine learning techniques used to convert voice data into text data and analyze that text data.

[0694] "Trustworthiness score" refers to an index that quantifies the reliability of a volunteer based on past evaluation data.

[0695] "Easy-to-operate buttons" refer to input devices designed to be easy to use even for elderly people who are unfamiliar with operating them.

[0696] "Response message" refers to the reply automatically generated by the server in response to a request entered by the elderly person.

[0697] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. The system of the present invention has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteer. This system allows elderly people to more easily solve problems in their daily lives.

[0698] Hardware and Software Configuration

[0699] The implementation of this system uses the following main hardware and software:

[0700] Hardware

[0701] 1. Dedicated device: A device equipped with a microphone and buttons for voice input by the elderly. This device is equipped with a microphone to capture voice data and a GPS module (e.g., U-Blox NEO-6M GPS module) to obtain location information.

[0702] 2. Smartphone: A device that allows volunteers to receive and respond to notifications. A dedicated application (e.g., ActionBound app) is installed.

[0703] software

[0704] 1. Generative AI model: This is a machine learning technology that converts voice data into text and then analyzes that text. For voice-to-text conversion, we use the Google Cloud Speech-to-Text API.

[0705] 2. Natural language processing engine: Use a natural language processing engine such as IBM Watson NLU to analyze the request content from text data.

[0706] 3. Server system: This is the central system that receives requests from seniors, analyzes them, and notifies and matches them with volunteers.

[0707] Program processing

[0708] The program of this system operates according to the following processing flow.

[0709] Voice input for the elderly

[0710] The device starts when the elderly person presses a button on the dedicated device to input voice commands, for example, "Please take me shopping."

[0711] Speech to text

[0712] When the device receives voice input, it converts the data into text using a generative AI model (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[0713] Text data analysis

[0714] The server receives the text data and analyzes the request using a natural language processing engine (IBM Watson NLU). As a result of the analysis, it determines that "shopping support" is required.

[0715] Acquiring location information of elderly people

[0716] The device acquires the elderly person's GPS location information and sends it to the server, where the GPS module installed in the device measures their current location.

[0717] Volunteer Selection and Notification

[0718] The server uses the elderly person's location information to create a list of nearby volunteers, checks their skill sets and availability, and selects the most suitable volunteer. For example, it verifies that "Volunteer A" is nearby and available to assist with shopping. The server then sends a request notification to the volunteer.

[0719] Volunteer Response

[0720] The device helps the volunteer receive the notification and respond to the server whether they are available. The volunteer responds "I'm available" using the ActionBound app on their smartphone.

[0721] Support and history management

[0722] The user (volunteer) visits the elderly person and provides shopping support. The terminal reports to the server that the volunteer has completed the support. The server records the date, time, and content of the support, as well as the volunteer information, in a database. It also recalculates the reliability score and provides an interface for the elderly person and their family to evaluate the support.

[0723] Support for elderly people with dementia

[0724] The device allows elderly people with dementia to make voice requests using simple operation buttons, such as saying "I'm hungry." The server analyzes the voice in real time and generates an appropriate response message. The device then displays a message such as "We will contact a nearby volunteer."

[0725] Specific examples

[0726] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[0727] 2. Device: The generating AI (Google Cloud Speech-to-Text) converts the speech into text and sends it to the server.

[0728] 3. Server: Analyzes the text using IBM Watson NLU and determines if shopping assistance is required.

[0729] 4. Terminal: Sends the elderly person's location information (U-Blox NEO-6M GPS module) to the server.

[0730] 5. Server: Using the elderly person's location information, send a notification to nearby "Volunteer A."

[0731] 6. Device: Volunteer A receives a notification in the ActionBound app and replies that they are available.

[0732] 7. User (Volunteer A): A volunteer visits an elderly person and provides shopping assistance.

[0733] 8. Device: Volunteer A uses the ActionBound app to report completion of support to the server.

[0734] 9. Server: Records support history and ratings in a database and updates the trust score.

[0735] 10. Device: Elderly people and their families evaluate the quality of support through a smartphone rating app.

[0736] In this way, this system can effectively match local volunteers with elderly people and support their independent living.

[0737] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0738] Step 1:

[0739] Device: The elderly person presses a button on the dedicated device to input voice. For example, the elderly person may say, "I want you to take me shopping."

[0740] Input: Voice input for the elderly

[0741] Output: Audio data

[0742] Step 2:

[0743] Device: The voice input is converted into text using a generative AI (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[0744] Input: Audio data

[0745] Data processing: converting voice to text

[0746] Output: Text data

[0747] Step 3:

[0748] Terminal: The converted text data is sent to the server via an HTTP request.

[0749] Input: Text data

[0750] Output: Request sent to server

[0751] Step 4:

[0752] Server: Analyzes the received text data using IBM Watson NLU to identify the request. From the analysis results, it determines that "shopping support" is required.

[0753] Input: Text data

[0754] Data Calculation: Analyzing request content using natural language processing

[0755] Output: Identifying the request

[0756] Step 5:

[0757] Device: Acquires the elderly person's GPS location information and sends it to the server. The device is equipped with a GPS module (U-Blox NEO-6M GPS module).

[0758] Input: Location request

[0759] Data processing: Measure your current location and convert it into data

[0760] Output: Location data

[0761] Step 6:

[0762] Server: Creates a list of nearby volunteers based on the elderly person's location information.

[0763] Input: Location data

[0764] Data calculation: Generating a volunteer list through database searches

[0765] Output: Volunteer list

[0766] Step 7:

[0767] Server: Checks the skill sets and availability of volunteers and selects the most suitable volunteer. For example, identifies that "Volunteer A" is available to assist with shopping.

[0768] Input: Volunteer List

[0769] Data crunching: Checking volunteer skill sets and availability

[0770] Output: Selected volunteers

[0771] Step 8:

[0772] Server: Sends a request notification to the selected volunteer. Sends a notification saying, "Elderly B wants shopping assistance."

[0773] Input: Selected volunteer, request details

[0774] Output: Notification to volunteer

[0775] Step 9:

[0776] Terminal: After receiving the notification, the volunteer replies to the server whether they are available or not. Specifically, the volunteer uses a dedicated smartphone application (ActionBound app) to select "available" or "unavailable" and respond.

[0777] Input: Volunteer response

[0778] Output: Sending response data

[0779] Step 10:

[0780] User (volunteer): A volunteer visits an elderly person and provides designated support, such as taking them shopping.

[0781] Input: Elderly Request

[0782] Output: Providing support

[0783] Step 11:

[0784] Device: The volunteer reports to the server that they have completed their support. The volunteer uses the ActionBound app on their smartphone to press the "Support Completed" button.

[0785] Input: Support Completion Information

[0786] Output: Report to server

[0787] Step 12:

[0788] Server: Records support history and ratings in a database and updates the reliability score. Records the date and time of support, content, volunteer information, etc., and recalculates the reliability score.

[0789] Input: Support completion information, evaluation data

[0790] Data calculation: Recording support history and recalculating confidence scores

[0791] Output: Updated database

[0792] Step 13:

[0793] Device: Elderly people and their families use a smartphone rating app to rate the quality of support and send the results to the server.

[0794] Input: Evaluation information

[0795] Output: Send to server

[0796] Step 14:

[0797] Server: When an elderly person with dementia makes a voice request using the simple operation buttons, the voice is analyzed in real time and a response message is generated.

[0798] Input: Voice request

[0799] Data calculation: Real-time voice analysis, response message generation

[0800] Output: Response message

[0801] Step 15:

[0802] Server: Sends a response message to the terminal and displays it on the terminal. It displays a message such as "We will contact a nearby volunteer."

[0803] Input: Response message

[0804] Output: Display to terminal

[0805] (Application example 1)

[0806] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0807] The problem to be solved by this invention is to provide a system that provides prompt and appropriate assistance to seniors to eliminate the difficulties they face when searching for items in brick-and-mortar stores. In particular, this system allows seniors with dementia and those who require mobility in their daily lives to easily receive the necessary support through voice requests. Another problem is to ensure the reliability of the supporter and achieve effective support.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0809] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the requested content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response in response to the request, and means for matching requests from elderly people who need item search or guidance with supporters in a physical store based on location information, thereby enabling items needed by elderly people to be searched for and guided to in a physical store quickly.

[0810] "Means for accepting voice input" refers to a device or apparatus that allows the elderly person to input requests by voice.

[0811] "Means for converting received voice into text" refers to technology or a system that converts voice input into text data.

[0812] "Means for analyzing textual content to identify the request content" refers to technologies and systems that understand textual data and grasp specific needs and requests.

[0813] A "means for searching for supporters who can provide the requested support" is a system for searching for people or organizations that can provide appropriate support based on the specified request content.

[0814] The "means for sending a request notification to the supporter" refers to a communication technology or system for informing the supporter of the request content.

[0815] "Means for receiving responses from supporters" refers to technologies and systems for receiving responses from supporters, such as whether they can respond or not.

[0816] The "means for recording request history and evaluation" is a system for storing the support provided and its evaluation in a database.

[0817] "Means that allow elderly people with dementia to make voice requests using simple operation buttons" refers to devices and technologies that provide an interface that can be easily operated by elderly people with dementia.

[0818] The "means for generating and transmitting a response in response to a request" is a system for generating an appropriate response to the request content and conveying it to the elderly person.

[0819] "Means for matching elderly people who need to search for items or receive guidance with helpers in physical stores based on location information" refers to technology or a system that uses the elderly person's location information to find helpers in physical stores who can provide appropriate assistance.

[0820] This invention is a system that solves the difficulties that elderly people face when searching for items in physical stores and provides them with prompt and appropriate assistance. This system is configured as follows.

[0821] First, the elderly person uses a device such as smart glasses or a smartphone to input voice commands. This device is equipped with a means to accept voice commands. For example, the elderly person can input a request by saying, "Please help me find milk."

[0822] The device uses generative AI technology to convert speech into text data, which then activates a means of converting received speech into text. OpenAI's GPT-3.5, for example, is used as the generative AI model.

[0823] Next, a means for analyzing the textual content and identifying the content of the request operates on the server. This analysis means identifies that a "product guide" is required from the text data "Please help me find milk."

[0824] After the request content is specified, a means for searching for an assistant who can provide the requested support based on the specified request content operates on the server. This means matches the optimal assistant based on the location information, skill set, available time, etc. of the assistant in the physical store. For example, an assistant who is nearby is selected.

[0825] The server then runs a mechanism to notify the supporter of the request, and the supporter then responds to the server using a mechanism to respond to the notification, indicating whether they are able to respond. For example, notifications are sent using Firebase Cloud Messaging (FCM).

[0826] If the assistant is available, they visit the elderly person and assist them with product information and search. Once the assistance is complete, a system runs on the server to record the request history and evaluation. This records the assistance details and evaluation in a database (e.g., MySQL, Amazon RDS).

[0827] To accommodate elderly people with dementia, the terminals are equipped with easy-to-operate buttons and interfaces, allowing elderly people with dementia to make voice requests by simply operating buttons.

[0828] A means for generating and transmitting a response according to the request operates on the server. This means creates an appropriate response to the request content and displays it on the terminal. For example, a message such as "We will contact a nearby supporter" is displayed.

[0829] For example, an elderly person might say to the smart glasses, "Please help me find some milk." This speech is converted to text using a generative AI model, and the request is analyzed. An appropriate helper in the physical store is selected and notified. The helper visits the elderly person, provides them with the item, and their evaluation after completing the assistance is recorded in a database.

[0830] Example prompt sentence:

[0831] "An elderly person has made a voice request using smart glasses. Please analyze the following request to identify the appropriate action:

[0832] Voice Request: "Help me find some milk."

[0833] Analysis result: The request for this article is identified as "Product Information."

[0834] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0835] Step 1:

[0836] Elderly people input information by voice into a device (smart glasses or smartphone).

[0837] Input: Senior's voice request (e.g., "Please help me find milk.")

[0838] Output: Audio data

[0839] Specific operation: The device accepts voice input through the microphone and stores it as digital voice data.

[0840] Step 2:

[0841] The device converts the received voice data into text data using generative AI technology.

[0842] Input: Audio data

[0843] Output: Text data (e.g. "Please help me find milk")

[0844] How it works: The device uses a speech recognition library (e.g., Google Speech-to-Text API) to convert voice data into text, using a generative AI model (e.g., OpenAI GPT-3.5).

[0845] Step 3:

[0846] The server analyzes the text data and identifies the request content.

[0847] Input: Text data

[0848] Output: Request details (e.g. "Product information required")

[0849] How it works: The server uses natural language processing technology to analyze the text data and identify what the elderly person wants. This process uses a generative AI model.

[0850] Step 4:

[0851] The server obtains the elderly person's location information and searches for a supporter.

[0852] Input: Request details, location information of the elderly person

[0853] Output: List of supporters

[0854] Specific operation: Based on the GPS location information sent from the device, the server searches the supporter database and lists supporters who meet the requirements, using Google Maps API and iBeacon technology.

[0855] Step 5:

[0856] The server sends a notification of the request to the identified supporter.

[0857] Input: List of supporters, request details

[0858] Output: Notification message

[0859] Specific operation: The server sends a notification message to nearby supporters using Firebase Cloud Messaging (FCM).

[0860] Step 6:

[0861] The supporter receives the notification and responds to the server whether or not they can respond.

[0862] Input: Notification message

[0863] Output: Response regarding availability

[0864] Specific operation: The supporter checks the notification on a device such as a smartphone or tablet, selects whether they can respond, and responds to the server.

[0865] Step 7:

[0866] The corresponding support person provides assistance to the elderly person.

[0867] Input: Elderly person's location information, request details

[0868] Output: The results of the assistance provided

[0869] Specific actions: The supporter visits the elderly person and assists them in guiding them to items and searching for them. In this case, the support is provided face-to-face with the elderly person.

[0870] Step 8:

[0871] The server records the history of assistance and the elderly person's evaluation.

[0872] Input: Support details, evaluation data

[0873] Output: Updated database

[0874] Specific operation: After the supporter completes the support, he / she reports to the server via the terminal, and the elderly person and their family evaluate the quality of the support on the terminal. The evaluation results and support history are stored in a database.

[0875] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0876] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions.

[0877] Program processing overview

[0878] 1. Request registration and emotion recognition for elderly people

[0879] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[0880] On the device: It uses generative AI to convert speech to text, while simultaneously using an emotion engine to recognize the elderly person's emotions from their speech.

[0881] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "shopping support" is needed and recognizes that the elderly person is feeling "anxiety."

[0882] Device: Obtains the elderly person's GPS location information and sends it to the server.

[0883] 2. Volunteer matching and emotion-based prioritization

[0884] Server: Based on the elderly person's location information, it lists nearby available volunteers. For example, it checks whether "Volunteer A" is nearby.

[0885] Server: Check each volunteer's skill set and available time, and select the most suitable volunteer. Confirm that "Volunteer A" is available to assist with shopping.

[0886] Server: Adjust the priority of requests based on the perceived emotion. For example, requests from elderly people who are anxious will be treated as high priority.

[0887] Server: Sends a request notification to the volunteer. For example, it may say, "Elderly B would like shopping assistance. Elderly B is currently feeling anxious."

[0888] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[0889] 3. Support implementation and history management

[0890] User (volunteer): Visits elderly people and provides shopping support.

[0891] Terminal: Report to the server that the volunteer has discontinued support.

[0892] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[0893] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[0894] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[0895] Specific examples

[0896] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[0897] 2. Device: Generative AI and emotion engine converts speech into text and emotion data to identify "shopping assistance" and "anxiety."

[0898] 3. Server: Analyzes text and emotion data and determines that shopping support is needed and is a high priority.

[0899] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[0900] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[0901] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[0902] 7. Terminal: Reports to the server that volunteer A has completed the support.

[0903] 8. Server: Records support history and ratings, and updates the trust score.

[0904] 9. Terminal: Older people and their families evaluate the quality of support.

[0905] Communication with elderly people with dementia

[0906] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[0907] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[0908] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[0909] This invention provides a system that quickly and effectively matches local volunteers with elderly people and also responds appropriately based on the user's emotions, thereby supporting the independent living of elderly people and improving the quality of emotional support.

[0910] The processing flow will be explained below.

[0911] Step 1:

[0912] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[0913] Action: The device captures audio data and begins recording.

[0914] Step 2:

[0915] On-device: Uses generative AI to convert recorded audio into text in real time.

[0916] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[0917] Step 3:

[0918] Terminal: Recognize the emotions of elderly people from their voice using an emotion engine.

[0919] How it works: Analyzes audio data and generates emotion data (e.g., "anxiety") along with text data.

[0920] Step 4:

[0921] Terminal: Sends text data and emotion data to the server.

[0922] How it works: The generated text data and emotion data are uploaded to a server via the Internet.

[0923] Step 5:

[0924] Server: Receives text data and emotion data, analyzes and identifies the request content.

[0925] How it works: It uses text analytics algorithms to identify the need for "shopping assistance" and that the senior is feeling "anxious."

[0926] Step 6:

[0927] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[0928] What it does: Uses the GPS module to capture current location information and upload it to a server.

[0929] Step 7:

[0930] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[0931] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[0932] Step 8:

[0933] Server: Check each volunteer's skill set and availability and select the best volunteer.

[0934] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[0935] Step 9:

[0936] Server: Adjust the priority of requests based on perceived sentiment.

[0937] Behavior: If the emotional data is determined to be "anxiety," it is processed as high priority.

[0938] Step 10:

[0939] Server: Sends notification of the request to selected volunteers.

[0940] What it does: Sends a push notification to the volunteer's device with the message, "Elderly B wants shopping assistance. Elderly B is currently feeling anxious."

[0941] Step 11:

[0942] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[0943] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[0944] Step 12:

[0945] User (volunteer): Visits elderly people and provides them with the necessary support.

[0946] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[0947] Step 13:

[0948] Terminal: Report to the server that the volunteer has discontinued support.

[0949] Operation: Press the support completion button and send a completion report to the server.

[0950] Step 14:

[0951] Server: Records support details and ratings in a database.

[0952] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[0953] Step 15:

[0954] Device: Elderly people and their families rate the quality of support and send the results to the server.

[0955] Action: The evaluation form is filled out on the device and uploaded to the server.

[0956] Step 16:

[0957] Server: Updates the trust database of volunteers based on the evaluation data.

[0958] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[0959] Step 17:

[0960] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[0961] What it does: Captures voice input from elderly people with dementia.

[0962] Step 18:

[0963] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[0964] What it does: Analyzes audio data and generates a conclusion that determines "eating assistance is needed."

[0965] Step 19:

[0966] Server: Sends a response message to the terminal.

[0967] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[0968] Example 2

[0969] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0970] Responding appropriately and promptly to the difficulties faced by elderly people, especially those with dementia, in their daily lives is an important issue in modern society. However, current support systems lack sufficient functionality to effectively analyze user requests and quickly match them with the most suitable volunteers. They also lack the functionality to adjust response priorities while taking into account the emotional state of the elderly. This can result in delays in actual assistance or in the elderly not receiving sufficient support.

[0971] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0972] In this invention, the server includes a means for recognizing the user's emotion from voice, a means for acquiring the elderly person's location information, and a means for sending a request notification to a volunteer, thereby enabling prompt and appropriate volunteer matching that takes into account the elderly person's emotional state.

[0973] "Voice input" refers to the act of elderly people making requests using voice through a dedicated device.

[0974] "Text conversion" is the process of analyzing audio data and converting it into text information.

[0975] "Request content" refers to the specific content of the request conveyed by the elderly person through voice input.

[0976] "Emotion recognition" is a technology that identifies the emotional state of elderly people from voice data.

[0977] "Location information" is data that identifies the current location of an elderly person using GPS functionality.

[0978] A "volunteer" is someone who provides assistance in response to requests from elderly people.

[0979] A "notification" is a message sent by the server to a volunteer to inform them of the contents of the request.

[0980] "Responding" is the act of a volunteer replying to the server about whether they are available to respond to a request.

[0981] "History" is data that records the implementation details and evaluations of past requests.

[0982] The "easy operation button" is an input device designed to be easily operated by elderly people with dementia.

[0983] "Generative artificial intelligence technology" is a technology that uses generative AI models to analyze voice data and recognize emotions.

[0984] A "trust score" is an indicator of a volunteer's trustworthiness calculated based on their past evaluation data.

[0985] This invention is a system for responding quickly and appropriately to the difficulties that elderly people face in their daily lives, with the aim of providing support that takes into account the emotional state of the elderly. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, the system also has the ability to adjust the priority of requests and notification content using emotion recognition technology.

[0986] Hardware and Software Configuration

[0987] Devices: Older adults use dedicated devices (e.g., smart speakers with voice assistants, tablets).

[0988] Analysis software: Generative AI (e.g., Google Cloud Speech-to-Text) is used to convert voice data into text. Emotion engine (e.g., Microsoft Azure Emotion API) is used to recognize emotions.

[0989] Server: Receives text and emotion data, analyzes requests, searches for volunteers, sends notifications, receives responses, and records history and ratings.

[0990] GPS function: Use a smartphone or tablet with GPS functionality to track the senior's current location.

[0991] Specific examples of program processing

[0992] 1. Request registration and emotion recognition for elderly people

[0993] - Device: The elderly person presses a button on the device and voice-inputs, "I want you to take me shopping."

[0994] - Device: Uses generative AI to convert speech into text data, while simultaneously using an emotion engine to recognize emotions. Identifies that "shopping assistance" is needed and that the elderly person is feeling "anxious."

[0995] - Server: Receives text data and emotion data and analyzes the request content.

[0996] - Device: The elderly person's GPS location information is also sent to the server.

[0997] 2. Volunteer matching and emotion-based prioritization

[0998] - Server: Based on the received location information, it creates a list of nearby volunteers and checks each volunteer's skill set and available time. For example, it checks whether "Volunteer A" is available to provide shopping assistance.

[0999] - Server: Prioritize requests based on emotion data. Requests from elderly people who are anxious are given high priority.

[1000] - Server: Sends a request notification to the volunteer. The notification reads, "Elderly B would like shopping support. Elderly B is currently feeling anxious."

[1001] - Device: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" responds that they are available.

[1002] 3. Support implementation and history management

[1003] - User (volunteer): Visits elderly people and provides shopping support.

[1004] - Terminal: Report to the server that the volunteer has ended support.

[1005] - Server: Records support details and their evaluations in a database, such as support date and time, content, volunteer information, etc.

[1006] - Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[1007] - Server: Updates the trust database of volunteers based on the evaluation results, recalculates the trust score, and displays it to other seniors.

[1008] Examples of prompt statements

[1009] An elderly person speaks into the terminal, "I want you to take me shopping."

[1010] This system allows elderly people to receive daily support with peace of mind, enables volunteers to respond quickly and appropriately, and improves the quality of support by taking into account emotional states.

[1011] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1012] Detailed flow of program processing

[1013] 1. Request registration and emotion recognition for elderly people

[1014] Step 1: Getting voice input

[1015] Device: The elderly person presses a button on the dedicated device to input voice commands, such as "I want you to take me shopping."

[1016] Specific operation: After the elderly person presses the button, the microphone is activated and captures voice data.

[1017] Step 2: Speech-to-text and emotion recognition

[1018] Terminal: The voice data is converted into text using generative AI, and emotion data is extracted using an emotion engine. The input is voice data, and the output is text data ("Please take me shopping") and emotion data ("anxiety").

[1019] Specific operation: Send voice data to a generative AI (e.g., Google Cloud Speech-to-Text) to obtain text data, and simultaneously extract emotion data using an emotion engine (e.g., Microsoft Azure Emotion API).

[1020] Step 3: Sending text and emotion data

[1021] Terminal: Sends text data and emotion data to the server. The input is text data and emotion data, and the output is the server's confirmation of receipt.

[1022] Specific operation: The device sends data to the server's API endpoint, and the server receives the data.

[1023] Step 4: Obtaining location information

[1024] Terminal: Acquires the elderly person's GPS location information and sends it to the server. The input is GPS data, and the output is sending location information to the server.

[1025] Specific operation: The device uses the built-in GPS function to obtain its current location and sends the location information to the server.

[1026] 2. Volunteer matching and emotion-based prioritization

[1027] Step 5: Create a list of neighborhood volunteers

[1028] Server: Based on the received location information, it creates a list of available volunteers in the vicinity. The input is the location information, and the output is the volunteer list.

[1029] What happens: The server searches the database and lists volunteers within the specified radius.

[1030] Step 6: Identify volunteer skillsets and availability

[1031] Server: Identifies the best volunteers by checking the skill sets and availability of each volunteer. Input is the list of volunteers, output is the best volunteers.

[1032] What happens: The server looks up the volunteer profile information and identifies volunteers who match the requirements.

[1033] Step 7: Prioritize requests based on sentiment data

[1034] Server: Adjusts the priority of requests based on the recognized emotion. The input is emotion data, and the output is the adjusted priority.

[1035] What it does: A priority adjustment algorithm analyzes sentiment data and adjusts the request's queue position.

[1036] Step 8: Sending a Request Notification

[1037] Server: Sends request notifications to volunteers. Inputs are the best volunteer and the request content, and output is the notification message.

[1038] Specific operation: The server sends a push notification to the volunteer's device, displaying the request details.

[1039] Step 9: Receiving volunteer responses

[1040] Terminal: The volunteer receives the notification and replies to the server whether they are available. The input is the volunteer's response, and the output is the response sent to the server.

[1041] What happens: The volunteer taps the notification message to send a reply message from the application.

[1042] 3. Support implementation and history management

[1043] Step 10: Providing support

[1044] User (volunteer): Visits elderly people and provides shopping support. The input is the elderly person's request, and the output is the implementation of the support.

[1045] Specific actions: Volunteers visit elderly people's homes and provide shopping assistance.

[1046] Step 11: Report support complete

[1047] Terminal: Reports to the server that the volunteer has ended support. Input is the completion status, and output is the report to the server.

[1048] Specific action: The volunteer operates the application on the device to update the status of support completion.

[1049] Step 12: Record support and evaluation

[1050] Server: Records support details and evaluations in a database. The input is support details and evaluation data, and the output is the recorded data.

[1051] Specific behavior: The server receives the evaluation result and adds a record to the database.

[1052] Step 13: Evaluate the quality of support

[1053] Terminal: Elderly people and their families evaluate the quality of support and send the results to the server. The input is the evaluation result, and the output is the data sent to the server.

[1054] Specific operation: Elderly people and their families fill out an evaluation form and send the results to the server.

[1055] Step 14: Update the Reliability Database

[1056] Server: Updates the trust database of volunteers based on the evaluation results. The input is the evaluation results, and the output is the updated trust scores.

[1057] Specific behavior: The server recalculates the trust score based on the new reputation score and updates the database.

[1058] (Application example 2)

[1059] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1060] In the daily lives of elderly people, a system that can respond quickly and appropriately is necessary, especially in emergencies or situations where they feel anxious. Conventional systems can convert voice-input requests into text and match them with supporters, but no systems exist that have security features or priority adjustment that take user emotions into account. Therefore, there is a need for a method to adjust the priority of requests based on emotions and ensure emergency responses according to specific situations.

[1061] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1062] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the request content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response according to the request, means for recognizing emotions and adjusting the priority of the request, and means having a security function for immediately notifying after detecting the situation. This makes it possible to take emotions into consideration when an elderly person makes a voice request and quickly respond to requests with high urgency.

[1063] "Means for accepting voice input" refers to a device or interface that receives voice commands issued by the senior.

[1064] "Means of converting received voice into text" refers to the technology or software that converts voice data into text data.

[1065] "Means for analyzing the textual content and identifying the request content" refers to algorithms or programs that analyze textual data and identify the elderly person's requests from its content.

[1066] "Means for searching for supporters who can provide the requested support based on the specified request content" refers to a function for searching a database for supporters who match the request.

[1067] "Means for sending a request notification to a supporter" refers to a communication technology for sending request information to a specified supporter.

[1068] "Means for accepting responses from supporters" refers to the function of receiving confirmation of receipt and replies from supporters.

[1069] "Means for recording request history and evaluation" refers to a system that stores and manages history and evaluation information regarding requests and support for older adults.

[1070] "A means for enabling elderly people with dementia to make voice requests using simple operation buttons" refers to an interface that allows elderly people with dementia to input voice commands with simple operations.

[1071] "Means for generating and sending a response in response to a request" refers to the function of generating an appropriate response based on the content of the request and sending it to the elderly person.

[1072] "Means for recognizing emotions and adjusting the priority of requests" refers to an algorithm that analyzes the emotional state of the elderly person and determines the priority of processing requests accordingly.

[1073] "Having security functions and means to immediately notify after detecting a situation" refers to the function to detect abnormal or emergency situations and immediately notify the relevant security services of that information.

[1074] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with appropriate supporters (volunteers, etc.), this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions. The system also has security functions and is equipped with a means to immediately notify users when a suspicious situation is detected.

[1075] 1. Voice Input and Emotion Recognition

[1076] Device: The elderly person speaks into a dedicated device (e.g., smartphone, robot) and says, for example, "Someone is coming into the house."

[1077] Device: Uses generative AI (e.g., Google Speech-to-Text API) to convert speech to text, while simultaneously using an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the elderly person's emotions from their speech.

[1078] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "suspicious person detection" is required and that the elderly person is feeling "fear."

[1079] 2. Matching supporters and adjusting priorities based on emotions

[1080] Server: Adjust the priority of requests based on the recognized emotion. For example, requests from elderly people who feel "fear" will be given top priority.

[1081] Server: Based on the elderly person's location information and request details, the server creates a list of nearby available supporters. For example, it checks whether "Supporter A" is nearby.

[1082] Server: Sends a request notification to the supporter. For example, it may say, "Elderly B wants to detect suspicious people. Elderly B is currently feeling scared."

[1083] Terminal: The supporter receives the notification and replies to the server whether they are available. For example, "Supporter A" responds that they are available.

[1084] 3. Support implementation and history management

[1085] User (supporter): Visits the elderly person's home and provides support. For example, checking the house for suspicious people.

[1086] Terminal: Report to the server that the supporter has ended support.

[1087] Server: Records the support content and its evaluation in a database, including the date and time of the support, the content, and information about the supporter.

[1088] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. For example, they may rate "Supporter A" five stars.

[1089] Server: Updates the database of supporter reliability based on the evaluation results. Recalculates the reliability score and displays it to other seniors.

[1090] 4. Implementation of security functions

[1091] Server: Detects suspicious situations from voice input and immediately notifies the security service.

[1092] Security Services: Review surveillance footage at the senior's home in real time and dispatch emergency services if necessary.

[1093] Specific examples

[1094] 1. An elderly person tells a robot installed in their living room, "It feels like someone has come into my house."

[1095] 2. The robot converts the voice into text and emotional data (e.g., fear), analyzes it, and determines whether suspicious person detection is necessary.

[1096] 3. The server processes the request with high priority and sends a notification to the security service.

[1097] 4. Security services will review surveillance footage and respond to emergencies as necessary.

[1098] Example of input prompt for generative AI model

[1099] Input prompt: Describe a suspicious person detection system for when an elderly person says "someone has entered the house" and then feels scared. Please be specific and include the voice analysis, emotion engine, and security notification flow.

[1100] Example output: When an elderly person tells the robot, "Someone has come into the house," a speech analysis engine (e.g., Google Speech-to-Text API) converts the speech into text, and an emotion engine (e.g., IBM Watson Tone Analyzer) detects and analyzes the elderly person's fear. The analysis information is notified to the security service with high priority, which checks surveillance camera footage and takes any necessary emergency action.

[1101] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1102] Step 1:

[1103] The device accepts voice input from the elderly person. For example, the elderly person may say, "Someone has come into the house." The input voice is converted into digital voice data via the device's microphone.

[1104] Step 2:

[1105] The speech received by the device is converted into text data using a generation AI (e.g., Google Speech-to-Text API). The input speech data is processed by a speech recognition model via the API, and text data is generated as a result. The output is the text data "Someone has entered the house."

[1106] Step 3:

[1107] The device passes the text data to an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to generate emotion data. The input text data is processed by the emotion analysis model, and the output emotion data is "fear."

[1108] Step 4:

[1109] The server receives the text data and emotion data and analyzes the request content. Using the text data "Someone entered the house" and the emotion data "fear" as input, the text analysis algorithm identifies the request content "suspicious person detected." The request content is identified as the output.

[1110] Step 5:

[1111] The server adjusts the priority of the request based on the request content and emotion data. Using the "suspicious person detected" content and the emotion data of "fear" as input, the priority adjustment algorithm gives the request the highest priority. The output is set as a "high priority" request.

[1112] Step 6:

[1113] The server searches for nearby available supporters based on the elderly person's location information and the request details. Using the elderly person's GPS location information and the request details for "suspicious person detection" as input, a database search algorithm is used to identify an appropriate supporter. The output is "Supporter A."

[1114] Step 7:

[1115] The server sends a request notification to the supporter. Using the identified supporter "Supporter A" and the request content "Suspicious person detected" as input, the request information is sent through the notification system to the supporter. The notification is delivered to the supporter as output.

[1116] Step 8:

[1117] The terminal receives a response from the supporter. Using the supporter's receipt notification as input, the supporter responds that they are available. The response "available" is obtained as output.

[1118] Step 9:

[1119] The user (supporter) visits the elderly person's home and provides support, specifically by checking the house for any suspicious individuals.

[1120] Step 10:

[1121] The device reports to the server that the supporter has ended support. The supporter's termination notification is used as input and reported to the server. The "support end" report is recorded as output.

[1122] Step 11:

[1123] The server records the support content and its evaluation in a database. The support date and time, content, supporter information, and evaluation are used as input and saved in the database. The support history is updated as output.

[1124] Step 12:

[1125] The terminal allows elderly people and their families to evaluate the quality of support and sends the results to the server. The evaluation information is used as input and sent to the server. The evaluation results are recorded in the database as output.

[1126] Step 13:

[1127] The server updates the trust database of supporters based on the evaluation results. The trust database is updated using an algorithm that uses the evaluation data as input and calculates the trust score. The updated trust score is obtained as output.

[1128] Step 14:

[1129] The server detects suspicious situations from voice input and immediately notifies the security service. Using the elderly person's voice request and emotion data as input, the server identifies suspicious situations using an anomaly detection algorithm and sends a notification to the security service. An emergency notification is sent as output.

[1130] Step 15:

[1131] Security services receive the notification, review the surveillance footage at the senior's home, and dispatch emergency services if necessary. The notification information is used as input to review the situation on the surveillance video system. The output is the initiation of an emergency response.

[1132] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1133] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1134] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1135] [Third embodiment]

[1136] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1137] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1138] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1139] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1140] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1141] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1142] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1143] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1144] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1145] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1146] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1147] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1148] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. This system has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteers, thereby resolving the issues that elderly people face.

[1149] Program processing overview

[1150] 1. Elderly request registration

[1151] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[1152] Device: Uses generative AI to convert speech to text. The speech data is converted into text data: "I want you to take me shopping."

[1153] Server: Receives the text data and analyzes the request. As a result of the analysis, it determines that "shopping support" is required.

[1154] Device: Obtains the elderly person's GPS location information and sends it to the server.

[1155] 2. Volunteer matching

[1156] Server: Based on the elderly person's location information, create a list of nearby volunteers. For example, confirm that "Volunteer A" is nearby.

[1157] Server: Check the skill sets and available time of the volunteers and select the most suitable volunteer. Confirm that "Volunteer A" is available to provide shopping support.

[1158] Server: Sends a request notification to the volunteer. It notifies the volunteer that "Elderly B wants shopping assistance."

[1159] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[1160] 3. Support implementation and history management

[1161] User (volunteer): Visits elderly people and provides shopping support.

[1162] Terminal: Reports to the server that the volunteer has completed their support.

[1163] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[1164] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[1165] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[1166] 4. Communication with elderly people with dementia

[1167] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[1168] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[1169] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[1170] Specific examples

[1171] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[1172] 2. Terminal: The generation AI converts the speech into text and sends it to the server.

[1173] 3. Server: Analyzes the text and determines if shopping assistance is required.

[1174] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[1175] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[1176] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[1177] 7. Terminal: Reports to the server that volunteer A has completed the support.

[1178] 8. Server: Records support history and ratings, and updates the trust score.

[1179] 9. Terminal: Older people and their families evaluate the quality of support.

[1180] In this way, the present invention makes it possible to quickly and effectively match local volunteers with elderly people, thereby supporting the elderly in living independently.

[1181] The processing flow will be explained below.

[1182] Step 1:

[1183] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[1184] Action: The device captures audio data and begins recording.

[1185] Step 2:

[1186] On-device: Uses generative AI to convert recorded audio into text in real time.

[1187] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[1188] Step 3:

[1189] Terminal: Sends text data to the server.

[1190] What it does: Uploads text data to a server over the internet.

[1191] Step 4:

[1192] Server: Receives and analyzes the text data to identify the request content.

[1193] How it works: Uses text analytics algorithms to identify that "shopping help" is needed.

[1194] Step 5:

[1195] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[1196] What it does: Uses the GPS module to capture current location information and upload it to a server.

[1197] Step 6:

[1198] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[1199] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[1200] Step 7:

[1201] Server: Check each volunteer's skill set and availability and select the best volunteer.

[1202] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[1203] Step 8:

[1204] Server: Sends notification of the request to selected volunteers.

[1205] What it does: Sends a push notification to the volunteer's device, displaying the request details.

[1206] Step 9:

[1207] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[1208] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[1209] Step 10:

[1210] User (volunteer): Visits elderly people and provides them with the necessary support.

[1211] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[1212] Step 11:

[1213] Terminal: Report to the server that the volunteer has discontinued support.

[1214] Operation: Press the support completion button and send a completion report to the server.

[1215] Step 12:

[1216] Server: Records support details and ratings in a database.

[1217] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[1218] Step 13:

[1219] Terminal: The elderly person or their family member enters their evaluation of the support and sends it to the server.

[1220] Action: The evaluation form is filled out on the device and uploaded to the server.

[1221] Step 14:

[1222] Server: Updates the trust database of volunteers based on the evaluation data.

[1223] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[1224] Step 15:

[1225] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[1226] What it does: Captures voice input from elderly people with dementia.

[1227] Step 16:

[1228] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[1229] What it does: Analyzes audio data and generates the conclusion "Needs meal assistance."

[1230] Step 17:

[1231] Server: Sends a response message to the terminal.

[1232] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[1233] Example 1

[1234] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1235] There is a problem that it is difficult for elderly people to receive prompt and appropriate support from local volunteers when they face difficulties in their daily lives. Furthermore, in the case of elderly people with dementia, it becomes even more difficult to provide appropriate support due to the increased difficulty of responding to emergencies and communicating. Therefore, a system that provides an effective solution to these issues is needed.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1237] In this invention, the server includes means for the elderly to input voice, means for converting the voice input into text data, means for analyzing the text data to identify the request content, means for searching for an appropriate volunteer based on the identified request content, means for sending a request notification to the volunteer, means for accepting responses from the volunteer, means for recording the request history and evaluation, means for allowing the elderly to make voice requests using simple operation buttons, and means for generating and sending a response in accordance with the request. This enables the elderly to receive prompt and appropriate support from local volunteers for difficulties they face in their daily lives.

[1238] "Elderly" refers to individuals in an age group who require some assistance in daily living.

[1239] "Voice input" refers to a method in which elderly people speak into a dedicated device to capture voice information as digital data.

[1240] "Text data" refers to digital data converted from voice input into text information.

[1241] "Request content" refers to the specific request for assistance or support expressed by the elderly person through voice input.

[1242] "Volunteers" refer to local supporters whose purpose is to provide assistance to the elderly.

[1243] "Notification" refers to a message sent from the server to the volunteer informing them of the request.

[1244] "Responding" refers to the act of a volunteer responding to the server after receiving a request notification to let the server know whether or not they can respond.

[1245] "History" refers to the digital data stored record of past requests and their responses.

[1246] "Evaluation" refers to feedback given by seniors and their families regarding the volunteer's response to their request.

[1247] "Generative AI model" refers to the machine learning techniques used to convert voice data into text data and analyze that text data.

[1248] "Trustworthiness score" refers to an index that quantifies the reliability of a volunteer based on past evaluation data.

[1249] "Easy-to-operate buttons" refer to input devices designed to be easy to use even for elderly people who are unfamiliar with operating them.

[1250] "Response message" refers to the reply automatically generated by the server in response to a request entered by the elderly person.

[1251] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. The system of the present invention has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteer. This system allows elderly people to more easily solve problems in their daily lives.

[1252] Hardware and Software Configuration

[1253] The implementation of this system uses the following main hardware and software:

[1254] Hardware

[1255] 1. Dedicated device: A device equipped with a microphone and buttons for voice input by the elderly. This device is equipped with a microphone to capture voice data and a GPS module (e.g., U-Blox NEO-6M GPS module) to obtain location information.

[1256] 2. Smartphone: A device that allows volunteers to receive and respond to notifications. A dedicated application (e.g., ActionBound app) is installed.

[1257] software

[1258] 1. Generative AI model: This is a machine learning technology that converts voice data into text and then analyzes that text. For voice-to-text conversion, we use the Google Cloud Speech-to-Text API.

[1259] 2. Natural language processing engine: Use a natural language processing engine such as IBM Watson NLU to analyze the request content from text data.

[1260] 3. Server system: This is the central system that receives requests from seniors, analyzes them, and notifies and matches them with volunteers.

[1261] Program processing

[1262] The program of this system operates according to the following processing flow.

[1263] Voice input for the elderly

[1264] The device starts when the elderly person presses a button on the dedicated device to input voice commands, for example, "Please take me shopping."

[1265] Speech to text

[1266] When the device receives voice input, it converts the data into text using a generative AI model (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[1267] Text data analysis

[1268] The server receives the text data and analyzes the request using a natural language processing engine (IBM Watson NLU). As a result of the analysis, it determines that "shopping support" is required.

[1269] Acquiring location information of elderly people

[1270] The device acquires the elderly person's GPS location information and sends it to the server, where the GPS module installed in the device measures their current location.

[1271] Volunteer Selection and Notification

[1272] The server uses the elderly person's location information to create a list of nearby volunteers, checks their skill sets and availability, and selects the most suitable volunteer. For example, it verifies that "Volunteer A" is nearby and available to assist with shopping. The server then sends a request notification to the volunteer.

[1273] Volunteer Response

[1274] The device helps the volunteer receive the notification and respond to the server whether they are available. The volunteer responds "I'm available" using the ActionBound app on their smartphone.

[1275] Support and history management

[1276] The user (volunteer) visits the elderly person and provides shopping support. The terminal reports to the server that the volunteer has completed the support. The server records the date, time, and content of the support, as well as the volunteer information, in a database. It also recalculates the reliability score and provides an interface for the elderly person and their family to evaluate the support.

[1277] Support for elderly people with dementia

[1278] The device allows elderly people with dementia to make voice requests using simple operation buttons, such as saying "I'm hungry." The server analyzes the voice in real time and generates an appropriate response message. The device then displays a message such as "We will contact a nearby volunteer."

[1279] Specific examples

[1280] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[1281] 2. Device: The generating AI (Google Cloud Speech-to-Text) converts the speech into text and sends it to the server.

[1282] 3. Server: Analyzes the text using IBM Watson NLU and determines if shopping assistance is required.

[1283] 4. Terminal: Sends the elderly person's location information (U-Blox NEO-6M GPS module) to the server.

[1284] 5. Server: Using the elderly person's location information, send a notification to nearby "Volunteer A."

[1285] 6. Device: Volunteer A receives a notification in the ActionBound app and replies that they are available.

[1286] 7. User (Volunteer A): A volunteer visits an elderly person and provides shopping assistance.

[1287] 8. Device: Volunteer A uses the ActionBound app to report completion of support to the server.

[1288] 9. Server: Records support history and ratings in a database and updates the trust score.

[1289] 10. Device: Elderly people and their families evaluate the quality of support through a smartphone rating app.

[1290] In this way, this system can effectively match local volunteers with elderly people and support their independent living.

[1291] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1292] Step 1:

[1293] Device: The elderly person presses a button on the dedicated device to input voice. For example, the elderly person may say, "I want you to take me shopping."

[1294] Input: Voice input for the elderly

[1295] Output: Audio data

[1296] Step 2:

[1297] Device: The voice input is converted into text using a generative AI (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[1298] Input: Audio data

[1299] Data processing: converting voice to text

[1300] Output: Text data

[1301] Step 3:

[1302] Terminal: The converted text data is sent to the server via an HTTP request.

[1303] Input: Text data

[1304] Output: Request sent to server

[1305] Step 4:

[1306] Server: Analyzes the received text data using IBM Watson NLU to identify the request. From the analysis results, it determines that "shopping support" is required.

[1307] Input: Text data

[1308] Data Calculation: Analyzing request content using natural language processing

[1309] Output: Identifying the request

[1310] Step 5:

[1311] Device: Acquires the elderly person's GPS location information and sends it to the server. The device is equipped with a GPS module (U-Blox NEO-6M GPS module).

[1312] Input: Location request

[1313] Data processing: Measure your current location and convert it into data

[1314] Output: Location data

[1315] Step 6:

[1316] Server: Creates a list of nearby volunteers based on the elderly person's location information.

[1317] Input: Location data

[1318] Data calculation: Generating a volunteer list through database searches

[1319] Output: Volunteer list

[1320] Step 7:

[1321] Server: Checks the skill sets and availability of volunteers and selects the most suitable volunteer. For example, identifies that "Volunteer A" is available to assist with shopping.

[1322] Input: Volunteer List

[1323] Data crunching: Checking volunteer skill sets and availability

[1324] Output: Selected volunteers

[1325] Step 8:

[1326] Server: Sends a request notification to the selected volunteer. Sends a notification saying, "Elderly B wants shopping assistance."

[1327] Input: Selected volunteer, request details

[1328] Output: Notification to volunteer

[1329] Step 9:

[1330] Terminal: After receiving the notification, the volunteer replies to the server whether they are available or not. Specifically, the volunteer uses a dedicated smartphone application (ActionBound app) to select "available" or "unavailable" and respond.

[1331] Input: Volunteer response

[1332] Output: Sending response data

[1333] Step 10:

[1334] User (volunteer): A volunteer visits an elderly person and provides designated support, such as taking them shopping.

[1335] Input: Elderly Request

[1336] Output: Providing support

[1337] Step 11:

[1338] Device: The volunteer reports to the server that they have completed their support. The volunteer uses the ActionBound app on their smartphone to press the "Support Completed" button.

[1339] Input: Support Completion Information

[1340] Output: Report to server

[1341] Step 12:

[1342] Server: Records support history and ratings in a database and updates the reliability score. Records the date and time of support, content, volunteer information, etc., and recalculates the reliability score.

[1343] Input: Support completion information, evaluation data

[1344] Data calculation: Recording support history and recalculating confidence scores

[1345] Output: Updated database

[1346] Step 13:

[1347] Device: Elderly people and their families use a smartphone rating app to rate the quality of support and send the results to the server.

[1348] Input: Evaluation information

[1349] Output: Send to server

[1350] Step 14:

[1351] Server: When an elderly person with dementia makes a voice request using the simple operation buttons, the voice is analyzed in real time and a response message is generated.

[1352] Input: Voice request

[1353] Data calculation: Real-time voice analysis, response message generation

[1354] Output: Response message

[1355] Step 15:

[1356] Server: Sends a response message to the terminal and displays it on the terminal. It displays a message such as "We will contact a nearby volunteer."

[1357] Input: Response message

[1358] Output: Display to terminal

[1359] (Application example 1)

[1360] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1361] The problem to be solved by this invention is to provide a system that provides prompt and appropriate assistance to seniors to eliminate the difficulties they face when searching for items in brick-and-mortar stores. In particular, this system allows seniors with dementia and those who require mobility in their daily lives to easily receive the necessary support through voice requests. Another problem is to ensure the reliability of the supporter and achieve effective support.

[1362] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1363] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the requested content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response in response to the request, and means for matching requests from elderly people who need item search or guidance with supporters in a physical store based on location information, thereby enabling items needed by elderly people to be searched for and guided to in a physical store quickly.

[1364] "Means for accepting voice input" refers to a device or apparatus that allows the elderly person to input requests by voice.

[1365] "Means for converting received voice into text" refers to technology or a system that converts voice input into text data.

[1366] "Means for analyzing textual content to identify the request content" refers to technologies and systems that understand textual data and grasp specific needs and requests.

[1367] A "means for searching for supporters who can provide the requested support" is a system for searching for people or organizations that can provide appropriate support based on the specified request content.

[1368] The "means for sending a request notification to the supporter" refers to a communication technology or system for informing the supporter of the request content.

[1369] "Means for receiving responses from supporters" refers to technologies and systems for receiving responses from supporters, such as whether they can respond or not.

[1370] The "means for recording request history and evaluation" is a system for storing the support provided and its evaluation in a database.

[1371] "Means that allow elderly people with dementia to make voice requests using simple operation buttons" refers to devices and technologies that provide an interface that can be easily operated by elderly people with dementia.

[1372] The "means for generating and transmitting a response in response to a request" is a system for generating an appropriate response to the request content and conveying it to the elderly person.

[1373] "Means for matching elderly people who need to search for items or receive guidance with helpers in physical stores based on location information" refers to technology or a system that uses the elderly person's location information to find helpers in physical stores who can provide appropriate assistance.

[1374] This invention is a system that solves the difficulties that elderly people face when searching for items in physical stores and provides them with prompt and appropriate assistance. This system is configured as follows.

[1375] First, the elderly person uses a device such as smart glasses or a smartphone to input voice commands. This device is equipped with a means to accept voice commands. For example, the elderly person can input a request by saying, "Please help me find milk."

[1376] The device uses generative AI technology to convert speech into text data, which then activates a means of converting received speech into text. OpenAI's GPT-3.5, for example, is used as the generative AI model.

[1377] Next, a means for analyzing the textual content and identifying the content of the request operates on the server. This analysis means identifies that a "product guide" is required from the text data "Please help me find milk."

[1378] After the request content is specified, a means for searching for an assistant who can provide the requested support based on the specified request content operates on the server. This means matches the optimal assistant based on the location information, skill set, available time, etc. of the assistant in the physical store. For example, an assistant who is nearby is selected.

[1379] The server then runs a mechanism to notify the supporter of the request, and the supporter then responds to the server using a mechanism to respond to the notification, indicating whether they are able to respond. For example, notifications are sent using Firebase Cloud Messaging (FCM).

[1380] If the assistant is available, they visit the elderly person and assist them with product information and search. Once the assistance is complete, a system runs on the server to record the request history and evaluation. This records the assistance details and evaluation in a database (e.g., MySQL, Amazon RDS).

[1381] To accommodate elderly people with dementia, the terminals are equipped with easy-to-operate buttons and interfaces, allowing elderly people with dementia to make voice requests by simply operating buttons.

[1382] A means for generating and transmitting a response according to the request operates on the server. This means creates an appropriate response to the request content and displays it on the terminal. For example, a message such as "We will contact a nearby supporter" is displayed.

[1383] For example, an elderly person might say to the smart glasses, "Please help me find some milk." This speech is converted to text using a generative AI model, and the request is analyzed. An appropriate helper in the physical store is selected and notified. The helper visits the elderly person, provides them with the item, and their evaluation after completing the assistance is recorded in a database.

[1384] Example prompt sentence:

[1385] "An elderly person has made a voice request using smart glasses. Please analyze the following request to identify the appropriate action:

[1386] Voice Request: "Help me find some milk."

[1387] Analysis result: The request for this article is identified as "Product Information."

[1388] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1389] Step 1:

[1390] Elderly people input information by voice into a device (smart glasses or smartphone).

[1391] Input: Senior's voice request (e.g., "Please help me find milk.")

[1392] Output: Audio data

[1393] Specific operation: The device accepts voice input through the microphone and stores it as digital voice data.

[1394] Step 2:

[1395] The device converts the received voice data into text data using generative AI technology.

[1396] Input: Audio data

[1397] Output: Text data (e.g. "Please help me find milk")

[1398] How it works: The device uses a speech recognition library (e.g., Google Speech-to-Text API) to convert voice data into text, using a generative AI model (e.g., OpenAI GPT-3.5).

[1399] Step 3:

[1400] The server analyzes the text data and identifies the request content.

[1401] Input: Text data

[1402] Output: Request details (e.g. "Product information required")

[1403] How it works: The server uses natural language processing technology to analyze the text data and identify what the elderly person wants. This process uses a generative AI model.

[1404] Step 4:

[1405] The server obtains the elderly person's location information and searches for a supporter.

[1406] Input: Request details, location information of the elderly person

[1407] Output: List of supporters

[1408] Specific operation: Based on the GPS location information sent from the device, the server searches the supporter database and lists supporters who meet the requirements, using Google Maps API and iBeacon technology.

[1409] Step 5:

[1410] The server sends a notification of the request to the identified supporter.

[1411] Input: List of supporters, request details

[1412] Output: Notification message

[1413] Specific operation: The server sends a notification message to nearby supporters using Firebase Cloud Messaging (FCM).

[1414] Step 6:

[1415] The supporter receives the notification and responds to the server whether or not they can respond.

[1416] Input: Notification message

[1417] Output: Response regarding availability

[1418] Specific operation: The supporter checks the notification on a device such as a smartphone or tablet, selects whether they can respond, and responds to the server.

[1419] Step 7:

[1420] The corresponding support person provides assistance to the elderly person.

[1421] Input: Elderly person's location information, request details

[1422] Output: The results of the assistance provided

[1423] Specific actions: The supporter visits the elderly person and assists them in guiding them to items and searching for them. In this case, the support is provided face-to-face with the elderly person.

[1424] Step 8:

[1425] The server records the history of assistance and the elderly person's evaluation.

[1426] Input: Support details, evaluation data

[1427] Output: Updated database

[1428] Specific operation: After the supporter completes the support, he / she reports to the server via the terminal, and the elderly person and their family evaluate the quality of the support on the terminal. The evaluation results and support history are stored in a database.

[1429] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1430] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions.

[1431] Program processing overview

[1432] 1. Request registration and emotion recognition for elderly people

[1433] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[1434] On the device: It uses generative AI to convert speech to text, while simultaneously using an emotion engine to recognize the elderly person's emotions from their speech.

[1435] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "shopping support" is needed and recognizes that the elderly person is feeling "anxiety."

[1436] Device: Obtains the elderly person's GPS location information and sends it to the server.

[1437] 2. Volunteer matching and emotion-based prioritization

[1438] Server: Based on the elderly person's location information, it lists nearby available volunteers. For example, it checks whether "Volunteer A" is nearby.

[1439] Server: Check each volunteer's skill set and available time, and select the most suitable volunteer. Confirm that "Volunteer A" is available to assist with shopping.

[1440] Server: Adjust the priority of requests based on the perceived emotion. For example, requests from elderly people who are anxious will be treated as high priority.

[1441] Server: Sends a request notification to the volunteer. For example, it may say, "Elderly B would like shopping assistance. Elderly B is currently feeling anxious."

[1442] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[1443] 3. Support implementation and history management

[1444] User (volunteer): Visits elderly people and provides shopping support.

[1445] Terminal: Report to the server that the volunteer has discontinued support.

[1446] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[1447] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[1448] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[1449] Specific examples

[1450] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[1451] 2. Device: Generative AI and emotion engine converts speech into text and emotion data to identify "shopping assistance" and "anxiety."

[1452] 3. Server: Analyzes text and emotion data and determines that shopping support is needed and is a high priority.

[1453] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[1454] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[1455] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[1456] 7. Terminal: Reports to the server that volunteer A has completed the support.

[1457] 8. Server: Records support history and ratings, and updates the trust score.

[1458] 9. Terminal: Older people and their families evaluate the quality of support.

[1459] Communication with elderly people with dementia

[1460] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[1461] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[1462] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[1463] This invention provides a system that quickly and effectively matches local volunteers with elderly people and also responds appropriately based on the user's emotions, thereby supporting the independent living of elderly people and improving the quality of emotional support.

[1464] The processing flow will be explained below.

[1465] Step 1:

[1466] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[1467] Action: The device captures audio data and begins recording.

[1468] Step 2:

[1469] On-device: Uses generative AI to convert recorded audio into text in real time.

[1470] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[1471] Step 3:

[1472] Terminal: Recognize the emotions of elderly people from their voice using an emotion engine.

[1473] How it works: Analyzes audio data and generates emotion data (e.g., "anxiety") along with text data.

[1474] Step 4:

[1475] Terminal: Sends text data and emotion data to the server.

[1476] How it works: The generated text data and emotion data are uploaded to a server via the Internet.

[1477] Step 5:

[1478] Server: Receives text data and emotion data, analyzes and identifies the request content.

[1479] How it works: It uses text analytics algorithms to identify the need for "shopping assistance" and that the senior is feeling "anxious."

[1480] Step 6:

[1481] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[1482] What it does: Uses the GPS module to capture current location information and upload it to a server.

[1483] Step 7:

[1484] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[1485] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[1486] Step 8:

[1487] Server: Check each volunteer's skill set and availability and select the best volunteer.

[1488] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[1489] Step 9:

[1490] Server: Adjust the priority of requests based on perceived sentiment.

[1491] Behavior: If the emotional data is determined to be "anxiety," it is processed as high priority.

[1492] Step 10:

[1493] Server: Sends notification of the request to selected volunteers.

[1494] What it does: Sends a push notification to the volunteer's device with the message, "Elderly B wants shopping assistance. Elderly B is currently feeling anxious."

[1495] Step 11:

[1496] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[1497] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[1498] Step 12:

[1499] User (volunteer): Visits elderly people and provides them with the necessary support.

[1500] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[1501] Step 13:

[1502] Terminal: Report to the server that the volunteer has discontinued support.

[1503] Operation: Press the support completion button and send a completion report to the server.

[1504] Step 14:

[1505] Server: Records support details and ratings in a database.

[1506] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[1507] Step 15:

[1508] Device: Elderly people and their families rate the quality of support and send the results to the server.

[1509] Action: The evaluation form is filled out on the device and uploaded to the server.

[1510] Step 16:

[1511] Server: Updates the trust database of volunteers based on the evaluation data.

[1512] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[1513] Step 17:

[1514] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[1515] What it does: Captures voice input from elderly people with dementia.

[1516] Step 18:

[1517] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[1518] What it does: Analyzes audio data and generates a conclusion that determines "eating assistance is needed."

[1519] Step 19:

[1520] Server: Sends a response message to the terminal.

[1521] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[1522] Example 2

[1523] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1524] Responding appropriately and promptly to the difficulties faced by elderly people, especially those with dementia, in their daily lives is an important issue in modern society. However, current support systems lack sufficient functionality to effectively analyze user requests and quickly match them with the most suitable volunteers. They also lack the functionality to adjust response priorities while taking into account the emotional state of the elderly. This can result in delays in actual assistance or in the elderly not receiving sufficient support.

[1525] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1526] In this invention, the server includes a means for recognizing the user's emotion from voice, a means for acquiring the elderly person's location information, and a means for sending a request notification to a volunteer, thereby enabling prompt and appropriate volunteer matching that takes into account the elderly person's emotional state.

[1527] "Voice input" refers to the act of elderly people making requests using voice through a dedicated device.

[1528] "Text conversion" is the process of analyzing audio data and converting it into text information.

[1529] "Request content" refers to the specific content of the request conveyed by the elderly person through voice input.

[1530] "Emotion recognition" is a technology that identifies the emotional state of elderly people from voice data.

[1531] "Location information" is data that identifies the current location of an elderly person using GPS functionality.

[1532] A "volunteer" is someone who provides assistance in response to requests from elderly people.

[1533] A "notification" is a message sent by the server to a volunteer to inform them of the contents of the request.

[1534] "Responding" is the act of a volunteer replying to the server about whether they are available to respond to a request.

[1535] "History" is data that records the implementation details and evaluations of past requests.

[1536] The "easy operation button" is an input device designed to be easily operated by elderly people with dementia.

[1537] "Generative artificial intelligence technology" is a technology that uses generative AI models to analyze voice data and recognize emotions.

[1538] A "trust score" is an indicator of a volunteer's trustworthiness calculated based on their past evaluation data.

[1539] This invention is a system for responding quickly and appropriately to the difficulties that elderly people face in their daily lives, with the aim of providing support that takes into account the emotional state of the elderly. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, the system also has the ability to adjust the priority of requests and notification content using emotion recognition technology.

[1540] Hardware and Software Configuration

[1541] Devices: Older adults use dedicated devices (e.g., smart speakers with voice assistants, tablets).

[1542] Analysis software: Generative AI (e.g., Google Cloud Speech-to-Text) is used to convert voice data into text. Emotion engine (e.g., Microsoft Azure Emotion API) is used to recognize emotions.

[1543] Server: Receives text and emotion data, analyzes requests, searches for volunteers, sends notifications, receives responses, and records history and ratings.

[1544] GPS function: Use a smartphone or tablet with GPS functionality to track the senior's current location.

[1545] Specific examples of program processing

[1546] 1. Request registration and emotion recognition for elderly people

[1547] - Device: The elderly person presses a button on the device and voice-inputs, "I want you to take me shopping."

[1548] - Device: Uses generative AI to convert speech into text data, while simultaneously using an emotion engine to recognize emotions. Identifies that "shopping assistance" is needed and that the elderly person is feeling "anxious."

[1549] - Server: Receives text data and emotion data and analyzes the request content.

[1550] - Device: The elderly person's GPS location information is also sent to the server.

[1551] 2. Volunteer matching and emotion-based prioritization

[1552] - Server: Based on the received location information, it creates a list of nearby volunteers and checks each volunteer's skill set and available time. For example, it checks whether "Volunteer A" is available to provide shopping assistance.

[1553] - Server: Prioritize requests based on emotion data. Requests from elderly people who are anxious are given high priority.

[1554] - Server: Sends a request notification to the volunteer. The notification reads, "Elderly B would like shopping support. Elderly B is currently feeling anxious."

[1555] - Device: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" responds that they are available.

[1556] 3. Support implementation and history management

[1557] - User (volunteer): Visits elderly people and provides shopping support.

[1558] - Terminal: Report to the server that the volunteer has ended support.

[1559] - Server: Records support details and their evaluations in a database, such as support date and time, content, volunteer information, etc.

[1560] - Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[1561] - Server: Updates the trust database of volunteers based on the evaluation results, recalculates the trust score, and displays it to other seniors.

[1562] Examples of prompt statements

[1563] An elderly person speaks into the terminal, "I want you to take me shopping."

[1564] This system allows elderly people to receive daily support with peace of mind, enables volunteers to respond quickly and appropriately, and improves the quality of support by taking into account emotional states.

[1565] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1566] Detailed flow of program processing

[1567] 1. Request registration and emotion recognition for elderly people

[1568] Step 1: Getting voice input

[1569] Device: The elderly person presses a button on the dedicated device to input voice commands, such as "I want you to take me shopping."

[1570] Specific operation: After the elderly person presses the button, the microphone is activated and captures voice data.

[1571] Step 2: Speech-to-text and emotion recognition

[1572] Terminal: The voice data is converted into text using generative AI, and emotion data is extracted using an emotion engine. The input is voice data, and the output is text data ("Please take me shopping") and emotion data ("anxiety").

[1573] Specific operation: Send voice data to a generative AI (e.g., Google Cloud Speech-to-Text) to obtain text data, and simultaneously extract emotion data using an emotion engine (e.g., Microsoft Azure Emotion API).

[1574] Step 3: Sending text and emotion data

[1575] Terminal: Sends text data and emotion data to the server. The input is text data and emotion data, and the output is the server's confirmation of receipt.

[1576] Specific operation: The device sends data to the server's API endpoint, and the server receives the data.

[1577] Step 4: Obtaining location information

[1578] Terminal: Acquires the elderly person's GPS location information and sends it to the server. The input is GPS data, and the output is sending location information to the server.

[1579] Specific operation: The device uses the built-in GPS function to obtain its current location and sends the location information to the server.

[1580] 2. Volunteer matching and emotion-based prioritization

[1581] Step 5: Create a list of neighborhood volunteers

[1582] Server: Based on the received location information, it creates a list of available volunteers in the vicinity. The input is the location information, and the output is the volunteer list.

[1583] What happens: The server searches the database and lists volunteers within the specified radius.

[1584] Step 6: Identify volunteer skillsets and availability

[1585] Server: Identifies the best volunteers by checking the skill sets and availability of each volunteer. Input is the list of volunteers, output is the best volunteers.

[1586] What happens: The server looks up the volunteer profile information and identifies volunteers who match the requirements.

[1587] Step 7: Prioritize requests based on sentiment data

[1588] Server: Adjusts the priority of requests based on the recognized emotion. The input is emotion data, and the output is the adjusted priority.

[1589] What it does: A priority adjustment algorithm analyzes sentiment data and adjusts the request's queue position.

[1590] Step 8: Sending a Request Notification

[1591] Server: Sends request notifications to volunteers. Inputs are the best volunteer and the request content, and output is the notification message.

[1592] Specific operation: The server sends a push notification to the volunteer's device, displaying the request details.

[1593] Step 9: Receiving volunteer responses

[1594] Terminal: The volunteer receives the notification and replies to the server whether they are available. The input is the volunteer's response, and the output is the response sent to the server.

[1595] What happens: The volunteer taps the notification message to send a reply message from the application.

[1596] 3. Support implementation and history management

[1597] Step 10: Providing support

[1598] User (volunteer): Visits elderly people and provides shopping support. The input is the elderly person's request, and the output is the implementation of the support.

[1599] Specific actions: Volunteers visit elderly people's homes and provide shopping assistance.

[1600] Step 11: Report support complete

[1601] Terminal: Reports to the server that the volunteer has ended support. Input is the completion status, and output is the report to the server.

[1602] Specific action: The volunteer operates the application on the device to update the status of support completion.

[1603] Step 12: Record support and evaluation

[1604] Server: Records support details and evaluations in a database. The input is support details and evaluation data, and the output is the recorded data.

[1605] Specific behavior: The server receives the evaluation result and adds a record to the database.

[1606] Step 13: Evaluate the quality of support

[1607] Terminal: Elderly people and their families evaluate the quality of support and send the results to the server. The input is the evaluation result, and the output is the data sent to the server.

[1608] Specific operation: Elderly people and their families fill out an evaluation form and send the results to the server.

[1609] Step 14: Update the Reliability Database

[1610] Server: Updates the trust database of volunteers based on the evaluation results. The input is the evaluation results, and the output is the updated trust scores.

[1611] Specific behavior: The server recalculates the trust score based on the new reputation score and updates the database.

[1612] (Application example 2)

[1613] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1614] In the daily lives of elderly people, a system that can respond quickly and appropriately is necessary, especially in emergencies or situations where they feel anxious. Conventional systems can convert voice-input requests into text and match them with supporters, but no systems exist that have security features or priority adjustment that take user emotions into account. Therefore, there is a need for a method to adjust the priority of requests based on emotions and ensure emergency responses according to specific situations.

[1615] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1616] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the request content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response according to the request, means for recognizing emotions and adjusting the priority of the request, and means having a security function for immediately notifying after detecting the situation. This makes it possible to take emotions into consideration when an elderly person makes a voice request and quickly respond to requests with high urgency.

[1617] "Means for accepting voice input" refers to a device or interface that receives voice commands issued by the senior.

[1618] "Means of converting received voice into text" refers to the technology or software that converts voice data into text data.

[1619] "Means for analyzing the textual content and identifying the request content" refers to algorithms or programs that analyze textual data and identify the elderly person's requests from its content.

[1620] "Means for searching for supporters who can provide the requested support based on the specified request content" refers to a function for searching a database for supporters who match the request.

[1621] "Means for sending a request notification to a supporter" refers to a communication technology for sending request information to a specified supporter.

[1622] "Means for accepting responses from supporters" refers to the function of receiving confirmation of receipt and replies from supporters.

[1623] "Means for recording request history and evaluation" refers to a system that stores and manages history and evaluation information regarding requests and support for older adults.

[1624] "A means for enabling elderly people with dementia to make voice requests using simple operation buttons" refers to an interface that allows elderly people with dementia to input voice commands with simple operations.

[1625] "Means for generating and sending a response in response to a request" refers to the function of generating an appropriate response based on the content of the request and sending it to the elderly person.

[1626] "Means for recognizing emotions and adjusting the priority of requests" refers to an algorithm that analyzes the emotional state of the elderly person and determines the priority of processing requests accordingly.

[1627] "Having security functions and means to immediately notify after detecting a situation" refers to the function to detect abnormal or emergency situations and immediately notify the relevant security services of that information.

[1628] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with appropriate supporters (volunteers, etc.), this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions. The system also has security functions and is equipped with a means to immediately notify users when a suspicious situation is detected.

[1629] 1. Voice Input and Emotion Recognition

[1630] Device: The elderly person speaks into a dedicated device (e.g., smartphone, robot) and says, for example, "Someone is coming into the house."

[1631] Device: Uses generative AI (e.g., Google Speech-to-Text API) to convert speech to text, while simultaneously using an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the elderly person's emotions from their speech.

[1632] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "suspicious person detection" is required and that the elderly person is feeling "fear."

[1633] 2. Matching supporters and adjusting priorities based on emotions

[1634] Server: Adjust the priority of requests based on the recognized emotion. For example, requests from elderly people who feel "fear" will be given top priority.

[1635] Server: Based on the elderly person's location information and request details, the server creates a list of nearby available supporters. For example, it checks whether "Supporter A" is nearby.

[1636] Server: Sends a request notification to the supporter. For example, it may say, "Elderly B wants to detect suspicious people. Elderly B is currently feeling scared."

[1637] Terminal: The supporter receives the notification and replies to the server whether they are available. For example, "Supporter A" responds that they are available.

[1638] 3. Support implementation and history management

[1639] User (supporter): Visits the elderly person's home and provides support. For example, checking the house for suspicious people.

[1640] Terminal: Report to the server that the supporter has ended support.

[1641] Server: Records the support content and its evaluation in a database, including the date and time of the support, the content, and information about the supporter.

[1642] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. For example, they may rate "Supporter A" five stars.

[1643] Server: Updates the database of supporter reliability based on the evaluation results. Recalculates the reliability score and displays it to other seniors.

[1644] 4. Implementation of security functions

[1645] Server: Detects suspicious situations from voice input and immediately notifies the security service.

[1646] Security Services: Review surveillance footage at the senior's home in real time and dispatch emergency services if necessary.

[1647] Specific examples

[1648] 1. An elderly person tells a robot installed in their living room, "It feels like someone has come into my house."

[1649] 2. The robot converts the voice into text and emotional data (e.g., fear), analyzes it, and determines whether suspicious person detection is necessary.

[1650] 3. The server processes the request with high priority and sends a notification to the security service.

[1651] 4. Security services will review surveillance footage and respond to emergencies as necessary.

[1652] Example of input prompt for generative AI model

[1653] Input prompt: Describe a suspicious person detection system for when an elderly person says "someone has entered the house" and then feels scared. Please be specific and include the voice analysis, emotion engine, and security notification flow.

[1654] Example output: When an elderly person tells the robot, "Someone has come into the house," a speech analysis engine (e.g., Google Speech-to-Text API) converts the speech into text, and an emotion engine (e.g., IBM Watson Tone Analyzer) detects and analyzes the elderly person's fear. The analysis information is notified to the security service with high priority, which checks surveillance camera footage and takes any necessary emergency action.

[1655] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1656] Step 1:

[1657] The device accepts voice input from the elderly person. For example, the elderly person may say, "Someone has come into the house." The input voice is converted into digital voice data via the device's microphone.

[1658] Step 2:

[1659] The speech received by the device is converted into text data using a generation AI (e.g., Google Speech-to-Text API). The input speech data is processed by a speech recognition model via the API, and text data is generated as a result. The output is the text data "Someone has entered the house."

[1660] Step 3:

[1661] The device passes the text data to an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to generate emotion data. The input text data is processed by the emotion analysis model, and the output emotion data is "fear."

[1662] Step 4:

[1663] The server receives the text data and emotion data and analyzes the request content. Using the text data "Someone entered the house" and the emotion data "fear" as input, the text analysis algorithm identifies the request content "suspicious person detected." The request content is identified as the output.

[1664] Step 5:

[1665] The server adjusts the priority of the request based on the request content and emotion data. Using the "suspicious person detected" content and the emotion data of "fear" as input, the priority adjustment algorithm gives the request the highest priority. The output is set as a "high priority" request.

[1666] Step 6:

[1667] The server searches for nearby available supporters based on the elderly person's location information and the request details. Using the elderly person's GPS location information and the request details for "suspicious person detection" as input, a database search algorithm is used to identify an appropriate supporter. The output is "Supporter A."

[1668] Step 7:

[1669] The server sends a request notification to the supporter. Using the identified supporter "Supporter A" and the request content "Suspicious person detected" as input, the request information is sent through the notification system to the supporter. The notification is delivered to the supporter as output.

[1670] Step 8:

[1671] The terminal receives a response from the supporter. Using the supporter's receipt notification as input, the supporter responds that they are available. The response "available" is obtained as output.

[1672] Step 9:

[1673] The user (supporter) visits the elderly person's home and provides support, specifically by checking the house for any suspicious individuals.

[1674] Step 10:

[1675] The device reports to the server that the supporter has ended support. The supporter's termination notification is used as input and reported to the server. The "support end" report is recorded as output.

[1676] Step 11:

[1677] The server records the support content and its evaluation in a database. The support date and time, content, supporter information, and evaluation are used as input and saved in the database. The support history is updated as output.

[1678] Step 12:

[1679] The terminal allows elderly people and their families to evaluate the quality of support and sends the results to the server. The evaluation information is used as input and sent to the server. The evaluation results are recorded in the database as output.

[1680] Step 13:

[1681] The server updates the trust database of supporters based on the evaluation results. The trust database is updated using an algorithm that uses the evaluation data as input and calculates the trust score. The updated trust score is obtained as output.

[1682] Step 14:

[1683] The server detects suspicious situations from voice input and immediately notifies the security service. Using the elderly person's voice request and emotion data as input, the server identifies suspicious situations using an anomaly detection algorithm and sends a notification to the security service. An emergency notification is sent as output.

[1684] Step 15:

[1685] Security services receive the notification, review the surveillance footage at the senior's home, and dispatch emergency services if necessary. The notification information is used as input to review the situation on the surveillance video system. The output is the initiation of an emergency response.

[1686] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1687] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1688] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1689] [Fourth embodiment]

[1690] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1691] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1692] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1693] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1694] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1695] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1696] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1697] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1698] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1699] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1700] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1701] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1702] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1703] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. This system has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteers, thereby resolving the issues that elderly people face.

[1704] Program processing overview

[1705] 1. Elderly request registration

[1706] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[1707] Device: Uses generative AI to convert speech to text. The speech data is converted into text data: "I want you to take me shopping."

[1708] Server: Receives the text data and analyzes the request. As a result of the analysis, it determines that "shopping support" is required.

[1709] Device: Obtains the elderly person's GPS location information and sends it to the server.

[1710] 2. Volunteer matching

[1711] Server: Based on the elderly person's location information, create a list of nearby volunteers. For example, confirm that "Volunteer A" is nearby.

[1712] Server: Check the skill sets and available time of the volunteers and select the most suitable volunteer. Confirm that "Volunteer A" is available to provide shopping support.

[1713] Server: Sends a request notification to the volunteer. It notifies the volunteer that "Elderly B wants shopping assistance."

[1714] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[1715] 3. Support implementation and history management

[1716] User (volunteer): Visits elderly people and provides shopping support.

[1717] Terminal: Reports to the server that the volunteer has completed their support.

[1718] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[1719] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[1720] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[1721] 4. Communication with elderly people with dementia

[1722] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[1723] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[1724] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[1725] Specific examples

[1726] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[1727] 2. Terminal: The generation AI converts the speech into text and sends it to the server.

[1728] 3. Server: Analyzes the text and determines if shopping assistance is required.

[1729] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[1730] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[1731] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[1732] 7. Terminal: Reports to the server that volunteer A has completed the support.

[1733] 8. Server: Records support history and ratings, and updates the trust score.

[1734] 9. Terminal: Older people and their families evaluate the quality of support.

[1735] In this way, the present invention makes it possible to quickly and effectively match local volunteers with elderly people, thereby supporting the elderly in living independently.

[1736] The processing flow will be explained below.

[1737] Step 1:

[1738] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[1739] Action: The device captures audio data and begins recording.

[1740] Step 2:

[1741] On-device: Uses generative AI to convert recorded audio into text in real time.

[1742] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[1743] Step 3:

[1744] Terminal: Sends text data to the server.

[1745] What it does: Uploads text data to a server over the internet.

[1746] Step 4:

[1747] Server: Receives and analyzes the text data to identify the request content.

[1748] How it works: Uses text analytics algorithms to identify that "shopping help" is needed.

[1749] Step 5:

[1750] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[1751] What it does: Uses the GPS module to capture current location information and upload it to a server.

[1752] Step 6:

[1753] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[1754] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[1755] Step 7:

[1756] Server: Check each volunteer's skill set and availability and select the best volunteer.

[1757] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[1758] Step 8:

[1759] Server: Sends notification of the request to selected volunteers.

[1760] What it does: Sends a push notification to the volunteer's device, displaying the request details.

[1761] Step 9:

[1762] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[1763] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[1764] Step 10:

[1765] User (volunteer): Visits elderly people and provides them with the necessary support.

[1766] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[1767] Step 11:

[1768] Terminal: Report to the server that the volunteer has discontinued support.

[1769] Operation: Press the support completion button and send a completion report to the server.

[1770] Step 12:

[1771] Server: Records support details and ratings in a database.

[1772] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[1773] Step 13:

[1774] Terminal: The elderly person or their family member enters their evaluation of the support and sends it to the server.

[1775] Action: The evaluation form is filled out on the device and uploaded to the server.

[1776] Step 14:

[1777] Server: Updates the trust database of volunteers based on the evaluation data.

[1778] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[1779] Step 15:

[1780] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[1781] What it does: Captures voice input from elderly people with dementia.

[1782] Step 16:

[1783] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[1784] What it does: Analyzes audio data and generates the conclusion "Needs meal assistance."

[1785] Step 17:

[1786] Server: Sends a response message to the terminal.

[1787] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[1788] Example 1

[1789] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1790] There is a problem that it is difficult for elderly people to receive prompt and appropriate support from local volunteers when they face difficulties in their daily lives. Furthermore, in the case of elderly people with dementia, it becomes even more difficult to provide appropriate support due to the increased difficulty of responding to emergencies and communicating. Therefore, a system that provides an effective solution to these issues is needed.

[1791] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1792] In this invention, the server includes means for the elderly to input voice, means for converting the voice input into text data, means for analyzing the text data to identify the request content, means for searching for an appropriate volunteer based on the identified request content, means for sending a request notification to the volunteer, means for accepting responses from the volunteer, means for recording the request history and evaluation, means for allowing the elderly to make voice requests using simple operation buttons, and means for generating and sending a response in accordance with the request. This enables the elderly to receive prompt and appropriate support from local volunteers for difficulties they face in their daily lives.

[1793] "Elderly" refers to individuals in an age group who require some assistance in daily living.

[1794] "Voice input" refers to a method in which elderly people speak into a dedicated device to capture voice information as digital data.

[1795] "Text data" refers to digital data converted from voice input into text information.

[1796] "Request content" refers to the specific request for assistance or support expressed by the elderly person through voice input.

[1797] "Volunteers" refer to local supporters whose purpose is to provide assistance to the elderly.

[1798] "Notification" refers to a message sent from the server to the volunteer informing them of the request.

[1799] "Responding" refers to the act of a volunteer responding to the server after receiving a request notification to let the server know whether or not they can respond.

[1800] "History" refers to the digital data stored record of past requests and their responses.

[1801] "Evaluation" refers to feedback given by seniors and their families regarding the volunteer's response to their request.

[1802] "Generative AI model" refers to the machine learning techniques used to convert voice data into text data and analyze that text data.

[1803] "Trustworthiness score" refers to an index that quantifies the reliability of a volunteer based on past evaluation data.

[1804] "Easy-to-operate buttons" refer to input devices designed to be easy to use even for elderly people who are unfamiliar with operating them.

[1805] "Response message" refers to the reply automatically generated by the server in response to a request entered by the elderly person.

[1806] This invention is a system that enables local volunteers to provide prompt and appropriate support for the difficulties that elderly people face in their daily lives. The system of the present invention has the function of automatically analyzing requests based on voice input and matching them with the most suitable volunteer. This system allows elderly people to more easily solve problems in their daily lives.

[1807] Hardware and Software Configuration

[1808] The implementation of this system uses the following main hardware and software:

[1809] Hardware

[1810] 1. Dedicated device: A device equipped with a microphone and buttons for voice input by the elderly. This device is equipped with a microphone to capture voice data and a GPS module (e.g., U-Blox NEO-6M GPS module) to obtain location information.

[1811] 2. Smartphone: A device that allows volunteers to receive and respond to notifications. A dedicated application (e.g., ActionBound app) is installed.

[1812] software

[1813] 1. Generative AI model: This is a machine learning technology that converts voice data into text and then analyzes that text. For voice-to-text conversion, we use the Google Cloud Speech-to-Text API.

[1814] 2. Natural language processing engine: Use a natural language processing engine such as IBM Watson NLU to analyze the request content from text data.

[1815] 3. Server system: This is the central system that receives requests from seniors, analyzes them, and notifies and matches them with volunteers.

[1816] Program processing

[1817] The program of this system operates according to the following processing flow.

[1818] Voice input for the elderly

[1819] The device starts when the elderly person presses a button on the dedicated device to input voice commands, for example, "Please take me shopping."

[1820] Speech to text

[1821] When the device receives voice input, it converts the data into text using a generative AI model (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[1822] Text data analysis

[1823] The server receives the text data and analyzes the request using a natural language processing engine (IBM Watson NLU). As a result of the analysis, it determines that "shopping support" is required.

[1824] Acquiring location information of elderly people

[1825] The device acquires the elderly person's GPS location information and sends it to the server, where the GPS module installed in the device measures their current location.

[1826] Volunteer Selection and Notification

[1827] The server uses the elderly person's location information to create a list of nearby volunteers, checks their skill sets and availability, and selects the most suitable volunteer. For example, it verifies that "Volunteer A" is nearby and available to assist with shopping. The server then sends a request notification to the volunteer.

[1828] Volunteer Response

[1829] The device helps the volunteer receive the notification and respond to the server whether they are available. The volunteer responds "I'm available" using the ActionBound app on their smartphone.

[1830] Support and history management

[1831] The user (volunteer) visits the elderly person and provides shopping support. The terminal reports to the server that the volunteer has completed the support. The server records the date, time, and content of the support, as well as the volunteer information, in a database. It also recalculates the reliability score and provides an interface for the elderly person and their family to evaluate the support.

[1832] Support for elderly people with dementia

[1833] The device allows elderly people with dementia to make voice requests using simple operation buttons, such as saying "I'm hungry." The server analyzes the voice in real time and generates an appropriate response message. The device then displays a message such as "We will contact a nearby volunteer."

[1834] Specific examples

[1835] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[1836] 2. Device: The generating AI (Google Cloud Speech-to-Text) converts the speech into text and sends it to the server.

[1837] 3. Server: Analyzes the text using IBM Watson NLU and determines if shopping assistance is required.

[1838] 4. Terminal: Sends the elderly person's location information (U-Blox NEO-6M GPS module) to the server.

[1839] 5. Server: Using the elderly person's location information, send a notification to nearby "Volunteer A."

[1840] 6. Device: Volunteer A receives a notification in the ActionBound app and replies that they are available.

[1841] 7. User (Volunteer A): A volunteer visits an elderly person and provides shopping assistance.

[1842] 8. Device: Volunteer A uses the ActionBound app to report completion of support to the server.

[1843] 9. Server: Records support history and ratings in a database and updates the trust score.

[1844] 10. Device: Elderly people and their families evaluate the quality of support through a smartphone rating app.

[1845] In this way, this system can effectively match local volunteers with elderly people and support their independent living.

[1846] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1847] Step 1:

[1848] Device: The elderly person presses a button on the dedicated device to input voice. For example, the elderly person may say, "I want you to take me shopping."

[1849] Input: Voice input for the elderly

[1850] Output: Audio data

[1851] Step 2:

[1852] Device: The voice input is converted into text using a generative AI (Google Cloud Speech-to-Text). The voice data is converted into text data such as "Please take me shopping."

[1853] Input: Audio data

[1854] Data processing: converting voice to text

[1855] Output: Text data

[1856] Step 3:

[1857] Terminal: The converted text data is sent to the server via an HTTP request.

[1858] Input: Text data

[1859] Output: Request sent to server

[1860] Step 4:

[1861] Server: Analyzes the received text data using IBM Watson NLU to identify the request. From the analysis results, it determines that "shopping support" is required.

[1862] Input: Text data

[1863] Data Calculation: Analyzing request content using natural language processing

[1864] Output: Identifying the request

[1865] Step 5:

[1866] Device: Acquires the elderly person's GPS location information and sends it to the server. The device is equipped with a GPS module (U-Blox NEO-6M GPS module).

[1867] Input: Location request

[1868] Data processing: Measure your current location and convert it into data

[1869] Output: Location data

[1870] Step 6:

[1871] Server: Creates a list of nearby volunteers based on the elderly person's location information.

[1872] Input: Location data

[1873] Data calculation: Generating a volunteer list through database searches

[1874] Output: Volunteer list

[1875] Step 7:

[1876] Server: Checks the skill sets and availability of volunteers and selects the most suitable volunteer. For example, identifies that "Volunteer A" is available to assist with shopping.

[1877] Input: Volunteer List

[1878] Data crunching: Checking volunteer skill sets and availability

[1879] Output: Selected volunteers

[1880] Step 8:

[1881] Server: Sends a request notification to the selected volunteer. Sends a notification saying, "Elderly B wants shopping assistance."

[1882] Input: Selected volunteer, request details

[1883] Output: Notification to volunteer

[1884] Step 9:

[1885] Terminal: After receiving the notification, the volunteer replies to the server whether they are available or not. Specifically, the volunteer uses a dedicated smartphone application (ActionBound app) to select "available" or "unavailable" and respond.

[1886] Input: Volunteer response

[1887] Output: Sending response data

[1888] Step 10:

[1889] User (volunteer): A volunteer visits an elderly person and provides designated support, such as taking them shopping.

[1890] Input: Elderly Request

[1891] Output: Providing support

[1892] Step 11:

[1893] Device: The volunteer reports to the server that they have completed their support. The volunteer uses the ActionBound app on their smartphone to press the "Support Completed" button.

[1894] Input: Support Completion Information

[1895] Output: Report to server

[1896] Step 12:

[1897] Server: Records support history and ratings in a database and updates the reliability score. Records the date and time of support, content, volunteer information, etc., and recalculates the reliability score.

[1898] Input: Support completion information, evaluation data

[1899] Data calculation: Recording support history and recalculating confidence scores

[1900] Output: Updated database

[1901] Step 13:

[1902] Device: Elderly people and their families use a smartphone rating app to rate the quality of support and send the results to the server.

[1903] Input: Evaluation information

[1904] Output: Send to server

[1905] Step 14:

[1906] Server: When an elderly person with dementia makes a voice request using the simple operation buttons, the voice is analyzed in real time and a response message is generated.

[1907] Input: Voice request

[1908] Data calculation: Real-time voice analysis, response message generation

[1909] Output: Response message

[1910] Step 15:

[1911] Server: Sends a response message to the terminal and displays it on the terminal. It displays a message such as "We will contact a nearby volunteer."

[1912] Input: Response message

[1913] Output: Display to terminal

[1914] (Application example 1)

[1915] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1916] The problem to be solved by this invention is to provide a system that provides prompt and appropriate assistance to seniors to eliminate the difficulties they face when searching for items in brick-and-mortar stores. In particular, this system allows seniors with dementia and those who require mobility in their daily lives to easily receive the necessary support through voice requests. Another problem is to ensure the reliability of the supporter and achieve effective support.

[1917] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1918] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the requested content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response in response to the request, and means for matching requests from elderly people who need item search or guidance with supporters in a physical store based on location information, thereby enabling items needed by elderly people to be searched for and guided to in a physical store quickly.

[1919] "Means for accepting voice input" refers to a device or apparatus that allows the elderly person to input requests by voice.

[1920] "Means for converting received voice into text" refers to technology or a system that converts voice input into text data.

[1921] "Means for analyzing textual content to identify the request content" refers to technologies and systems that understand textual data and grasp specific needs and requests.

[1922] A "means for searching for supporters who can provide the requested support" is a system for searching for people or organizations that can provide appropriate support based on the specified request content.

[1923] The "means for sending a request notification to the supporter" refers to a communication technology or system for informing the supporter of the request content.

[1924] "Means for receiving responses from supporters" refers to technologies and systems for receiving responses from supporters, such as whether they can respond or not.

[1925] The "means for recording request history and evaluation" is a system for storing the support provided and its evaluation in a database.

[1926] "Means that allow elderly people with dementia to make voice requests using simple operation buttons" refers to devices and technologies that provide an interface that can be easily operated by elderly people with dementia.

[1927] The "means for generating and transmitting a response in response to a request" is a system for generating an appropriate response to the request content and conveying it to the elderly person.

[1928] "Means for matching elderly people who need to search for items or receive guidance with helpers in physical stores based on location information" refers to technology or a system that uses the elderly person's location information to find helpers in physical stores who can provide appropriate assistance.

[1929] This invention is a system that solves the difficulties that elderly people face when searching for items in physical stores and provides them with prompt and appropriate assistance. This system is configured as follows.

[1930] First, the elderly person uses a device such as smart glasses or a smartphone to input voice commands. This device is equipped with a means to accept voice commands. For example, the elderly person can input a request by saying, "Please help me find milk."

[1931] The device uses generative AI technology to convert speech into text data, which then activates a means of converting received speech into text. OpenAI's GPT-3.5, for example, is used as the generative AI model.

[1932] Next, a means for analyzing the textual content and identifying the content of the request operates on the server. This analysis means identifies that a "product guide" is required from the text data "Please help me find milk."

[1933] After the request content is specified, a means for searching for an assistant who can provide the requested support based on the specified request content operates on the server. This means matches the optimal assistant based on the location information, skill set, available time, etc. of the assistant in the physical store. For example, an assistant who is nearby is selected.

[1934] The server then runs a mechanism to notify the supporter of the request, and the supporter then responds to the server using a mechanism to respond to the notification, indicating whether they are able to respond. For example, notifications are sent using Firebase Cloud Messaging (FCM).

[1935] If the assistant is available, they visit the elderly person and assist them with product information and search. Once the assistance is complete, a system runs on the server to record the request history and evaluation. This records the assistance details and evaluation in a database (e.g., MySQL, Amazon RDS).

[1936] To accommodate elderly people with dementia, the terminals are equipped with easy-to-operate buttons and interfaces, allowing elderly people with dementia to make voice requests by simply operating buttons.

[1937] A means for generating and transmitting a response according to the request operates on the server. This means creates an appropriate response to the request content and displays it on the terminal. For example, a message such as "We will contact a nearby supporter" is displayed.

[1938] For example, an elderly person might say to the smart glasses, "Please help me find some milk." This speech is converted to text using a generative AI model, and the request is analyzed. An appropriate helper in the physical store is selected and notified. The helper visits the elderly person, provides them with the item, and their evaluation after completing the assistance is recorded in a database.

[1939] Example prompt sentence:

[1940] "An elderly person has made a voice request using smart glasses. Please analyze the following request to identify the appropriate action:

[1941] Voice Request: "Help me find some milk."

[1942] Analysis result: The request for this article is identified as "Product Information."

[1943] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1944] Step 1:

[1945] Elderly people input information by voice into a device (smart glasses or smartphone).

[1946] Input: Senior's voice request (e.g., "Please help me find milk.")

[1947] Output: Audio data

[1948] Specific operation: The device accepts voice input through the microphone and stores it as digital voice data.

[1949] Step 2:

[1950] The device converts the received voice data into text data using generative AI technology.

[1951] Input: Audio data

[1952] Output: Text data (e.g. "Please help me find milk")

[1953] How it works: The device uses a speech recognition library (e.g., Google Speech-to-Text API) to convert voice data into text, using a generative AI model (e.g., OpenAI GPT-3.5).

[1954] Step 3:

[1955] The server analyzes the text data and identifies the request content.

[1956] Input: Text data

[1957] Output: Request details (e.g. "Product information required")

[1958] How it works: The server uses natural language processing technology to analyze the text data and identify what the elderly person wants. This process uses a generative AI model.

[1959] Step 4:

[1960] The server obtains the elderly person's location information and searches for a supporter.

[1961] Input: Request details, location information of the elderly person

[1962] Output: List of supporters

[1963] Specific operation: Based on the GPS location information sent from the device, the server searches the supporter database and lists supporters who meet the requirements, using Google Maps API and iBeacon technology.

[1964] Step 5:

[1965] The server sends a notification of the request to the identified supporter.

[1966] Input: List of supporters, request details

[1967] Output: Notification message

[1968] Specific operation: The server sends a notification message to nearby supporters using Firebase Cloud Messaging (FCM).

[1969] Step 6:

[1970] The supporter receives the notification and responds to the server whether or not they can respond.

[1971] Input: Notification message

[1972] Output: Response regarding availability

[1973] Specific operation: The supporter checks the notification on a device such as a smartphone or tablet, selects whether they can respond, and responds to the server.

[1974] Step 7:

[1975] The corresponding support person provides assistance to the elderly person.

[1976] Input: Elderly person's location information, request details

[1977] Output: The results of the assistance provided

[1978] Specific actions: The supporter visits the elderly person and assists them in guiding them to items and searching for them. In this case, the support is provided face-to-face with the elderly person.

[1979] Step 8:

[1980] The server records the history of assistance and the elderly person's evaluation.

[1981] Input: Support details, evaluation data

[1982] Output: Updated database

[1983] Specific operation: After the supporter completes the support, he / she reports to the server via the terminal, and the elderly person and their family evaluate the quality of the support on the terminal. The evaluation results and support history are stored in a database.

[1984] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1985] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions.

[1986] Program processing overview

[1987] 1. Request registration and emotion recognition for elderly people

[1988] Device: The elderly person presses a button on a dedicated device to input voice commands, for example, "Please take me shopping."

[1989] On the device: It uses generative AI to convert speech to text, while simultaneously using an emotion engine to recognize the elderly person's emotions from their speech.

[1990] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "shopping support" is needed and recognizes that the elderly person is feeling "anxiety."

[1991] Device: Obtains the elderly person's GPS location information and sends it to the server.

[1992] 2. Volunteer matching and emotion-based prioritization

[1993] Server: Based on the elderly person's location information, it lists nearby available volunteers. For example, it checks whether "Volunteer A" is nearby.

[1994] Server: Check each volunteer's skill set and available time, and select the most suitable volunteer. Confirm that "Volunteer A" is available to assist with shopping.

[1995] Server: Adjust the priority of requests based on the perceived emotion. For example, requests from elderly people who are anxious will be treated as high priority.

[1996] Server: Sends a request notification to the volunteer. For example, it may say, "Elderly B would like shopping assistance. Elderly B is currently feeling anxious."

[1997] Terminal: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" replies that they are available.

[1998] 3. Support implementation and history management

[1999] User (volunteer): Visits elderly people and provides shopping support.

[2000] Terminal: Report to the server that the volunteer has discontinued support.

[2001] Server: Records support details and their evaluation in a database, including the date and time of support, content, volunteer information, etc.

[2002] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[2003] Server: Updates the trust database of volunteers based on the evaluation results. Recalculates the trust score and displays it to other seniors.

[2004] Specific examples

[2005] 1. Elderly person's request: An elderly person speaks into the terminal, "I want you to take me shopping."

[2006] 2. Device: Generative AI and emotion engine converts speech into text and emotion data to identify "shopping assistance" and "anxiety."

[2007] 3. Server: Analyzes text and emotion data and determines that shopping support is needed and is a high priority.

[2008] 4. Server: Based on the elderly person's location information, a notification is sent to a nearby "Volunteer A."

[2009] 5. Terminal: Volunteer A receives the notification and replies that he is available.

[2010] 6. User (Volunteer A): Visits elderly people and provides shopping support.

[2011] 7. Terminal: Reports to the server that volunteer A has completed the support.

[2012] 8. Server: Records support history and ratings, and updates the trust score.

[2013] 9. Terminal: Older people and their families evaluate the quality of support.

[2014] Communication with elderly people with dementia

[2015] Device: An elderly person with dementia uses simple operation buttons to make a voice request, for example, "I'm hungry."

[2016] Server: Analyzes the voice in real time and generates an appropriate response. Determines that "meal assistance is needed."

[2017] Server: Sends a response message to the device. A message such as "Contacting a nearby volunteer" is displayed on the device.

[2018] This invention provides a system that quickly and effectively matches local volunteers with elderly people and also responds appropriately based on the user's emotions, thereby supporting the independent living of elderly people and improving the quality of emotional support.

[2019] The processing flow will be explained below.

[2020] Step 1:

[2021] Device: The elderly person presses a button on the dedicated device to input voice commands, for example, saying, "I want you to take me shopping."

[2022] Action: The device captures audio data and begins recording.

[2023] Step 2:

[2024] On-device: Uses generative AI to convert recorded audio into text in real time.

[2025] Operation: The voice data is converted into text and the text data is "Please take me shopping."

[2026] Step 3:

[2027] Terminal: Recognize the emotions of elderly people from their voice using an emotion engine.

[2028] How it works: Analyzes audio data and generates emotion data (e.g., "anxiety") along with text data.

[2029] Step 4:

[2030] Terminal: Sends text data and emotion data to the server.

[2031] How it works: The generated text data and emotion data are uploaded to a server via the Internet.

[2032] Step 5:

[2033] Server: Receives text data and emotion data, analyzes and identifies the request content.

[2034] How it works: It uses text analytics algorithms to identify the need for "shopping assistance" and that the senior is feeling "anxious."

[2035] Step 6:

[2036] Device: Obtains the elderly person's location information from GPS and sends it to the server.

[2037] What it does: Uses the GPS module to capture current location information and upload it to a server.

[2038] Step 7:

[2039] Server: Based on the location information of the elderly person, it creates a list of available volunteers in the vicinity.

[2040] How it works: Uses a database query to find volunteers within a certain radius of the current location.

[2041] Step 8:

[2042] Server: Check each volunteer's skill set and availability and select the best volunteer.

[2043] How it works: Gets volunteer skills and schedules from a database and runs a matching algorithm to select the best volunteers.

[2044] Step 9:

[2045] Server: Adjust the priority of requests based on perceived sentiment.

[2046] Behavior: If the emotional data is determined to be "anxiety," it is processed as high priority.

[2047] Step 10:

[2048] Server: Sends notification of the request to selected volunteers.

[2049] What it does: Sends a push notification to the volunteer's device with the message, "Elderly B wants shopping assistance. Elderly B is currently feeling anxious."

[2050] Step 11:

[2051] Terminal: The volunteer receives the notification and responds to the server whether they are available or not.

[2052] How it works: The volunteer checks the notification message, chooses whether they are available or not, and sends a response back to the server.

[2053] Step 12:

[2054] User (volunteer): Visits elderly people and provides them with the necessary support.

[2055] How it works: Volunteers visit elderly people's homes and provide shopping assistance.

[2056] Step 13:

[2057] Terminal: Report to the server that the volunteer has discontinued support.

[2058] Operation: Press the support completion button and send a completion report to the server.

[2059] Step 14:

[2060] Server: Records support details and ratings in a database.

[2061] Operation: Saves the support date and time, content, evaluation score, etc. in a database.

[2062] Step 15:

[2063] Device: Elderly people and their families rate the quality of support and send the results to the server.

[2064] Action: The evaluation form is filled out on the device and uploaded to the server.

[2065] Step 16:

[2066] Server: Updates the trust database of volunteers based on the evaluation data.

[2067] Behavior: Recalculates the trust score and updates the volunteer's trust information.

[2068] Step 17:

[2069] Terminal: Elderly people with dementia can press a simple operation button to make voice requests.

[2070] What it does: Captures voice input from elderly people with dementia.

[2071] Step 18:

[2072] Server: Analyzes the voice of elderly people with dementia in real time and generates appropriate responses.

[2073] What it does: Analyzes audio data and generates a conclusion that determines "eating assistance is needed."

[2074] Step 19:

[2075] Server: Sends a response message to the terminal.

[2076] What it does: Displays the message "Contacting a nearby volunteer" on the device.

[2077] Example 2

[2078] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2079] Responding appropriately and promptly to the difficulties faced by elderly people, especially those with dementia, in their daily lives is an important issue in modern society. However, current support systems lack sufficient functionality to effectively analyze user requests and quickly match them with the most suitable volunteers. They also lack the functionality to adjust response priorities while taking into account the emotional state of the elderly. This can result in delays in actual assistance or in the elderly not receiving sufficient support.

[2080] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2081] In this invention, the server includes a means for recognizing the user's emotion from voice, a means for acquiring the elderly person's location information, and a means for sending a request notification to a volunteer, thereby enabling prompt and appropriate volunteer matching that takes into account the elderly person's emotional state.

[2082] "Voice input" refers to the act of elderly people making requests using voice through a dedicated device.

[2083] "Text conversion" is the process of analyzing audio data and converting it into text information.

[2084] "Request content" refers to the specific content of the request conveyed by the elderly person through voice input.

[2085] "Emotion recognition" is a technology that identifies the emotional state of elderly people from voice data.

[2086] "Location information" is data that identifies the current location of an elderly person using GPS functionality.

[2087] A "volunteer" is someone who provides assistance in response to requests from elderly people.

[2088] A "notification" is a message sent by the server to a volunteer to inform them of the contents of the request.

[2089] "Responding" is the act of a volunteer replying to the server about whether they are available to respond to a request.

[2090] "History" is data that records the implementation details and evaluations of past requests.

[2091] The "easy operation button" is an input device designed to be easily operated by elderly people with dementia.

[2092] "Generative artificial intelligence technology" is a technology that uses generative AI models to analyze voice data and recognize emotions.

[2093] A "trust score" is an indicator of a volunteer's trustworthiness calculated based on their past evaluation data.

[2094] This invention is a system for responding quickly and appropriately to the difficulties that elderly people face in their daily lives, with the aim of providing support that takes into account the emotional state of the elderly. In addition to the basic function of analyzing requests based on voice input and matching them with the most suitable volunteers, the system also has the ability to adjust the priority of requests and notification content using emotion recognition technology.

[2095] Hardware and Software Configuration

[2096] Devices: Older adults use dedicated devices (e.g., smart speakers with voice assistants, tablets).

[2097] Analysis software: Generative AI (e.g., Google Cloud Speech-to-Text) is used to convert voice data into text. Emotion engine (e.g., Microsoft Azure Emotion API) is used to recognize emotions.

[2098] Server: Receives text and emotion data, analyzes requests, searches for volunteers, sends notifications, receives responses, and records history and ratings.

[2099] GPS function: Use a smartphone or tablet with GPS functionality to track the senior's current location.

[2100] Specific examples of program processing

[2101] 1. Request registration and emotion recognition for elderly people

[2102] - Device: The elderly person presses a button on the device and voice-inputs, "I want you to take me shopping."

[2103] - Device: Uses generative AI to convert speech into text data, while simultaneously using an emotion engine to recognize emotions. Identifies that "shopping assistance" is needed and that the elderly person is feeling "anxious."

[2104] - Server: Receives text data and emotion data and analyzes the request content.

[2105] - Device: The elderly person's GPS location information is also sent to the server.

[2106] 2. Volunteer matching and emotion-based prioritization

[2107] - Server: Based on the received location information, it creates a list of nearby volunteers and checks each volunteer's skill set and available time. For example, it checks whether "Volunteer A" is available to provide shopping assistance.

[2108] - Server: Prioritize requests based on emotion data. Requests from elderly people who are anxious are given high priority.

[2109] - Server: Sends a request notification to the volunteer. The notification reads, "Elderly B would like shopping support. Elderly B is currently feeling anxious."

[2110] - Device: The volunteer receives the notification and replies to the server whether they are available. "Volunteer A" responds that they are available.

[2111] 3. Support implementation and history management

[2112] - User (volunteer): Visits elderly people and provides shopping support.

[2113] - Terminal: Report to the server that the volunteer has ended support.

[2114] - Server: Records support details and their evaluations in a database, such as support date and time, content, volunteer information, etc.

[2115] - Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. They rate "Volunteer A" with five stars.

[2116] - Server: Updates the trust database of volunteers based on the evaluation results, recalculates the trust score, and displays it to other seniors.

[2117] Examples of prompt statements

[2118] An elderly person speaks into the terminal, "I want you to take me shopping."

[2119] This system allows elderly people to receive daily support with peace of mind, enables volunteers to respond quickly and appropriately, and improves the quality of support by taking into account emotional states.

[2120] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2121] Detailed flow of program processing

[2122] 1. Request registration and emotion recognition for elderly people

[2123] Step 1: Getting voice input

[2124] Device: The elderly person presses a button on the dedicated device to input voice commands, such as "I want you to take me shopping."

[2125] Specific operation: After the elderly person presses the button, the microphone is activated and captures voice data.

[2126] Step 2: Speech-to-text and emotion recognition

[2127] Terminal: The voice data is converted into text using generative AI, and emotion data is extracted using an emotion engine. The input is voice data, and the output is text data ("Please take me shopping") and emotion data ("anxiety").

[2128] Specific operation: Send voice data to a generative AI (e.g., Google Cloud Speech-to-Text) to obtain text data, and simultaneously extract emotion data using an emotion engine (e.g., Microsoft Azure Emotion API).

[2129] Step 3: Sending text and emotion data

[2130] Terminal: Sends text data and emotion data to the server. The input is text data and emotion data, and the output is the server's confirmation of receipt.

[2131] Specific operation: The device sends data to the server's API endpoint, and the server receives the data.

[2132] Step 4: Obtaining location information

[2133] Terminal: Acquires the elderly person's GPS location information and sends it to the server. The input is GPS data, and the output is sending location information to the server.

[2134] Specific operation: The device uses the built-in GPS function to obtain its current location and sends the location information to the server.

[2135] 2. Volunteer matching and emotion-based prioritization

[2136] Step 5: Create a list of neighborhood volunteers

[2137] Server: Based on the received location information, it creates a list of available volunteers in the vicinity. The input is the location information, and the output is the volunteer list.

[2138] What happens: The server searches the database and lists volunteers within the specified radius.

[2139] Step 6: Identify volunteer skillsets and availability

[2140] Server: Identifies the best volunteers by checking the skill sets and availability of each volunteer. Input is the list of volunteers, output is the best volunteers.

[2141] What happens: The server looks up the volunteer profile information and identifies volunteers who match the requirements.

[2142] Step 7: Prioritize requests based on sentiment data

[2143] Server: Adjusts the priority of requests based on the recognized emotion. The input is emotion data, and the output is the adjusted priority.

[2144] What it does: A priority adjustment algorithm analyzes sentiment data and adjusts the request's queue position.

[2145] Step 8: Sending a Request Notification

[2146] Server: Sends request notifications to volunteers. Inputs are the best volunteer and the request content, and output is the notification message.

[2147] Specific operation: The server sends a push notification to the volunteer's device, displaying the request details.

[2148] Step 9: Receiving volunteer responses

[2149] Terminal: The volunteer receives the notification and replies to the server whether they are available. The input is the volunteer's response, and the output is the response sent to the server.

[2150] What happens: The volunteer taps the notification message to send a reply message from the application.

[2151] 3. Support implementation and history management

[2152] Step 10: Providing support

[2153] User (volunteer): Visits elderly people and provides shopping support. The input is the elderly person's request, and the output is the implementation of the support.

[2154] Specific actions: Volunteers visit elderly people's homes and provide shopping assistance.

[2155] Step 11: Report support complete

[2156] Terminal: Reports to the server that the volunteer has ended support. Input is the completion status, and output is the report to the server.

[2157] Specific action: The volunteer operates the application on the device to update the status of support completion.

[2158] Step 12: Record support and evaluation

[2159] Server: Records support details and evaluations in a database. The input is support details and evaluation data, and the output is the recorded data.

[2160] Specific behavior: The server receives the evaluation result and adds a record to the database.

[2161] Step 13: Evaluate the quality of support

[2162] Terminal: Elderly people and their families evaluate the quality of support and send the results to the server. The input is the evaluation result, and the output is the data sent to the server.

[2163] Specific operation: Elderly people and their families fill out an evaluation form and send the results to the server.

[2164] Step 14: Update the Reliability Database

[2165] Server: Updates the trust database of volunteers based on the evaluation results. The input is the evaluation results, and the output is the updated trust scores.

[2166] Specific behavior: The server recalculates the trust score based on the new reputation score and updates the database.

[2167] (Application example 2)

[2168] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2169] In the daily lives of elderly people, a system that can respond quickly and appropriately is necessary, especially in emergencies or situations where they feel anxious. Conventional systems can convert voice-input requests into text and match them with supporters, but no systems exist that have security features or priority adjustment that take user emotions into account. Therefore, there is a need for a method to adjust the priority of requests based on emotions and ensure emergency responses according to specific situations.

[2170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2171] In this invention, the server includes means for accepting voice input, means for converting the accepted voice into text, means for analyzing the converted text to identify the request content, means for searching for a supporter who can provide the requested support based on the identified request content, means for sending a request notification to the supporter, means for accepting responses from the supporter, means for recording the request history and evaluation, means for enabling an elderly person with dementia to make a voice request using simple operation buttons, means for generating and sending a response according to the request, means for recognizing emotions and adjusting the priority of the request, and means having a security function for immediately notifying after detecting the situation. This makes it possible to take emotions into consideration when an elderly person makes a voice request and quickly respond to requests with high urgency.

[2172] "Means for accepting voice input" refers to a device or interface that receives voice commands issued by the senior.

[2173] "Means of converting received voice into text" refers to the technology or software that converts voice data into text data.

[2174] "Means for analyzing the textual content and identifying the request content" refers to algorithms or programs that analyze textual data and identify the elderly person's requests from its content.

[2175] "Means for searching for supporters who can provide the requested support based on the specified request content" refers to a function for searching a database for supporters who match the request.

[2176] "Means for sending a request notification to a supporter" refers to a communication technology for sending request information to a specified supporter.

[2177] "Means for accepting responses from supporters" refers to the function of receiving confirmation of receipt and replies from supporters.

[2178] "Means for recording request history and evaluation" refers to a system that stores and manages history and evaluation information regarding requests and support for older adults.

[2179] "A means for enabling elderly people with dementia to make voice requests using simple operation buttons" refers to an interface that allows elderly people with dementia to input voice commands with simple operations.

[2180] "Means for generating and sending a response in response to a request" refers to the function of generating an appropriate response based on the content of the request and sending it to the elderly person.

[2181] "Means for recognizing emotions and adjusting the priority of requests" refers to an algorithm that analyzes the emotional state of the elderly person and determines the priority of processing requests accordingly.

[2182] "Having security functions and means to immediately notify after detecting a situation" refers to the function to detect abnormal or emergency situations and immediately notify the relevant security services of that information.

[2183] This invention is a system for quickly and appropriately responding to the difficulties that elderly people face in their daily lives, and aims to further improve the quality of the service by incorporating a new emotion engine. In addition to the basic function of analyzing requests based on voice input and matching them with appropriate supporters (volunteers, etc.), this system also recognizes the user's emotions and adjusts the priority of requests and notification content based on those emotions. The system also has security functions and is equipped with a means to immediately notify users when a suspicious situation is detected.

[2184] 1. Voice Input and Emotion Recognition

[2185] Device: The elderly person speaks into a dedicated device (e.g., smartphone, robot) and says, for example, "Someone is coming into the house."

[2186] Device: Uses generative AI (e.g., Google Speech-to-Text API) to convert speech to text, while simultaneously using an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the elderly person's emotions from their speech.

[2187] Server: Receives the text data and emotion data and analyzes the request. As a result of the analysis, it determines that "suspicious person detection" is required and that the elderly person is feeling "fear."

[2188] 2. Matching supporters and adjusting priorities based on emotions

[2189] Server: Adjust the priority of requests based on the recognized emotion. For example, requests from elderly people who feel "fear" will be given top priority.

[2190] Server: Based on the elderly person's location information and request details, the server creates a list of nearby available supporters. For example, it checks whether "Supporter A" is nearby.

[2191] Server: Sends a request notification to the supporter. For example, it may say, "Elderly B wants to detect suspicious people. Elderly B is currently feeling scared."

[2192] Terminal: The supporter receives the notification and replies to the server whether they are available. For example, "Supporter A" responds that they are available.

[2193] 3. Support implementation and history management

[2194] User (supporter): Visits the elderly person's home and provides support. For example, checking the house for suspicious people.

[2195] Terminal: Report to the server that the supporter has ended support.

[2196] Server: Records the support content and its evaluation in a database, including the date and time of the support, the content, and information about the supporter.

[2197] Device: The elderly person or their family member evaluates the quality of support and sends the results to the server. For example, they may rate "Supporter A" five stars.

[2198] Server: Updates the database of supporter reliability based on the evaluation results. Recalculates the reliability score and displays it to other seniors.

[2199] 4. Implementation of security functions

[2200] Server: Detects suspicious situations from voice input and immediately notifies the security service.

[2201] Security Services: Review surveillance footage at the senior's home in real time and dispatch emergency services if necessary.

[2202] Specific examples

[2203] 1. An elderly person tells a robot installed in their living room, "It feels like someone has come into my house."

[2204] 2. The robot converts the voice into text and emotional data (e.g., fear), analyzes it, and determines whether suspicious person detection is necessary.

[2205] 3. The server processes the request with high priority and sends a notification to the security service.

[2206] 4. Security services will review surveillance footage and respond to emergencies as necessary.

[2207] Example of input prompt for generative AI model

[2208] Input prompt: Describe a suspicious person detection system for when an elderly person says "someone has entered the house" and then feels scared. Please be specific and include the voice analysis, emotion engine, and security notification flow.

[2209] Example output: When an elderly person tells the robot, "Someone has come into the house," a speech analysis engine (e.g., Google Speech-to-Text API) converts the speech into text, and an emotion engine (e.g., IBM Watson Tone Analyzer) detects and analyzes the elderly person's fear. The analysis information is notified to the security service with high priority, which checks surveillance camera footage and takes any necessary emergency action.

[2210] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2211] Step 1:

[2212] The device accepts voice input from the elderly person. For example, the elderly person may say, "Someone has come into the house." The input voice is converted into digital voice data via the device's microphone.

[2213] Step 2:

[2214] The speech received by the device is converted into text data using a generation AI (e.g., Google Speech-to-Text API). The input speech data is processed by a speech recognition model via the API, and text data is generated as a result. The output is the text data "Someone has entered the house."

[2215] Step 3:

[2216] The device passes the text data to an emotion recognition engine (e.g., IBM Watson Tone Analyzer) to generate emotion data. The input text data is processed by the emotion analysis model, and the output emotion data is "fear."

[2217] Step 4:

[2218] The server receives the text data and emotion data and analyzes the request content. Using the text data "Someone entered the house" and the emotion data "fear" as input, the text analysis algorithm identifies the request content "suspicious person detected." The request content is identified as the output.

[2219] Step 5:

[2220] The server adjusts the priority of the request based on the request content and emotion data. Using the "suspicious person detected" content and the emotion data of "fear" as input, the priority adjustment algorithm gives the request the highest priority. The output is set as a "high priority" request.

[2221] Step 6:

[2222] The server searches for nearby available supporters based on the elderly person's location information and the request details. Using the elderly person's GPS location information and the request details for "suspicious person detection" as input, a database search algorithm is used to identify an appropriate supporter. The output is "Supporter A."

[2223] Step 7:

[2224] The server sends a request notification to the supporter. Using the identified supporter "Supporter A" and the request content "Suspicious person detected" as input, the request information is sent through the notification system to the supporter. The notification is delivered to the supporter as output.

[2225] Step 8:

[2226] The terminal receives a response from the supporter. Using the supp...

Claims

1. means for accepting voice input; a means for converting the received speech into text; A means for analyzing the textual content to identify the request content; A means of searching for volunteers who can provide the requested support based on the identified request; a means for sending notification of the request to the volunteer; a means for accepting responses from volunteers; a means of recording the history and evaluation of requests; A means for enabling elderly people with dementia to make voice requests using simple operation buttons; A system that includes a means for generating and transmitting a response in response to a request.

2. 10. The system of claim 1, wherein the request identification utilizes generative artificial intelligence techniques.

3. 10. The system of claim 1, further comprising means for aggregating the collected evaluation data and generating a reliability score to ensure the reliability of the volunteer.

Images