system

Abstract

Provide a system. [Solution] A telephone terminal equipped with generative artificial intelligence, a means for determining whether a call is fraudulent; A means for the generative artificial intelligence to automatically respond to the determined fraudulent call; If the call is not a scam, how can we transfer the call to an elderly person? A system that includes a means for generative artificial intelligence to converse with elderly people living alone who want someone to talk to.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Scam phone calls cause serious harm to the elderly, and elderly people living alone are often targeted. Conventional methods for solving this problem have technical limitations and high costs, resulting in a lack of effective and sustainable solutions. Furthermore, elderly people living alone often feel lonely and need someone to watch over them and talk to, something that current systems are unable to adequately address. The objective of this invention is to provide technology that can effectively prevent elderly people from falling victim to scam phone calls and also function as a conversation partner to reduce their sense of loneliness. [Means for solving the problem]

[0005] The present invention provides a telephone terminal equipped with generative artificial intelligence. This telephone terminal has a means for determining in real time whether a call is likely to be fraudulent, and if the determination determines that the call is fraudulent, the generative artificial intelligence is configured to automatically continue the conversation with the fraudster. It also has a means for transferring the call to an elderly person if the call is not fraudulent. Furthermore, if an elderly person living alone needs someone to talk to, the generative artificial intelligence can act as a conversation partner. To determine whether a call is fraudulent, a means is used to convert voice data into text and compare it with fraudulent call patterns, and a vector database is used to determine whether the call is fraudulent. In this way, a system is provided that allows elderly people to use the telephone with peace of mind.

[0006] "Generative AI" is an AI technology that includes advanced algorithms for interacting with users and generating information.

[0007] A "telephone terminal" is a communication device that has a voice call function, and includes a landline phone and a feature phone.

[0008] A "scam call" is a call made to defraud someone of information or money for fraudulent purposes.

[0009] "Means for determining" refers to the function of analyzing voice and text data and determining the possibility of a fraudulent call based on specific conditions.

[0010] "Means for automatic response" refers to the ability of artificial intelligence to automatically generate and respond to specific responses without human intervention.

[0011] "Means for transferring calls" refers to the function of directly connecting the caller with the elderly person.

[0012] "Elderly people living alone" refers primarily to people aged 60 or older who live alone.

[0013] "Means for converting voice data into text" refers to a function that generates text from voice using voice recognition technology.

[0014] A "fraudulent call pattern" is a characteristic pattern created based on data on past fraudulent calls, and defines in detail the type of fraud and the method used.

[0015] A "vector database" is a database for efficiently searching and storing high-dimensional data points (e.g., word embeddings). [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention uses a telephone terminal equipped with generative artificial intelligence to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. A specific embodiment of this system is described in detail below.

[0038] System configuration

[0039] The system is built around a telephone terminal with a built-in generative artificial intelligence module. The main components are:

[0040] 1. Telephone terminal:

[0041] Ringing function when receiving a call

[0042] Ability to record incoming calls in real time and send the audio data to an analysis module

[0043] 2. Generative artificial intelligence:

[0044] Natural language processing functions for user interaction

[0045] Automated response for fraudulent calls

[0046] Daily conversation function for elderly people living alone

[0047] 3. Vector Database:

[0048] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[0049] Program processing

[0050] The program for this system mainly executes processing in the following steps. A specific processing flow and operation example are shown below.

[0051] Receiving and analyzing calls

[0052] Device:

[0053] When an incoming call occurs, the device immediately rings and automatically answers after a specified number of rings.

[0054] As soon as the call begins, the audio data is recorded in real time and sent to the analysis module.

[0055] Identifying fraudulent calls

[0056] server:

[0057] Voice recognition technology is used to convert voice data sent from the terminal into text.

[0058] The text data is compared with a vector database to determine whether it matches the characteristic patterns of fraudulent calls.

[0059] If there is a high degree of match, the call is determined to be fraudulent and the response is handed over to generative artificial intelligence.

[0060] Response to fraudulent phone calls

[0061] Generative artificial intelligence:

[0062] If a call is determined to be fraudulent, the AI ​​automatically initiates a fraudulent call and response, generating responses to the fraudster's questions that will not affect the user and buy time.

[0063] Responses are flexibly changed to prevent elderly people from becoming victims by continuing the conversation with the scammer.

[0064] Normal call handling

[0065] Device:

[0066] If the call is determined not to be a scam, the device will transfer the call to the elderly person.

[0067] The senior citizen can continue the conversation by following normal call procedures.

[0068] Features for seniors living alone

[0069] User:

[0070] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[0071] Generative artificial intelligence:

[0072] The generative AI recognizes the command and automatically starts a conversation. The generative AI generates an appropriate response to the user's utterance and continues the conversation in a natural way.

[0073] Specific examples

[0074] Receiving and responding to fraudulent phone calls

[0075] Situation: An elderly person receives a fraudulent phone call at their home.

[0076] Device: The phone starts ringing. Because it is set to auto-answer mode, the call is automatically answered after three rings.

[0077] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[0078] Server: The call is determined to be fraudulent. The generative AI begins responding.

[0079] Generative AI: Buys time by responding to scammers with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[0080] Ability to talk to elderly people

[0081] Situation: An elderly person living alone feels lonely.

[0082] User: Say "Please talk to me" to the device.

[0083] Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[0084] User: "The flowers in my garden have been blooming beautifully lately."

[0085] Generator: "That's lovely. Can you tell me what kind of flowers bloomed?"

[0086] This invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI function for everyday conversations reduces the mental burden on elderly people living alone.

[0087] The processing flow will be explained below.

[0088] Step 1:

[0089] Device:

[0090] When powered on, the terminal boots the system.

[0091] During the startup process, a self-test is performed to verify that each module is operating correctly.

[0092] Loads the generative artificial intelligence module and establishes a connection with the built-in vector database.

[0093] Step 2:

[0094] Device:

[0095] When an incoming call occurs, a ring tone sounds to notify the user.

[0096] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[0097] As soon as the call begins, audio data recording begins and is sent to the analysis module in real time.

[0098] Step 3:

[0099] server:

[0100] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[0101] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[0102] If the score exceeds the threshold, the call is determined to be fraudulent.

[0103] Step 4:

[0104] Generative artificial intelligence:

[0105] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[0106] Generate questions and responses that will buy time for fraudsters without affecting the user.

[0107] Responses can be flexibly changed, and conversations with scammers are constantly recorded and analyzed.

[0108] Step 5:

[0109] Device:

[0110] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[0111] The normal call process will be resumed and the call will not be recorded.

[0112] Step 6:

[0113] User:

[0114] If an elderly person living alone feels lonely, the robot will issue a specific command (e.g., "talk to me") via voice command.

[0115] Device:

[0116] It recognizes commands and sends instructions to generative artificial intelligence.

[0117] Generative artificial intelligence:

[0118] It receives commands and starts a conversation with the user, using natural language processing to provide appropriate responses to what the user says.

[0119] The conversation content is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[0120] Step 7:

[0121] Device:

[0122] When the call or conversation ends, the communication records and analysis results are saved.

[0123] Safely shuts down the generative artificial intelligence module and puts it into standby mode until next use.

[0124] This system not only protects seniors from fraudulent phone calls, but also serves as a daily conversation partner, improving their sense of security and quality of life.

[0125] Example 1

[0126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] The risk of elderly people becoming victims of fraudulent phone calls is increasing. In addition, elderly people who live alone often feel lonely because they have no one to talk to on a daily basis. To solve these issues, a system is needed that can identify fraudulent calls, automatically answer them, and provide elderly people living alone with someone to talk to on a daily basis.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0129] In this invention, the server includes a means for ringing and switching to automatic answering when an incoming call is received, a means for recording voice data as soon as the call begins and sending it to an analysis module, a means for converting the received voice data into text and comparing it with fraudulent call characteristic patterns, a means for calculating the degree of match and automatically answering the call using a generative artificial intelligence if the call is determined to be fraudulent, a means for transferring the call to an elderly person if the call is not fraudulent, and a means for the generative artificial intelligence to converse when an elderly person living alone needs someone to talk to. This makes it possible to prevent damage caused by fraudulent calls and ensure the safety of the elderly. It also makes it possible to reduce the sense of loneliness felt by elderly people living alone.

[0130] "Means for ringing and switching to automatic answering when an incoming call occurs" refers to the function by which a telephone terminal detects an incoming call and automatically starts the call after ringing a predetermined number of times.

[0131] "Means for recording voice data as soon as a call starts and sending it to an analysis module" refers to a function for recording voice in real time as soon as a call starts and sending that data to a module for data analysis.

[0132] "Means for converting received voice data into text and comparing it with fraudulent call characteristic patterns" refers to a function that uses voice recognition technology to convert recorded voice data into text data and compares the text with predefined fraudulent call characteristic patterns.

[0133] "Means for calculating the degree of match and for the generative artificial intelligence to automatically respond if the call is determined to be fraudulent" refers to a function that evaluates the possibility of a fraudulent call based on the degree of match of the matching results, and for the generative artificial intelligence to automatically respond if the degree of match is high.

[0134] "Means of transferring calls to elderly people if they are not fraudulent calls" refers to a function that notifies elderly people of calls that are determined to be non-scam calls, allowing elderly people to actually make the calls.

[0135] "Means for generative AI to converse when an elderly person living alone wants someone to talk to" refers to the function of generative AI to engage in natural dialogue when an elderly person living alone utters a specific command.

[0136] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence to provide elderly people with everyday conversation partners while protecting them from fraudulent phone calls. The system's main components are a telephone terminal, a server, generative artificial intelligence, and a vector database.

[0137] Main components

[0138] 1. Telephone terminal

[0139] Device:

[0140] When an incoming call occurs, the device will ring and after a specified number of rings (e.g., three times) will switch to auto-answer mode.

[0141] As soon as the call begins, the audio data is recorded in real time and sent to the server.

[0142] 2. Server

[0143] server:

[0144] It receives voice data sent from the device and converts it into text using voice recognition technology (e.g., Google (registered trademark) Cloud Speech-to-Text).

[0145] The text data is compared with characteristic patterns of fraudulent calls stored in a vector database (e.g., ElasticSearch (registered trademark)) and the degree of match is calculated.

[0146] If the degree of match is high, the call is determined to be fraudulent and the generative artificial intelligence is notified.

[0147] 3. Generative artificial intelligence

[0148] Generative artificial intelligence:

[0149] If a call is determined to be fraudulent, the AI ​​will automatically respond to the call, generating responses to the fraudster's questions that will not affect the user and buy time.

[0150] When an elderly person living alone wants someone to talk to, the system provides natural dialogue in response to user instructions.

[0151] 4. Vector Database

[0152] Vector Database:

[0153] It stores characteristic patterns of fraudulent calls and compares them with text data sent from the server to calculate the degree of match.

[0154] Specific examples

[0155] Receiving and responding to fraudulent phone calls

[0156] Situation: An elderly person receives a fraudulent phone call at their home.

[0157] 1. Device: The phone starts ringing. After three rings, the phone automatically answers the call because it is set to auto-answer mode.

[0158] 2. Terminal: Records audio data in real time and sends it to the server.

[0159] 3. Server: The received voice data is converted into text using Google Cloud Speech-to-Text, and compared with fraudulent call patterns stored in Elasticsearch, a vector database.

[0160] 4. Server: Calculates the degree of match and determines whether the call is fraudulent. Notifies the generative AI.

[0161] 5. Generative AI: The system responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" and continues the conversation.

[0162] Ability to talk to elderly people

[0163] Situation: An elderly person living alone feels lonely.

[0164] 1. User: Speaks to the device, "Please talk to me."

[0165] 2. Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[0166] 3. User: "The flowers in my garden have been blooming beautifully lately."

[0167] 4. Generative AI: "That's lovely! Tell me what kind of flowers bloomed."

[0168] In this way, this invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI-based everyday conversation function reduces the mental burden on elderly people living alone.

[0169] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0170] Step 1:

[0171] Device:

[0172] Input: Incoming phone signal.

[0173] What it does: Rings when an incoming call comes in.

[0174] Output: The phone will start ringing. After the specified number of rings (e.g. 3), the phone will switch to auto-answer mode.

[0175] Specific operation: After three rings, the message "Now entering auto answer mode" will be played.

[0176] Step 2:

[0177] Device:

[0178] Input: Call initiation signal and audio data.

[0179] How it works: When a call starts, the audio data is recorded in real time and sent to the server.

[0180] Output: Recorded audio data.

[0181] Specific operation: Recording begins as soon as the call begins, and the voice data is divided into a certain number of data packets, encoded, and transferred to the server.

[0182] Step 3:

[0183] server:

[0184] Input: Audio data sent from the device.

[0185] How it works: Uses speech recognition technology (e.g., Google Cloud Speech-to-Text) to convert audio data into text.

[0186] Output: Textualized data.

[0187] Specific operation: The server receives the voice data and calls a speech recognition API to convert the data into text.

[0188] Step 4:

[0189] server:

[0190] Input: Textual data.

[0191] Operation: Compare the text data with the characteristic patterns of fraudulent calls stored in a vector database (e.g., Elasticsearch). Calculate the degree of match of the matching results.

[0192] Output: Matching result.

[0193] Specific operation: After the server receives the text data, it inputs the feature pattern vector into a matching algorithm, and if the degree of match is above a certain level, it generates a result that says "possibly a fraudulent call."

[0194] Step 5:

[0195] server:

[0196] Input: Matching result.

[0197] Operation: If there is a high match, the generative artificial intelligence is notified.

[0198] Output: Notification if the call is determined to be fraudulent.

[0199] Specific operation: The judgment result is sent to generative artificial intelligence, and a notification is sent stating that "there is a high possibility that the call is fraudulent."

[0200] Step 6:

[0201] Generative artificial intelligence:

[0202] Enter: Scam call notification.

[0203] How it works: If a call is determined to be fraudulent, the AI ​​automatically responds, generating a response that won't affect the user and buys time for the fraudster.

[0204] Output: Response to the scammer.

[0205] What it does: Generates enticing statements, such as "That's terrible! Which outstanding balance are you talking about?"

[0206] Step 7:

[0207] Device:

[0208] Input: If the server determines that the call is not a scam.

[0209] Operation: If the call is determined not to be a scam, the call is transferred to an elderly person.

[0210] Output: Call transfer.

[0211] Specific actions: The system conveys the message "This call is normal. Please continue speaking" to the elderly person and switches the call over to the elderly person.

[0212] Step 8:

[0213] User:

[0214] Enter: If you feel lonely.

[0215] Action: Say "Let me talk to you" to the device.

[0216] Output: The command passed.

[0217] Specific action: The user commands the device to "be a conversation partner."

[0218] Step 9:

[0219] Generative artificial intelligence:

[0220] Input: Commands from the user.

[0221] How it works: When an elderly person living alone needs someone to talk to, the AI ​​automatically starts a conversation.

[0222] Output: Natural dialogue.

[0223] Specific behavior: The generative AI responds, "Hello! What would you like to talk about today?" and provides an appropriate response to the user's statement, such as, "The flowers in the garden have been blooming beautifully recently," and continues, "That's wonderful. Please tell me what kind of flowers have bloomed."

[0224] This allows elderly people to use the telephone with peace of mind and reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud, and the everyday conversation function using generative artificial intelligence reduces the mental burden on elderly people living alone.

[0225] (Application example 1)

[0226] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0227] This invention relates to the provision of a system for protecting elderly people from fraud. Specifically, the objective is to protect elderly people from fraud by detecting possible fraud and automatically taking action, and to provide support for elderly people to enjoy shopping and staying in stores with peace of mind, thereby reducing the sense of loneliness felt by elderly people when they are alone.

[0228] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0229] In this invention, the server includes means for determining the possibility of fraud, means for a generative artificial intelligence to automatically respond to the determined fraud, means for transferring communication to an elderly person if fraud is not an attempt, means for the generative artificial intelligence to converse with an elderly person living alone when the elderly person wants someone to talk to, means for providing navigation and product explanations to elderly people visiting a store, means for analyzing conversations with suspicious people near the store and detecting fraud and means for issuing a warning based on the detection result, and means for the generative artificial intelligence to support the conversation when the elderly person is looking for someone to talk to. This protects elderly people from fraud, improves their shopping experience in stores, and reduces the sense of loneliness they feel when they are alone.

[0230] "Generative AI" is an AI technology that has the ability to generate new data and answers based on input data.

[0231] A "communication terminal" is an electronic device for sending and receiving voice and data.

[0232] "Fraud" is the act of deceiving others through dishonest means and stealing their property or information.

[0233] A "means of judgment" is a method or technique for making an appropriate judgment about an event or data.

[0234] "Means for automatic response" is a function that automatically generates a response to an external input without human intervention.

[0235] A "relaying means" is a method or device for relaying specific information or communication to a designated party.

[0236] "When someone wants someone to talk to" refers to a situation in which the other person wants to have a conversation.

[0237] "Navigation" is a function that provides guidance on the route to a destination.

[0238] "Product Description" is detailed information about the characteristics and usage of the product being offered.

[0239] "Means of analysis" refers to methods and techniques for analyzing input data in detail and extracting meaning and patterns.

[0240] "Warning means" means a method or technique for notifying people of a particular danger or abnormality.

[0241] "Means to support conversation" are auxiliary functions to facilitate smooth dialogue with the other person.

[0242] The system of the present invention uses a communication terminal equipped with generative artificial intelligence to protect elderly people from fraud and provide elderly people living alone with a daily conversation partner. It also provides support for elderly people to enjoy shopping and staying in physical stores with peace of mind. Specific embodiments of the present invention are described in detail below.

[0243] System configuration

[0244] The system is built around a communications terminal with a built-in generative artificial intelligence module, and its main components are as follows:

[0245] 1. Communication terminal:

[0246] Ability to receive communications and perform specified actions

[0247] A function that analyzes received voice data in real time and sends it to the server

[0248] 2. Generative artificial intelligence:

[0249] Natural language processing functions for user interaction

[0250] Automatic fraud detection function

[0251] Daily conversation function for elderly people living alone

[0252] Functions that provide in-store navigation and product information

[0253] 3. Server:

[0254] A function that stores characteristic patterns of fraudulent activity, converts voice data into text, and compares it with those patterns

[0255] 4. Warning system:

[0256] A function that analyzes interactions with suspicious people near stores, detects fraudulent activity, and issues a warning.

[0257] 5. Interface:

[0258] A user interface designed to make communication devices easier for the elderly to operate

[0259] Hardware and software used

[0260] Hardware:

[0261] Communication device: A smartphone with audio input (microphone) and audio output (speaker)

[0262] software:

[0263] Speech recognition: using HuggingFace's transformer pipeline

[0264] Natural Language Processing: Using spaCy

[0265] Generative AI: Uses OpenAI's (registered trademark) GPT-3 (registered trademark).5-turbo model

[0266] Data processing and calculation

[0267] The server uses a speech recognition module to convert voice data into text data. It then uses a natural language processing module to analyze the text data and compare it with characteristic patterns of fraudulent activity. The generative artificial intelligence module generates natural dialogue and acts as a pseudo-conversational partner for the elderly. It also provides in-store navigation and product information to help seniors enjoy their shopping experience.

[0268] Specific examples

[0269] Dealing with fraud

[0270] Situation: An elderly person receives a fraudulent phone call at their home.

[0271] Communication terminal: When the call starts ringing, it will be set to auto-answer mode after the specified number of rings. The voice data will be analyzed in real time and sent to the server.

[0272] Server: Converts the received voice data into text and compares it with characteristic patterns of fraudulent activity.

[0273] Server: If fraudulent activity is determined, the generative artificial intelligence will automatically initiate a response, generating a response that will not have any impact on the fraudster.

[0274] In-store navigation and product explanations

[0275] Situation: An elderly person is searching for a product in a physical store.

[0276] User: Speak into the communication device, "Please tell me where I need navigation."

[0277] Generative AI: Asks, "Which section should I go to?" and provides navigation and product descriptions.

[0278] Prompt Sentence Examples

[0279] Dealing with fraud: "I was told I have outstanding bills. What should I do?"

[0280] In-store navigation: "Tell me where the milk is."

[0281] Conversation between an elderly person living alone: ​​"The flowers in my garden have been blooming beautifully recently."

[0282] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0283] Step 1:

[0284] The communication terminal receives a communication and rings. After a specified number of rings, it switches to an auto-answer mode, where the input includes the incoming voice and the output is that the auto-answer mode is activated.

[0285] Step 2:

[0286] The communication terminal records the received voice data in real time and transmits the data to the server. The input includes the received voice data, and the output includes the transmitted voice data.

[0287] Step 3:

[0288] The server uses a speech recognition module to convert the transmitted voice data into text data, where the input includes the voice data and the output is the text data.

[0289] Step 4:

[0290] The server uses a natural language processing module (e.g., spaCy) to analyze the generated text data and compare it with fraud feature patterns, where the input includes the text data and a fraud pattern database, and the output generates a match determination for fraud.

[0291] Step 5:

[0292] The server determines whether the fraudulent activity is occurring, and if a high degree of match is found, the generative artificial intelligence initiates an automatic response. The input includes the result of the match determination, and the output is an automatic response message.

[0293] Step 6:

[0294] The server uses generative artificial intelligence to generate an automatic response message and sends it back to the communication terminal. The input includes the match determination result and a prompt sentence, and the automatic response message is generated as the output.

[0295] Step 7:

[0296] The communication terminal plays the received automatic response message and makes an appropriate response to the fraudulent activity, where the input includes the automatic response message from the server and the output is the played response message.

[0297] Step 8:

[0298] If it is not fraudulent, the communication terminal transfers the call to the elderly person. The input includes the result of the match determination, and the output is that the call is transferred to the elderly person.

[0299] Step 9:

[0300] When an elderly person needs someone to talk to, they say "please talk to me" through the communication terminal and send it to the server. The input includes the elderly person's voice instruction, and the output is the transmission of voice data to the server.

[0301] Step 10:

[0302] The server uses generative artificial intelligence to analyze the user's instructions and initiate a conversation with the elderly, where the input includes voice instructions and prompts, and the output generates a conversational content.

[0303] Step 11:

[0304] The server sends the generated conversation content back to the communication terminal, which then plays it back. The input includes the generated conversation content, and the output is a conversation with the elderly person.

[0305] Step 12:

[0306] When an elderly person needs navigation or product explanations in a store, they can say "Tell me where I need navigation" through a communication terminal and send it to the server. The input includes the elderly person's voice instructions, and the output is the transmission of voice data to the server.

[0307] Step 13:

[0308] The server uses generative artificial intelligence to analyze the user's instructions and provide navigation and product descriptions, where inputs include voice instructions and store data, and outputs generate navigation and product descriptions.

[0309] Step 14:

[0310] The server sends the generated navigation content and product description back to the communication terminal, which then plays them. The input includes the generated navigation content and product description, and the output is a voice guide for the elderly.

[0311] Step 15:

[0312] When an elderly person is approached by a suspicious person near a store, the communication terminal receives the conversation and transmits it to the server in real time. The input includes the received voice data, and the output is the transmission of the voice data to the server.

[0313] Step 16:

[0314] The server analyzes the received voice data using a natural language processing module to determine the likelihood of fraud, where the input includes the voice data and a fraud pattern database, and the output generates a fraud match determination.

[0315] Step 17:

[0316] If the match is determined to be high, the server sends a warning message back to the communication terminal, which then plays it back. The input includes the match determination result and the warning message, and the output is the played warning message.

[0317] Step 18:

[0318] If the degree of match is determined to be low, the communication terminal maintains its normal state without issuing any particular warning. The input includes the result of the degree of match determination, and the output maintains the normal state without any warning.

[0319] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0320] The present invention uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a daily conversation partner. A specific embodiment of this system is described in detail below.

[0321] System configuration

[0322] The system is built around a phone terminal with a built-in generative artificial intelligence module and emotion engine. The main components are:

[0323] 1. Telephone terminal:

[0324] Ringing function when receiving a call

[0325] Ability to record incoming calls in real time and send the audio data to an analysis module

[0326] 2. Generative artificial intelligence:

[0327] Natural language processing functions for user interaction

[0328] Automated response for fraudulent calls

[0329] Daily conversation function for elderly people living alone

[0330] 3. Emotion Engine:

[0331] A function that analyzes emotions from user and fraudster voice data and optimizes generative AI responses

[0332] 4. Vector Database:

[0333] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[0334] Program processing

[0335] The program of this system executes the process in the following procedure.

[0336] Receiving and analyzing calls

[0337] Device:

[0338] When an incoming call occurs, a ring tone sounds to notify the user.

[0339] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[0340] As soon as a call is initiated, audio data is recorded in real time and sent to the analysis module.

[0341] Identifying fraudulent calls

[0342] server:

[0343] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[0344] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[0345] If the score exceeds the threshold, the call is determined to be fraudulent.

[0346] Response to fraudulent phone calls

[0347] Generative artificial intelligence:

[0348] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[0349] Generate questions and responses that will buy time for fraudsters without affecting the user.

[0350] The emotion engine analyzes emotions from the fraudster's voice and optimizes responses accordingly.

[0351] Normal call handling

[0352] Device:

[0353] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[0354] The normal call process will be resumed and the call will not be recorded.

[0355] Features for seniors living alone

[0356] User:

[0357] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[0358] Device:

[0359] It recognizes commands and sends instructions to generative artificial intelligence.

[0360] Generative artificial intelligence:

[0361] Receives commands and initiates a conversation with the user.

[0362] The emotion engine analyzes emotions from the user's voice and generates appropriate responses based on those emotions.

[0363] The content of the conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[0364] Specific examples

[0365] Receiving and responding to fraudulent phone calls

[0366] Situation: An elderly person receives a fraudulent phone call at their home.

[0367] Device: The phone starts ringing and switches to auto-answer mode after three rings.

[0368] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[0369] Server: The call is determined to be fraudulent. Generative AI begins responding.

[0370] Generative AI: Responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[0371] Ability to talk to elderly people

[0372] Situation: An elderly person living alone feels lonely.

[0373] User: Say "Talk to me" to the device.

[0374] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[0375] User: "The flowers in my garden have been blooming beautifully lately."

[0376] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[0377] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[0378] The processing flow will be explained below.

[0379] Step 1:

[0380] Device:

[0381] When powered on, the device boots up the system.

[0382] During the boot process, a self-diagnosis is performed to ensure that each module is working properly.

[0383] It loads a generative artificial intelligence module and an emotion engine, and establishes a connection with the built-in vector database.

[0384] Step 2:

[0385] Device:

[0386] When an incoming call occurs, the phone will ring to notify the user.

[0387] When the ring tone is repeated a specified number of times (e.g. 3 times), it will switch to auto answer mode.

[0388] As soon as the call begins, audio data begins recording and is sent to the analysis module in real time.

[0389] Step 3:

[0390] server:

[0391] It receives voice data sent from the device and converts it into text data using voice recognition technology.

[0392] The text data is compared with fraudulent call patterns registered in a vector database and the degree of match is scored.

[0393] If the score exceeds the threshold, the call is deemed to be fraudulent.

[0394] Step 4:

[0395] Generative artificial intelligence:

[0396] If the call is determined to be fraudulent, an automatic answering mode will be initiated.

[0397] The idea is to generate questions and responses that buy the scammers time without impacting the user.

[0398] The emotion engine analyzes the emotions in the scammer's voice and optimizes the response accordingly.

[0399] Step 5:

[0400] Device:

[0401] If the call is determined not to be a scam, the call will be transferred to an elderly user.

[0402] The normal call process will be used and the call will not be recorded.

[0403] Step 6:

[0404] User:

[0405] If an elderly person living alone feels lonely, they can issue a specific command (e.g., "talk to me") via voice command.

[0406] Device:

[0407] It recognizes commands and sends instructions to generative artificial intelligence.

[0408] Generative artificial intelligence:

[0409] It receives commands, initiates a conversation with the user, and uses natural language processing to provide appropriate responses to what the user says.

[0410] The emotion engine analyzes the emotions from the user's voice and generates appropriate responses based on those emotions.

[0411] The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[0412] Step 7:

[0413] Device:

[0414] Once the call or conversation ends, the communication records and analysis results are saved.

[0415] The generative artificial intelligence module will be safely shut down and put into standby mode until next use.

[0416] Specific examples

[0417] Receiving and responding to fraudulent phone calls

[0418] Situation: An elderly person receives a fraudulent phone call at their home.

[0419] Device: The phone will start ringing and will switch to auto-answer mode after three rings.

[0420] Server: Analyzes the received voice data in real time, converts it into text, and compares it with the characteristic patterns of fraudulent calls.

[0421] Server: The call is determined to be fraudulent. The generative AI begins responding.

[0422] Generative AI: Responding to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[0423] Ability to talk to elderly people

[0424] Situation: An elderly person living alone feels lonely.

[0425] User: Speak into the device, "Talk to me."

[0426] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[0427] User: "The flowers in my garden have been blooming beautifully lately."

[0428] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[0429] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[0430] Example 2

[0431] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0432] There is a need for a communication system that reduces the risk of elderly people becoming victims of fraudulent phone calls and allows elderly people living alone to live safely and comfortably without feeling lonely. Fraudulent phone calls are becoming increasingly sophisticated, making it extremely difficult for elderly people to identify them. Furthermore, elderly people living alone often have no one to talk to on a daily basis, which makes them prone to psychological loneliness. It is necessary to solve these problems simultaneously.

[0433] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0434] In this invention, the server includes means for ringing when a call is received, means for recording voice data in real time and sending it to an analysis module, means for determining whether a call is likely to be a fraudulent call, means for a generative artificial intelligence to automatically respond to the determined fraudulent call, means for analyzing emotions from the user's voice data and optimizing the response, means for transferring the call to the user if the call is not a fraudulent call, and means for the generative artificial intelligence to converse when a user living alone requests someone to talk to. This enables early detection and response of fraudulent calls, not only protecting the elderly from fraud but also providing communication that reduces their everyday feelings of loneliness.

[0435] The "means for ringing when a call is received" is a function that notifies the user by voice when a call is received.

[0436] The "means for recording voice data in real time and transmitting it to an analysis module" is a function for instantly recording voice during a call and transmitting the data for analysis.

[0437] The "means for determining the possibility of a fraudulent call" is a function that includes an algorithm that analyzes the content of a received call and determines whether it is a fraudulent call.

[0438] "Means for automated response by generative artificial intelligence" is a function that automatically generates appropriate responses to identified fraudulent phone calls and continues the dialogue with the fraudster.

[0439] "Means for analyzing emotions from user voice data and optimizing responses" is a function that analyzes the voice during a call to determine the user's emotional state and generates appropriate response content based on that.

[0440] The "means of transferring the call to the user if it is not a fraudulent call" is a function of directly connecting the call to the user if it is determined that the call is not a fraudulent call.

[0441] "A means for generative AI to converse when a user living alone wants someone to talk to" is a function that, when a user feels lonely, receives that command and the AI ​​starts a dialogue and communicates with the user.

[0442] "Means for converting voice data to text" refers to a function that includes an algorithm for converting the voice content of a call into text information.

[0443] The "means for comparing with fraudulent call patterns" is a function for comparing the converted text with characteristic patterns of fraudulent calls registered in advance and evaluating the degree of match.

[0444] The "feature vector database" is a database that stores vector data that quantifies the characteristics of fraudulent calls and is used for comparative analysis.

[0445] This invention is a system that protects elderly people from fraudulent phone calls and provides elderly people living alone with a regular conversation partner. A specific implementation is built around a telephone terminal equipped with a generative artificial intelligence module and an emotion engine. The system configuration is as follows:

[0446] System Components

[0447] 1. Telephone terminal

[0448] Hardware: Includes a speaker that rings when a call comes in, a microphone that records audio in real time, and a communication module that sends the recorded data to the analysis module.

[0449] Software: Equipped with automatic answering function and command recognition function (e.g. Amazon Alexa Voice Service).

[0450] 2. Generative artificial intelligence

[0451] Hardware: A server with a powerful processor and sufficient memory.

[0452] Software: AI modules with natural language processing and dialogue generation capabilities, such as automated answering of fraudulent phone calls and everyday conversations with the elderly (e.g., using models such as GPT-3).

[0453] 3. Emotion Engine

[0454] Software: Algorithms that analyze emotions from voice data and optimize generative artificial intelligence responses (e.g., IBM Watson® Tone Analyzer).

[0455] 4. Vector Database

[0456] Hardware: A database server with high-speed access.

[0457] Software: A database that quantifies and stores characteristic patterns of fraudulent calls. APIs (e.g., Cosine Similarity algorithms) that convert voice data into text and match it with the patterns.

[0458] How it works

[0459] Receiving and analyzing calls

[0460] When a call comes in, the device will ring to notify the user. If there is no answer after three rings, the device will switch to auto-answer mode. Once the call is initiated, the device will record audio data in real time and send it to the analysis module.

[0461] Identifying fraudulent calls

[0462] The server receives the voice data sent from the device and converts it into text data using speech recognition technology. The converted text data is compared with fraudulent call patterns registered in a vector database and a score is assigned to determine the degree of match. If the score exceeds a threshold, the call is determined to be fraudulent.

[0463] Responding to fraudulent phone calls

[0464] If the generative AI determines that the call is a scam, it will initiate an automatic response mode, generating questions and responses to buy the scammer time without impacting the user. The emotion engine analyzes the emotions in the scammer's voice and optimizes the response based on that information.

[0465] Normal call handling

[0466] If the device determines that the call is not a scam, it will transfer the call to the user. The call will not be recorded.

[0467] Conversation starter for elderly people living alone

[0468] If the user feels lonely, they can say to the terminal, "Please talk to me."

[0469] The terminal recognizes this command and sends instructions to the generative artificial intelligence.

[0470] The generative AI receives instructions and begins a conversation with the user. The emotion engine analyzes the user's voice and generates appropriate responses based on their emotions. The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[0471] Specific examples

[0472] Receiving and responding to fraudulent phone calls

[0473] Situation: An elderly person receives a fraudulent phone call at their home.

[0474] Device: The phone starts ringing. After three rings, it switches to auto-answer mode. During this time, the device's display panel will show "Incoming call" and the LED will flash.

[0475] Server: Processes the received voice data in real time and converts it into text. It compares it with the characteristic patterns of fraudulent calls. As a result, it determines that "this is likely to be a fraud."

[0476] Generative AI: The system responds to the fraudster by saying, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the fraudster's emotions. For example, if the fraudster is frustrated, the system responds in a calming tone.

[0477] Ability to talk to elderly people

[0478] Situation: An elderly person living alone feels lonely.

[0479] User: Say "Talk to me" to the device.

[0480] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[0481] User: "The flowers in my garden have been blooming beautifully lately."

[0482] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[0483] Example of input prompt for the generative AI model to be used

[0484] Prompt 1: Analyze the characteristics of the scam call and inform the user that it is a scam.

[0485] Prompt 2: Please start a fun conversation with an elderly person who lives alone and feels lonely.

[0486] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0487] Program processing steps

[0488] Step 1:

[0489] Input: When a call comes in, a signal is sent to the telephone terminal.

[0490] Terminal: When a call comes in, the terminal will ring to notify the user, and the display panel will show "Incoming call" and the LED will flash.

[0491] Output: The user hears the phone ringing or sees the device's display panel and a flashing LED.

[0492] Step 2:

[0493] Input: After three rings or when the user is determined not to answer the call.

[0494] Terminal: If there is no answer after three rings, the device will switch to auto-answer mode. As soon as the call starts, it will start recording the audio data in real time. The recorded audio data will be immediately sent to the analysis module.

[0495] Output: The recorded audio data is sent to the analysis module.

[0496] Step 3:

[0497] Input: Audio data sent from the device.

[0498] Server: The server converts the received voice data into text data using voice recognition technology (e.g., Google Speech-to-Text API).

[0499] Output: Text data is generated.

[0500] Step 4:

[0501] Input: Text data sent from the server.

[0502] Server: Converts the text data into vector format and matches it with a vector database containing patterns of fraudulent calls, scoring the match using, for example, the Cosine Similarity algorithm.

[0503] Output: A match score is generated and a scam call determination is given.

[0504] Step 5:

[0505] Input: Match score and fraud call determination result.

[0506] Generative AI: If the score exceeds a threshold, it is determined to be a fraudulent call and an automatic response mode is initiated. Questions and responses are generated for the fraudster that will buy time without affecting the user. An emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the emotions in the fraudster's voice and optimizes the response based on that information. For example, it generates a response such as, "That's terrible. Can you tell me more about which outstanding balance you're referring to?"

[0507] Output: An automated response to the scammer is generated and sent.

[0508] Step 6:

[0509] Inputs: Match score and if the call was determined not to be fraudulent.

[0510] Terminal: If the call is not a fraudulent call, the terminal will transfer the call to the user, notifying the user of the call via voice guidance and a display panel.

[0511] Output: The call is transferred to the user, who prepares to answer the call.

[0512] Step 7:

[0513] Input: If the user feels lonely, they can speak a voice command to the device saying, "Talk to me."

[0514] Terminal: Recognizes voice commands and sends instructions to the generative artificial intelligence.

[0515] Output: The recognition results of the voice command are sent to the generative artificial intelligence.

[0516] Step 8:

[0517] Input: The recognition result of the voice command.

[0518] Generative AI: Initiates a conversation with the user upon receiving instructions. The emotion engine analyzes the user's voice and generates an appropriate response based on the user's emotion. For example, "Hello! What would you like to talk about today?" It also responds to user comments with, "That's lovely. Tell me what kind of flowers have bloomed."

[0519] Output: A personalized conversation is provided to the user.

[0520] Specific examples

[0521] Prompt 1: "Analyze the characteristics of scam calls and notify the user that they are scams."

[0522] Prompt 2: "Please start a fun conversation with an elderly person who lives alone and feels lonely."

[0523] (Application example 2)

[0524] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0525] The number of victims of fraudulent phone calls targeting elderly people is increasing, and as a countermeasure, a system that can identify fraudulent calls and automatically respond to them is needed. Furthermore, to reduce the sense of loneliness felt by elderly people living alone, a system that can provide daily conversation partners is also needed. This invention aims to protect elderly people from fraudulent phone calls and provide them with daily conversation partners by utilizing generative artificial intelligence and an emotion engine.

[0526] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for determining whether the call is a fraudulent call, means for the generative artificial intelligence to automatically respond to the determined fraudulent call, means for transferring the call to an elderly person if the call is not a fraudulent call, means for the generative artificial intelligence to converse with an elderly person living alone when they need someone to talk to, means for analyzing voice data and detecting emotions, and means for optimizing the response content according to the detected emotions. This allows elderly people to avoid falling victim to fraudulent calls, further reduce their sense of loneliness, and live their lives with peace of mind.

[0527] "Generative AI" is AI that has the ability to generate language and content based on user input.

[0528] A "telephone terminal" is a device with a calling function that supports calls with elderly people.

[0529] "Fraud call detection" is the process of determining whether a call is likely to be fraudulent based on the received voice data.

[0530] "Means for automatic response" refers to a system function in which artificial intelligence automatically generates and responds to identified fraudulent calls.

[0531] "Means of transferring calls to elderly people" is a function that connects normal calls that are determined not to be fraudulent calls to elderly people.

[0532] "Means for generative AI to converse when someone wants someone to talk to" refers to a function that allows AI to automatically start a conversation and respond when an elderly person requests a conversation.

[0533] The "means for analyzing voice data" is a function for analyzing received voice data and converting it into text.

[0534] "Means for detecting emotions" refers to a function that reads emotions from the content and tone of analyzed voice data.

[0535] "Means for optimizing response content" is a function that generates and provides the optimal response based on the detected emotion.

[0536] "Fraud call patterns" are a collection of data summarizing the characteristics and commonalities of past fraud calls.

[0537] A "vector database" is a database that stores and manages data in vector format and allows for comparison of similarities and features.

[0538] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. The specific system configuration and various means for realizing this are described in detail below.

[0539] System configuration

[0540] The system consists of the following main components:

[0541] Telephone terminal: A device with a calling function. Suitable for use by the elderly.

[0542] Generative artificial intelligence (AI): An AI module with natural language processing capabilities.

[0543] Emotion engine: An engine that analyzes emotions from voice data and optimizes responses.

[0544] Vector database: A database that stores patterns of fraudulent calls.

[0545] Program processing

[0546] The program of this system is realized using the following hardware and software.

[0547] Hardware:

[0548] Smartphone

[0549] IoT device with speakerphone functionality

[0550] software:

[0551] Speech recognition module (speech_recognition library)

[0552] Natural language generation module (transformers library)

[0553] Sentiment analysis module (pipeline library)

[0554] Database management system (sqlite3)

[0555] The server first detects an incoming call from the user and acquires the voice data. Next, it analyzes this voice data, converts it into text, and compares it with a vector database to determine whether it is likely to be fraud. If it is determined to be fraud, the generative AI automatically responds. On the other hand, if it is determined not to be fraud, the call can be transferred directly to the elderly person.

[0556] Furthermore, the system's emotion engine analyzes emotions from voice data and optimizes responses based on those emotions. If an elderly person living alone says, "I want someone to talk to," the system recognizes this and initiates a conversation with the generative AI, generating an appropriate response.

[0557] Specific examples

[0558] Receiving and responding to fraudulent phone calls

[0559] Situation: An elderly person receives a fraudulent phone call at their home.

[0560] User: Import audio into the system as "input_audio.wav".

[0561] Server: Analyzes received voice data in real time and converts it into text.

[0562] Server: Compares with the characteristic patterns of fraudulent calls and determines that it is a fraudulent call.

[0563] Generative AI: Responds to the scammer with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[0564] Ability to talk to elderly people

[0565] Situation: An elderly person living alone feels lonely and wants someone to talk to.

[0566] User: Say "Talk to me" to the device.

[0567] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[0568] User: "The flowers in my garden have been blooming beautifully lately."

[0569] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[0570] Examples of specific prompts include:

[0571] "Scam call voice: Mom, it's me and I want to talk to you about an unpaid bill."

[0572] The present invention enables elderly people to live without becoming victims of fraudulent phone calls, and further reduces their sense of loneliness, allowing them to live with peace of mind.

[0573] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0574] Step 1:

[0575] Terminal: The phone starts ringing. It notifies the user of the incoming call and plays the ringtone. The input is the incoming call signal and the output is the ringtone.

[0576] Step 2:

[0577] Terminal: Switch to auto-answer mode after the specified number of rings (e.g. 3 times). Input is the number of rings, output is to switch to auto-answer mode.

[0578] Step 3:

[0579] Terminal: When a call is started, the terminal records the voice data in real time and sends it to the server. The input is the call start signal, and the output is the recorded voice data.

[0580] Step 4:

[0581] Server: Converts received voice data into text using a speech recognition module (speech_recognition library). The input is recorded voice data, and the output is text data. Specifically, the audio file is analyzed and the voice is converted into text.

[0582] Step 5:

[0583] Server: Compares the text data with a vector database to determine the likelihood of a fraudulent call. The input is text data, and the output is the result of the fraudulent call determination. Specifically, it compares the data with fraudulent call patterns registered in the vector database and scores the degree of match.

[0584] Step 6:

[0585] Server: If the call is determined to be fraudulent, an automatic response is generated using generative artificial intelligence (the transformers library). The input is the fraudulent call determination result and text data, and the output is the generated response. Specifically, questions and responses are generated for the fraudster.

[0586] Step 7:

[0587] Server: Sends the generated response to the device and plays it back to the fraudster. The input is the generated response, and the output is the played response. Specifically, the emotion engine analyzes the emotion from the fraudster's voice and optimizes the response based on the results.

[0588] Step 8:

[0589] Server: If the call is determined not to be a scam, the server transfers the call to the elderly person. The input is the result of the scam call determination, and the output is the call transfer.

[0590] Step 9:

[0591] Terminal: The elderly person utters a specific command, such as "Please be my conversation partner." The input is the command voice, and the output is command recognition.

[0592] Step 10:

[0593] Generative AI: Recognizes commands and starts a conversation. The input is a spoken command, and the output is a generated conversation starter. Specifically, it responds, "Hello! What would you like to talk about today?"

[0594] Step 11:

[0595] Terminal and Generative AI: The content of what the user says is analyzed in real time, and the emotion engine analyzes the emotion. The input is the user's voice, and the output is the result of the emotion analysis. Specifically, in response to the statement, "The flowers in the garden have been blooming beautifully recently," the response generated is, "That's lovely. Please tell me what kind of flowers have bloomed."

[0596] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0597] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0598] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0599] [Second embodiment]

[0600] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0601] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0602] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0603] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0604] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0606] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0607] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0608] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0609] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0610] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0611] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0612] The present invention uses a telephone terminal equipped with generative artificial intelligence to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. A specific embodiment of this system is described in detail below.

[0613] System configuration

[0614] The system is built around a telephone terminal with a built-in generative artificial intelligence module. The main components are:

[0615] 1. Telephone terminal:

[0616] Ringing function when receiving a call

[0617] Ability to record incoming calls in real time and send the audio data to an analysis module

[0618] 2. Generative artificial intelligence:

[0619] Natural language processing functions for user interaction

[0620] Automated response for fraudulent calls

[0621] Daily conversation function for elderly people living alone

[0622] 3. Vector Database:

[0623] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[0624] Program processing

[0625] The program for this system mainly executes processing in the following steps. A specific processing flow and operation example are shown below.

[0626] Receiving and analyzing calls

[0627] Device:

[0628] When an incoming call occurs, the device immediately rings and automatically answers after a specified number of rings.

[0629] As soon as the call begins, the audio data is recorded in real time and sent to the analysis module.

[0630] Identifying fraudulent calls

[0631] server:

[0632] Voice recognition technology is used to convert voice data sent from the terminal into text.

[0633] The text data is compared with a vector database to determine whether it matches the characteristic patterns of fraudulent calls.

[0634] If there is a high degree of match, the call is determined to be fraudulent and the response is handed over to generative artificial intelligence.

[0635] Response to fraudulent phone calls

[0636] Generative artificial intelligence:

[0637] If a call is determined to be fraudulent, the AI ​​automatically initiates a fraudulent call and response, generating responses to the fraudster's questions that will not affect the user and buy time.

[0638] Responses are flexibly changed to prevent elderly people from becoming victims by continuing the conversation with the scammer.

[0639] Normal call handling

[0640] Device:

[0641] If the call is determined not to be a scam, the device will transfer the call to the elderly person.

[0642] The senior citizen can continue the conversation by following normal call procedures.

[0643] Features for seniors living alone

[0644] User:

[0645] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[0646] Generative artificial intelligence:

[0647] The generative AI recognizes the command and automatically starts a conversation. The generative AI generates an appropriate response to the user's utterance and continues the conversation in a natural way.

[0648] Specific examples

[0649] Receiving and responding to fraudulent phone calls

[0650] Situation: An elderly person receives a fraudulent phone call at their home.

[0651] Device: The phone starts ringing. Because it is set to auto-answer mode, the call is automatically answered after three rings.

[0652] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[0653] Server: The call is determined to be fraudulent. The generative AI begins responding.

[0654] Generative AI: Buys time by responding to scammers with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[0655] Ability to talk to elderly people

[0656] Situation: An elderly person living alone feels lonely.

[0657] User: Say "Please talk to me" to the device.

[0658] Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[0659] User: "The flowers in my garden have been blooming beautifully lately."

[0660] Generator: "That's lovely. Can you tell me what kind of flowers bloomed?"

[0661] This invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI function for everyday conversations reduces the mental burden on elderly people living alone.

[0662] The processing flow will be explained below.

[0663] Step 1:

[0664] Device:

[0665] When powered on, the terminal boots the system.

[0666] During the startup process, a self-test is performed to verify that each module is operating correctly.

[0667] Loads the generative artificial intelligence module and establishes a connection with the built-in vector database.

[0668] Step 2:

[0669] Device:

[0670] When an incoming call occurs, a ring tone sounds to notify the user.

[0671] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[0672] As soon as the call begins, audio data recording begins and is sent to the analysis module in real time.

[0673] Step 3:

[0674] server:

[0675] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[0676] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[0677] If the score exceeds the threshold, the call is determined to be fraudulent.

[0678] Step 4:

[0679] Generative artificial intelligence:

[0680] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[0681] Generate questions and responses that will buy time for fraudsters without affecting the user.

[0682] Responses can be flexibly changed, and conversations with scammers are constantly recorded and analyzed.

[0683] Step 5:

[0684] Device:

[0685] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[0686] The normal call process will be resumed and the call will not be recorded.

[0687] Step 6:

[0688] User:

[0689] If an elderly person living alone feels lonely, the robot will issue a specific command (e.g., "talk to me") via voice command.

[0690] Device:

[0691] It recognizes commands and sends instructions to generative artificial intelligence.

[0692] Generative artificial intelligence:

[0693] It receives commands and starts a conversation with the user, using natural language processing to provide appropriate responses to what the user says.

[0694] The conversation content is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[0695] Step 7:

[0696] Device:

[0697] When the call or conversation ends, the communication records and analysis results are saved.

[0698] Safely shuts down the generative artificial intelligence module and puts it into standby mode until next use.

[0699] This system not only protects seniors from fraudulent phone calls, but also serves as a daily conversation partner, improving their sense of security and quality of life.

[0700] Example 1

[0701] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0702] The risk of elderly people becoming victims of fraudulent phone calls is increasing. In addition, elderly people who live alone often feel lonely because they have no one to talk to on a daily basis. To solve these issues, a system is needed that can identify fraudulent calls, automatically answer them, and provide elderly people living alone with someone to talk to on a daily basis.

[0703] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0704] In this invention, the server includes a means for ringing and switching to automatic answering when an incoming call is received, a means for recording voice data as soon as the call begins and sending it to an analysis module, a means for converting the received voice data into text and comparing it with fraudulent call characteristic patterns, a means for calculating the degree of match and automatically answering the call using a generative artificial intelligence if the call is determined to be fraudulent, a means for transferring the call to an elderly person if the call is not fraudulent, and a means for the generative artificial intelligence to converse when an elderly person living alone needs someone to talk to. This makes it possible to prevent damage caused by fraudulent calls and ensure the safety of the elderly. It also makes it possible to reduce the sense of loneliness felt by elderly people living alone.

[0705] "Means for ringing and switching to automatic answering when an incoming call occurs" refers to the function by which a telephone terminal detects an incoming call and automatically starts the call after ringing a predetermined number of times.

[0706] "Means for recording voice data as soon as a call starts and sending it to an analysis module" refers to a function for recording voice in real time as soon as a call starts and sending that data to a module for data analysis.

[0707] "Means for converting received voice data into text and comparing it with fraudulent call characteristic patterns" refers to a function that uses voice recognition technology to convert recorded voice data into text data and compares the text with predefined fraudulent call characteristic patterns.

[0708] "Means for calculating the degree of match and for the generative artificial intelligence to automatically respond if the call is determined to be fraudulent" refers to a function that evaluates the possibility of a fraudulent call based on the degree of match of the matching results, and for the generative artificial intelligence to automatically respond if the degree of match is high.

[0709] "Means of transferring calls to elderly people if they are not fraudulent calls" refers to a function that notifies elderly people of calls that are determined to be non-scam calls, allowing elderly people to actually make the calls.

[0710] "Means for generative AI to converse when an elderly person living alone wants someone to talk to" refers to the function of generative AI to engage in natural dialogue when an elderly person living alone utters a specific command.

[0711] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence to provide elderly people with everyday conversation partners while protecting them from fraudulent phone calls. The system's main components are a telephone terminal, a server, generative artificial intelligence, and a vector database.

[0712] Main components

[0713] 1. Telephone terminal

[0714] Device:

[0715] When an incoming call occurs, the device will ring and after a specified number of rings (e.g., three times) will switch to auto-answer mode.

[0716] As soon as the call begins, the audio data is recorded in real time and sent to the server.

[0717] 2. Server

[0718] server:

[0719] It receives voice data sent from the device and converts it into text using voice recognition technology (e.g., Google Cloud Speech-to-Text).

[0720] The text data is compared with characteristic patterns of fraudulent calls stored in a vector database (e.g., Elasticsearch) and the degree of match is calculated.

[0721] If the degree of match is high, the call is determined to be fraudulent and the generative artificial intelligence is notified.

[0722] 3. Generative artificial intelligence

[0723] Generative artificial intelligence:

[0724] If a call is determined to be fraudulent, the AI ​​will automatically respond to the call, generating responses to the fraudster's questions that will not affect the user and buy time.

[0725] When an elderly person living alone wants someone to talk to, the system provides natural dialogue in response to user instructions.

[0726] 4. Vector Database

[0727] Vector Database:

[0728] It stores characteristic patterns of fraudulent calls and compares them with text data sent from the server to calculate the degree of match.

[0729] Specific examples

[0730] Receiving and responding to fraudulent phone calls

[0731] Situation: An elderly person receives a fraudulent phone call at their home.

[0732] 1. Device: The phone starts ringing. After three rings, the phone automatically answers the call because it is set to auto-answer mode.

[0733] 2. Terminal: Records audio data in real time and sends it to the server.

[0734] 3. Server: The received voice data is converted into text using Google Cloud Speech-to-Text, and compared with fraudulent call patterns stored in Elasticsearch, a vector database.

[0735] 4. Server: Calculates the degree of match and determines whether the call is fraudulent. Notifies the generative AI.

[0736] 5. Generative AI: The system responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" and continues the conversation.

[0737] Ability to talk to elderly people

[0738] Situation: An elderly person living alone feels lonely.

[0739] 1. User: Speaks to the device, "Please talk to me."

[0740] 2. Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[0741] 3. User: "The flowers in my garden have been blooming beautifully lately."

[0742] 4. Generative AI: "That's lovely! Tell me what kind of flowers bloomed."

[0743] In this way, this invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI-based everyday conversation function reduces the mental burden on elderly people living alone.

[0744] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0745] Step 1:

[0746] Device:

[0747] Input: Incoming phone signal.

[0748] What it does: Rings when an incoming call comes in.

[0749] Output: The phone will start ringing. After the specified number of rings (e.g. 3), the phone will switch to auto-answer mode.

[0750] Specific operation: After three rings, the message "Now entering auto answer mode" will be played.

[0751] Step 2:

[0752] Device:

[0753] Input: Call initiation signal and audio data.

[0754] How it works: When a call starts, the audio data is recorded in real time and sent to the server.

[0755] Output: Recorded audio data.

[0756] Specific operation: Recording begins as soon as the call begins, and the voice data is divided into a certain number of data packets, encoded, and transferred to the server.

[0757] Step 3:

[0758] server:

[0759] Input: Audio data sent from the device.

[0760] How it works: Uses speech recognition technology (e.g., Google Cloud Speech-to-Text) to convert audio data into text.

[0761] Output: Textualized data.

[0762] Specific operation: The server receives the voice data and calls a speech recognition API to convert the data into text.

[0763] Step 4:

[0764] server:

[0765] Input: Textual data.

[0766] Operation: Compare the text data with the characteristic patterns of fraudulent calls stored in a vector database (e.g., Elasticsearch). Calculate the degree of match of the matching results.

[0767] Output: Matching result.

[0768] Specific operation: After the server receives the text data, it inputs the feature pattern vector into a matching algorithm, and if the degree of match is above a certain level, it generates a result that says "possibly a fraudulent call."

[0769] Step 5:

[0770] server:

[0771] Input: Matching result.

[0772] Operation: If there is a high match, the generative artificial intelligence is notified.

[0773] Output: Notification if the call is determined to be fraudulent.

[0774] Specific operation: The judgment result is sent to generative artificial intelligence, and a notification is sent stating that "there is a high possibility that the call is fraudulent."

[0775] Step 6:

[0776] Generative artificial intelligence:

[0777] Enter: Scam call notification.

[0778] How it works: If a call is determined to be fraudulent, the AI ​​automatically responds, generating a response that won't affect the user and buys time for the fraudster.

[0779] Output: Response to the scammer.

[0780] What it does: Generates enticing statements, such as "That's terrible! Which outstanding balance are you talking about?"

[0781] Step 7:

[0782] Device:

[0783] Input: If the server determines that the call is not a scam.

[0784] Operation: If the call is determined not to be a scam, the call is transferred to an elderly person.

[0785] Output: Call transfer.

[0786] Specific actions: The system conveys the message "This call is normal. Please continue speaking" to the elderly person and switches the call over to the elderly person.

[0787] Step 8:

[0788] User:

[0789] Enter: If you feel lonely.

[0790] Action: Say "Let me talk to you" to the device.

[0791] Output: The command passed.

[0792] Specific action: The user commands the device to "be a conversation partner."

[0793] Step 9:

[0794] Generative artificial intelligence:

[0795] Input: Commands from the user.

[0796] How it works: When an elderly person living alone needs someone to talk to, the AI ​​automatically starts a conversation.

[0797] Output: Natural dialogue.

[0798] Specific behavior: The generative AI responds, "Hello! What would you like to talk about today?" and provides an appropriate response to the user's statement, such as, "The flowers in the garden have been blooming beautifully recently," and continues, "That's wonderful. Please tell me what kind of flowers have bloomed."

[0799] This allows elderly people to use the telephone with peace of mind and reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud, and the everyday conversation function using generative artificial intelligence reduces the mental burden on elderly people living alone.

[0800] (Application example 1)

[0801] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0802] This invention relates to the provision of a system for protecting elderly people from fraud. Specifically, the objective is to protect elderly people from fraud by detecting possible fraud and automatically taking action, and to provide support for elderly people to enjoy shopping and staying in stores with peace of mind, thereby reducing the sense of loneliness felt by elderly people when they are alone.

[0803] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0804] In this invention, the server includes means for determining the possibility of fraud, means for a generative artificial intelligence to automatically respond to the determined fraud, means for transferring communication to an elderly person if fraud is not an attempt, means for the generative artificial intelligence to converse with an elderly person living alone when the elderly person wants someone to talk to, means for providing navigation and product explanations to elderly people visiting a store, means for analyzing conversations with suspicious people near the store and detecting fraud and means for issuing a warning based on the detection result, and means for the generative artificial intelligence to support the conversation when the elderly person is looking for someone to talk to. This protects elderly people from fraud, improves their shopping experience in stores, and reduces the sense of loneliness they feel when they are alone.

[0805] "Generative AI" is an AI technology that has the ability to generate new data and answers based on input data.

[0806] A "communication terminal" is an electronic device for sending and receiving voice and data.

[0807] "Fraud" is the act of deceiving others through dishonest means and stealing their property or information.

[0808] A "means of judgment" is a method or technique for making an appropriate judgment about an event or data.

[0809] "Means for automatic response" is a function that automatically generates a response to an external input without human intervention.

[0810] A "relaying means" is a method or device for relaying specific information or communication to a designated party.

[0811] "When someone wants someone to talk to" refers to a situation in which the other person wants to have a conversation.

[0812] "Navigation" is a function that provides guidance on the route to a destination.

[0813] "Product Description" is detailed information about the characteristics and usage of the product being offered.

[0814] "Means of analysis" refers to methods and techniques for analyzing input data in detail and extracting meaning and patterns.

[0815] "Warning means" means a method or technique for notifying people of a particular danger or abnormality.

[0816] "Means to support conversation" are auxiliary functions to facilitate smooth dialogue with the other person.

[0817] The system of the present invention uses a communication terminal equipped with generative artificial intelligence to protect elderly people from fraud and provide elderly people living alone with a daily conversation partner. It also provides support for elderly people to enjoy shopping and staying in physical stores with peace of mind. Specific embodiments of the present invention are described in detail below.

[0818] System configuration

[0819] The system is built around a communications terminal with a built-in generative artificial intelligence module, and its main components are as follows:

[0820] 1. Communication terminal:

[0821] Ability to receive communications and perform specified actions

[0822] A function that analyzes received voice data in real time and sends it to the server

[0823] 2. Generative artificial intelligence:

[0824] Natural language processing functions for user interaction

[0825] Automatic fraud detection function

[0826] Daily conversation function for elderly people living alone

[0827] Functions that provide in-store navigation and product information

[0828] 3. Server:

[0829] A function that stores characteristic patterns of fraudulent activity, converts voice data into text, and compares it with those patterns

[0830] 4. Warning system:

[0831] A function that analyzes interactions with suspicious people near stores, detects fraudulent activity, and issues a warning.

[0832] 5. Interface:

[0833] A user interface designed to make communication devices easier for the elderly to operate

[0834] Hardware and software used

[0835] Hardware:

[0836] Communication device: A smartphone with audio input (microphone) and audio output (speaker)

[0837] software:

[0838] Speech recognition: using HuggingFace's transformer pipeline

[0839] Natural Language Processing: Using spaCy

[0840] Generative AI: Using OpenAI's GPT-3.5-turbo model

[0841] Data processing and calculation

[0842] The server uses a speech recognition module to convert voice data into text data. It then uses a natural language processing module to analyze the text data and compare it with characteristic patterns of fraudulent activity. The generative artificial intelligence module generates natural dialogue and acts as a pseudo-conversational partner for the elderly. It also provides in-store navigation and product information to help seniors enjoy their shopping experience.

[0843] Specific examples

[0844] Dealing with fraud

[0845] Situation: An elderly person receives a fraudulent phone call at their home.

[0846] Communication terminal: When the call starts ringing, it will be set to auto-answer mode after the specified number of rings. The voice data will be analyzed in real time and sent to the server.

[0847] Server: Converts the received voice data into text and compares it with characteristic patterns of fraudulent activity.

[0848] Server: If fraudulent activity is determined, the generative artificial intelligence will automatically initiate a response, generating a response that will not have any impact on the fraudster.

[0849] In-store navigation and product explanations

[0850] Situation: An elderly person is searching for a product in a physical store.

[0851] User: Speak into the communication device, "Please tell me where I need navigation."

[0852] Generative AI: Asks, "Which section should I go to?" and provides navigation and product descriptions.

[0853] Prompt Sentence Examples

[0854] Dealing with fraud: "I was told I have outstanding bills. What should I do?"

[0855] In-store navigation: "Tell me where the milk is."

[0856] Conversation between an elderly person living alone: ​​"The flowers in my garden have been blooming beautifully recently."

[0857] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0858] Step 1:

[0859] The communication terminal receives a communication and rings. After a specified number of rings, it switches to an auto-answer mode, where the input includes the incoming voice and the output is that the auto-answer mode is activated.

[0860] Step 2:

[0861] The communication terminal records the received voice data in real time and transmits the data to the server. The input includes the received voice data, and the output includes the transmitted voice data.

[0862] Step 3:

[0863] The server uses a speech recognition module to convert the transmitted voice data into text data, where the input includes the voice data and the output is the text data.

[0864] Step 4:

[0865] The server uses a natural language processing module (e.g., spaCy) to analyze the generated text data and compare it with fraud feature patterns, where the input includes the text data and a fraud pattern database, and the output generates a match determination for fraud.

[0866] Step 5:

[0867] The server determines whether the fraudulent activity is occurring, and if a high degree of match is found, the generative artificial intelligence initiates an automatic response. The input includes the result of the match determination, and the output is an automatic response message.

[0868] Step 6:

[0869] The server uses generative artificial intelligence to generate an automatic response message and sends it back to the communication terminal. The input includes the match determination result and a prompt sentence, and the automatic response message is generated as the output.

[0870] Step 7:

[0871] The communication terminal plays the received automatic response message and makes an appropriate response to the fraudulent activity, where the input includes the automatic response message from the server and the output is the played response message.

[0872] Step 8:

[0873] If it is not fraudulent, the communication terminal transfers the call to the elderly person. The input includes the result of the match determination, and the output is that the call is transferred to the elderly person.

[0874] Step 9:

[0875] When an elderly person needs someone to talk to, they say "please talk to me" through the communication terminal and send it to the server. The input includes the elderly person's voice instruction, and the output is the transmission of voice data to the server.

[0876] Step 10:

[0877] The server uses generative artificial intelligence to analyze the user's instructions and initiate a conversation with the elderly, where the input includes voice instructions and prompts, and the output generates a conversational content.

[0878] Step 11:

[0879] The server sends the generated conversation content back to the communication terminal, which then plays it back. The input includes the generated conversation content, and the output is a conversation with the elderly person.

[0880] Step 12:

[0881] When an elderly person needs navigation or product explanations in a store, they can say "Tell me where I need navigation" through a communication terminal and send it to the server. The input includes the elderly person's voice instructions, and the output is the transmission of voice data to the server.

[0882] Step 13:

[0883] The server uses generative artificial intelligence to analyze the user's instructions and provide navigation and product descriptions, where inputs include voice instructions and store data, and outputs generate navigation and product descriptions.

[0884] Step 14:

[0885] The server sends the generated navigation content and product description back to the communication terminal, which then plays them. The input includes the generated navigation content and product description, and the output is a voice guide for the elderly.

[0886] Step 15:

[0887] When an elderly person is approached by a suspicious person near a store, the communication terminal receives the conversation and transmits it to the server in real time. The input includes the received voice data, and the output is the transmission of the voice data to the server.

[0888] Step 16:

[0889] The server analyzes the received voice data using a natural language processing module to determine the likelihood of fraud, where the input includes the voice data and a fraud pattern database, and the output generates a fraud match determination.

[0890] Step 17:

[0891] If the match is determined to be high, the server sends a warning message back to the communication terminal, which then plays it back. The input includes the match determination result and the warning message, and the output is the played warning message.

[0892] Step 18:

[0893] If the degree of match is determined to be low, the communication terminal maintains its normal state without issuing any particular warning. The input includes the result of the degree of match determination, and the output maintains the normal state without any warning.

[0894] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0895] The present invention uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a daily conversation partner. A specific embodiment of this system is described in detail below.

[0896] System configuration

[0897] The system is built around a phone terminal with a built-in generative artificial intelligence module and emotion engine. The main components are:

[0898] 1. Telephone terminal:

[0899] Ringing function when receiving a call

[0900] Ability to record incoming calls in real time and send the audio data to an analysis module

[0901] 2. Generative artificial intelligence:

[0902] Natural language processing functions for user interaction

[0903] Automated response for fraudulent calls

[0904] Daily conversation function for elderly people living alone

[0905] 3. Emotion Engine:

[0906] A function that analyzes emotions from user and fraudster voice data and optimizes generative AI responses

[0907] 4. Vector Database:

[0908] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[0909] Program processing

[0910] The program of this system executes the process in the following procedure.

[0911] Receiving and analyzing calls

[0912] Device:

[0913] When an incoming call occurs, a ring tone sounds to notify the user.

[0914] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[0915] As soon as a call is initiated, audio data is recorded in real time and sent to the analysis module.

[0916] Identifying fraudulent calls

[0917] server:

[0918] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[0919] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[0920] If the score exceeds the threshold, the call is determined to be fraudulent.

[0921] Response to fraudulent phone calls

[0922] Generative artificial intelligence:

[0923] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[0924] Generate questions and responses that will buy time for fraudsters without affecting the user.

[0925] The emotion engine analyzes emotions from the fraudster's voice and optimizes responses accordingly.

[0926] Normal call handling

[0927] Device:

[0928] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[0929] The normal call process will be resumed and the call will not be recorded.

[0930] Features for seniors living alone

[0931] User:

[0932] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[0933] Device:

[0934] It recognizes commands and sends instructions to generative artificial intelligence.

[0935] Generative artificial intelligence:

[0936] Receives commands and initiates a conversation with the user.

[0937] The emotion engine analyzes emotions from the user's voice and generates appropriate responses based on those emotions.

[0938] The content of the conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[0939] Specific examples

[0940] Receiving and responding to fraudulent phone calls

[0941] Situation: An elderly person receives a fraudulent phone call at their home.

[0942] Device: The phone starts ringing and switches to auto-answer mode after three rings.

[0943] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[0944] Server: The call is determined to be fraudulent. Generative AI begins responding.

[0945] Generative AI: Responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[0946] Ability to talk to elderly people

[0947] Situation: An elderly person living alone feels lonely.

[0948] User: Say "Talk to me" to the device.

[0949] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[0950] User: "The flowers in my garden have been blooming beautifully lately."

[0951] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[0952] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[0953] The processing flow will be explained below.

[0954] Step 1:

[0955] Device:

[0956] When powered on, the device boots up the system.

[0957] During the boot process, a self-diagnosis is performed to ensure that each module is working properly.

[0958] It loads a generative artificial intelligence module and an emotion engine, and establishes a connection with the built-in vector database.

[0959] Step 2:

[0960] Device:

[0961] When an incoming call occurs, the phone will ring to notify the user.

[0962] When the ring tone is repeated a specified number of times (e.g. 3 times), it will switch to auto answer mode.

[0963] As soon as the call begins, audio data begins recording and is sent to the analysis module in real time.

[0964] Step 3:

[0965] server:

[0966] It receives voice data sent from the device and converts it into text data using voice recognition technology.

[0967] The text data is compared with fraudulent call patterns registered in a vector database and the degree of match is scored.

[0968] If the score exceeds the threshold, the call is deemed to be fraudulent.

[0969] Step 4:

[0970] Generative artificial intelligence:

[0971] If the call is determined to be fraudulent, an automatic answering mode will be initiated.

[0972] The idea is to generate questions and responses that buy the scammers time without impacting the user.

[0973] The emotion engine analyzes the emotions in the scammer's voice and optimizes the response accordingly.

[0974] Step 5:

[0975] Device:

[0976] If the call is determined not to be a scam, the call will be transferred to an elderly user.

[0977] The normal call process will be used and the call will not be recorded.

[0978] Step 6:

[0979] User:

[0980] If an elderly person living alone feels lonely, they can issue a specific command (e.g., "talk to me") via voice command.

[0981] Device:

[0982] It recognizes commands and sends instructions to generative artificial intelligence.

[0983] Generative artificial intelligence:

[0984] It receives commands, initiates a conversation with the user, and uses natural language processing to provide appropriate responses to what the user says.

[0985] The emotion engine analyzes the emotions from the user's voice and generates appropriate responses based on those emotions.

[0986] The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[0987] Step 7:

[0988] Device:

[0989] Once the call or conversation ends, the communication records and analysis results are saved.

[0990] The generative artificial intelligence module will be safely shut down and put into standby mode until next use.

[0991] Specific examples

[0992] Receiving and responding to fraudulent phone calls

[0993] Situation: An elderly person receives a fraudulent phone call at their home.

[0994] Device: The phone will start ringing and will switch to auto-answer mode after three rings.

[0995] Server: Analyzes the received voice data in real time, converts it into text, and compares it with the characteristic patterns of fraudulent calls.

[0996] Server: The call is determined to be fraudulent. The generative AI begins responding.

[0997] Generative AI: Responding to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[0998] Ability to talk to elderly people

[0999] Situation: An elderly person living alone feels lonely.

[1000] User: Speak into the device, "Talk to me."

[1001] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[1002] User: "The flowers in my garden have been blooming beautifully lately."

[1003] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[1004] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[1005] Example 2

[1006] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1007] There is a need for a communication system that reduces the risk of elderly people becoming victims of fraudulent phone calls and allows elderly people living alone to live safely and comfortably without feeling lonely. Fraudulent phone calls are becoming increasingly sophisticated, making it extremely difficult for elderly people to identify them. Furthermore, elderly people living alone often have no one to talk to on a daily basis, which makes them prone to psychological loneliness. It is necessary to solve these problems simultaneously.

[1008] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1009] In this invention, the server includes means for ringing when a call is received, means for recording voice data in real time and sending it to an analysis module, means for determining whether a call is likely to be a fraudulent call, means for a generative artificial intelligence to automatically respond to the determined fraudulent call, means for analyzing emotions from the user's voice data and optimizing the response, means for transferring the call to the user if the call is not a fraudulent call, and means for the generative artificial intelligence to converse when a user living alone requests someone to talk to. This enables early detection and response of fraudulent calls, not only protecting the elderly from fraud but also providing communication that reduces their everyday feelings of loneliness.

[1010] The "means for ringing when a call is received" is a function that notifies the user by voice when a call is received.

[1011] The "means for recording voice data in real time and transmitting it to an analysis module" is a function for instantly recording voice during a call and transmitting the data for analysis.

[1012] The "means for determining the possibility of a fraudulent call" is a function that includes an algorithm that analyzes the content of a received call and determines whether it is a fraudulent call.

[1013] "Means for automated response by generative artificial intelligence" is a function that automatically generates appropriate responses to identified fraudulent phone calls and continues the dialogue with the fraudster.

[1014] "Means for analyzing emotions from user voice data and optimizing responses" is a function that analyzes the voice during a call to determine the user's emotional state and generates appropriate response content based on that.

[1015] The "means of transferring the call to the user if it is not a fraudulent call" is a function of directly connecting the call to the user if it is determined that the call is not a fraudulent call.

[1016] "A means for generative AI to converse when a user living alone wants someone to talk to" is a function that, when a user feels lonely, receives that command and the AI ​​starts a dialogue and communicates with the user.

[1017] "Means for converting voice data to text" refers to a function that includes an algorithm for converting the voice content of a call into text information.

[1018] The "means for comparing with fraudulent call patterns" is a function for comparing the converted text with characteristic patterns of fraudulent calls registered in advance and evaluating the degree of match.

[1019] The "feature vector database" is a database that stores vector data that quantifies the characteristics of fraudulent calls and is used for comparative analysis.

[1020] This invention is a system that protects elderly people from fraudulent phone calls and provides elderly people living alone with a regular conversation partner. A specific implementation is built around a telephone terminal equipped with a generative artificial intelligence module and an emotion engine. The system configuration is as follows:

[1021] System Components

[1022] 1. Telephone terminal

[1023] Hardware: Includes a speaker that rings when a call comes in, a microphone that records audio in real time, and a communication module that sends the recorded data to the analysis module.

[1024] Software: Equipped with automatic answering function and command recognition function (e.g. Amazon Alexa Voice Service).

[1025] 2. Generative artificial intelligence

[1026] Hardware: A server with a powerful processor and sufficient memory.

[1027] Software: AI modules with natural language processing and dialogue generation capabilities, such as automated answering of fraudulent phone calls and everyday conversations with the elderly (e.g., using models such as GPT-3).

[1028] 3. Emotion Engine

[1029] Software: Algorithms that analyze emotions from voice data and optimize generative AI responses (e.g., IBM Watson Tone Analyzer).

[1030] 4. Vector Database

[1031] Hardware: A database server with high-speed access.

[1032] Software: A database that quantifies and stores characteristic patterns of fraudulent calls. APIs (e.g., Cosine Similarity algorithms) that convert voice data into text and match it with the patterns.

[1033] How it works

[1034] Receiving and analyzing calls

[1035] When a call comes in, the device will ring to notify the user. If there is no answer after three rings, the device will switch to auto-answer mode. Once the call is initiated, the device will record audio data in real time and send it to the analysis module.

[1036] Identifying fraudulent calls

[1037] The server receives the voice data sent from the device and converts it into text data using speech recognition technology. The converted text data is compared with fraudulent call patterns registered in a vector database and a score is assigned to determine the degree of match. If the score exceeds a threshold, the call is determined to be fraudulent.

[1038] Responding to fraudulent phone calls

[1039] If the generative AI determines that the call is a scam, it will initiate an automatic response mode, generating questions and responses to buy the scammer time without impacting the user. The emotion engine analyzes the emotions in the scammer's voice and optimizes the response based on that information.

[1040] Normal call handling

[1041] If the device determines that the call is not a scam, it will transfer the call to the user. The call will not be recorded.

[1042] Conversation starter for elderly people living alone

[1043] If the user feels lonely, they can say to the terminal, "Please talk to me."

[1044] The terminal recognizes this command and sends instructions to the generative artificial intelligence.

[1045] The generative AI receives instructions and begins a conversation with the user. The emotion engine analyzes the user's voice and generates appropriate responses based on their emotions. The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[1046] Specific examples

[1047] Receiving and responding to fraudulent phone calls

[1048] Situation: An elderly person receives a fraudulent phone call at their home.

[1049] Device: The phone starts ringing. After three rings, it switches to auto-answer mode. During this time, the device's display panel will show "Incoming call" and the LED will flash.

[1050] Server: Processes the received voice data in real time and converts it into text. It compares it with the characteristic patterns of fraudulent calls. As a result, it determines that "this is likely to be a fraud."

[1051] Generative AI: The system responds to the fraudster by saying, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the fraudster's emotions. For example, if the fraudster is frustrated, the system responds in a calming tone.

[1052] Ability to talk to elderly people

[1053] Situation: An elderly person living alone feels lonely.

[1054] User: Say "Talk to me" to the device.

[1055] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[1056] User: "The flowers in my garden have been blooming beautifully lately."

[1057] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[1058] Example of input prompt for the generative AI model to be used

[1059] Prompt 1: Analyze the characteristics of the scam call and inform the user that it is a scam.

[1060] Prompt 2: Please start a fun conversation with an elderly person who lives alone and feels lonely.

[1061] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1062] Program processing steps

[1063] Step 1:

[1064] Input: When a call comes in, a signal is sent to the telephone terminal.

[1065] Terminal: When a call comes in, the terminal will ring to notify the user, and the display panel will show "Incoming call" and the LED will flash.

[1066] Output: The user hears the phone ringing or sees the device's display panel and a flashing LED.

[1067] Step 2:

[1068] Input: After three rings or when the user is determined not to answer the call.

[1069] Terminal: If there is no answer after three rings, the device will switch to auto-answer mode. As soon as the call starts, it will start recording the audio data in real time. The recorded audio data will be immediately sent to the analysis module.

[1070] Output: The recorded audio data is sent to the analysis module.

[1071] Step 3:

[1072] Input: Audio data sent from the device.

[1073] Server: The server converts the received voice data into text data using voice recognition technology (e.g., Google Speech-to-Text API).

[1074] Output: Text data is generated.

[1075] Step 4:

[1076] Input: Text data sent from the server.

[1077] Server: Converts the text data into vector format and matches it with a vector database containing patterns of fraudulent calls, scoring the match using, for example, the Cosine Similarity algorithm.

[1078] Output: A match score is generated and a scam call determination is given.

[1079] Step 5:

[1080] Input: Match score and fraud call determination result.

[1081] Generative AI: If the score exceeds a threshold, it is determined to be a fraudulent call and an automatic response mode is initiated. Questions and responses are generated for the fraudster that will buy time without affecting the user. An emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the emotions in the fraudster's voice and optimizes the response based on that information. For example, it generates a response such as, "That's terrible. Can you tell me more about which outstanding balance you're referring to?"

[1082] Output: An automated response to the scammer is generated and sent.

[1083] Step 6:

[1084] Inputs: Match score and if the call was determined not to be fraudulent.

[1085] Terminal: If the call is not a fraudulent call, the terminal will transfer the call to the user, notifying the user of the call via voice guidance and a display panel.

[1086] Output: The call is transferred to the user, who prepares to answer the call.

[1087] Step 7:

[1088] Input: If the user feels lonely, they can speak a voice command to the device saying, "Talk to me."

[1089] Terminal: Recognizes voice commands and sends instructions to the generative artificial intelligence.

[1090] Output: The recognition results of the voice command are sent to the generative artificial intelligence.

[1091] Step 8:

[1092] Input: The recognition result of the voice command.

[1093] Generative AI: Initiates a conversation with the user upon receiving instructions. The emotion engine analyzes the user's voice and generates an appropriate response based on the user's emotion. For example, "Hello! What would you like to talk about today?" It also responds to user comments with, "That's lovely. Tell me what kind of flowers have bloomed."

[1094] Output: A personalized conversation is provided to the user.

[1095] Specific examples

[1096] Prompt 1: "Analyze the characteristics of scam calls and notify the user that they are scams."

[1097] Prompt 2: "Please start a fun conversation with an elderly person who lives alone and feels lonely."

[1098] (Application example 2)

[1099] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1100] The number of victims of fraudulent phone calls targeting elderly people is increasing, and as a countermeasure, a system that can identify fraudulent calls and automatically respond to them is needed. Furthermore, to reduce the sense of loneliness felt by elderly people living alone, a system that can provide daily conversation partners is also needed. This invention aims to protect elderly people from fraudulent phone calls and provide them with daily conversation partners by utilizing generative artificial intelligence and an emotion engine.

[1101] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for determining whether the call is a fraudulent call, means for the generative artificial intelligence to automatically respond to the determined fraudulent call, means for transferring the call to an elderly person if the call is not a fraudulent call, means for the generative artificial intelligence to converse with an elderly person living alone when they need someone to talk to, means for analyzing voice data and detecting emotions, and means for optimizing the response content according to the detected emotions. This allows elderly people to avoid falling victim to fraudulent calls, further reduce their sense of loneliness, and live their lives with peace of mind.

[1102] "Generative AI" is AI that has the ability to generate language and content based on user input.

[1103] A "telephone terminal" is a device with a calling function that supports calls with elderly people.

[1104] "Fraud call detection" is the process of determining whether a call is likely to be fraudulent based on the received voice data.

[1105] "Means for automatic response" refers to a system function in which artificial intelligence automatically generates and responds to identified fraudulent calls.

[1106] "Means of transferring calls to elderly people" is a function that connects normal calls that are determined not to be fraudulent calls to elderly people.

[1107] "Means for generative AI to converse when someone wants someone to talk to" refers to a function that allows AI to automatically start a conversation and respond when an elderly person requests a conversation.

[1108] The "means for analyzing voice data" is a function for analyzing received voice data and converting it into text.

[1109] "Means for detecting emotions" refers to a function that reads emotions from the content and tone of analyzed voice data.

[1110] "Means for optimizing response content" is a function that generates and provides the optimal response based on the detected emotion.

[1111] "Fraud call patterns" are a collection of data summarizing the characteristics and commonalities of past fraud calls.

[1112] A "vector database" is a database that stores and manages data in vector format and allows for comparison of similarities and features.

[1113] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. The specific system configuration and various means for realizing this are described in detail below.

[1114] System configuration

[1115] The system consists of the following main components:

[1116] Telephone terminal: A device with a calling function. Suitable for use by the elderly.

[1117] Generative artificial intelligence (AI): An AI module with natural language processing capabilities.

[1118] Emotion engine: An engine that analyzes emotions from voice data and optimizes responses.

[1119] Vector database: A database that stores patterns of fraudulent calls.

[1120] Program processing

[1121] The program of this system is realized using the following hardware and software.

[1122] Hardware:

[1123] Smartphone

[1124] IoT device with speakerphone functionality

[1125] software:

[1126] Speech recognition module (speech_recognition library)

[1127] Natural language generation module (transformers library)

[1128] Sentiment analysis module (pipeline library)

[1129] Database management system (sqlite3)

[1130] The server first detects an incoming call from the user and acquires the voice data. Next, it analyzes this voice data, converts it into text, and compares it with a vector database to determine whether it is likely to be fraud. If it is determined to be fraud, the generative AI automatically responds. On the other hand, if it is determined not to be fraud, the call can be transferred directly to the elderly person.

[1131] Furthermore, the system's emotion engine analyzes emotions from voice data and optimizes responses based on those emotions. If an elderly person living alone says, "I want someone to talk to," the system recognizes this and initiates a conversation with the generative AI, generating an appropriate response.

[1132] Specific examples

[1133] Receiving and responding to fraudulent phone calls

[1134] Situation: An elderly person receives a fraudulent phone call at their home.

[1135] User: Import audio into the system as "input_audio.wav".

[1136] Server: Analyzes received voice data in real time and converts it into text.

[1137] Server: Compares with the characteristic patterns of fraudulent calls and determines that it is a fraudulent call.

[1138] Generative AI: Responds to the scammer with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[1139] Ability to talk to elderly people

[1140] Situation: An elderly person living alone feels lonely and wants someone to talk to.

[1141] User: Say "Talk to me" to the device.

[1142] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[1143] User: "The flowers in my garden have been blooming beautifully lately."

[1144] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[1145] Examples of specific prompts include:

[1146] "Scam call voice: Mom, it's me and I want to talk to you about an unpaid bill."

[1147] The present invention enables elderly people to live without becoming victims of fraudulent phone calls, and further reduces their sense of loneliness, allowing them to live with peace of mind.

[1148] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1149] Step 1:

[1150] Terminal: The phone starts ringing. It notifies the user of the incoming call and plays the ringtone. The input is the incoming call signal and the output is the ringtone.

[1151] Step 2:

[1152] Terminal: Switch to auto-answer mode after the specified number of rings (e.g. 3 times). Input is the number of rings, output is to switch to auto-answer mode.

[1153] Step 3:

[1154] Terminal: When a call is started, the terminal records the voice data in real time and sends it to the server. The input is the call start signal, and the output is the recorded voice data.

[1155] Step 4:

[1156] Server: Converts received voice data into text using a speech recognition module (speech_recognition library). The input is recorded voice data, and the output is text data. Specifically, the audio file is analyzed and the voice is converted into text.

[1157] Step 5:

[1158] Server: Compares the text data with a vector database to determine the likelihood of a fraudulent call. The input is text data, and the output is the result of the fraudulent call determination. Specifically, it compares the data with fraudulent call patterns registered in the vector database and scores the degree of match.

[1159] Step 6:

[1160] Server: If the call is determined to be fraudulent, an automatic response is generated using generative artificial intelligence (the transformers library). The input is the fraudulent call determination result and text data, and the output is the generated response. Specifically, questions and responses are generated for the fraudster.

[1161] Step 7:

[1162] Server: Sends the generated response to the device and plays it back to the fraudster. The input is the generated response, and the output is the played response. Specifically, the emotion engine analyzes the emotion from the fraudster's voice and optimizes the response based on the results.

[1163] Step 8:

[1164] Server: If the call is determined not to be a scam, the server transfers the call to the elderly person. The input is the result of the scam call determination, and the output is the call transfer.

[1165] Step 9:

[1166] Terminal: The elderly person utters a specific command, such as "Please be my conversation partner." The input is the command voice, and the output is command recognition.

[1167] Step 10:

[1168] Generative AI: Recognizes commands and starts a conversation. The input is a spoken command, and the output is a generated conversation starter. Specifically, it responds, "Hello! What would you like to talk about today?"

[1169] Step 11:

[1170] Terminal and Generative AI: The content of what the user says is analyzed in real time, and the emotion engine analyzes the emotion. The input is the user's voice, and the output is the result of the emotion analysis. Specifically, in response to the statement, "The flowers in the garden have been blooming beautifully recently," the response generated is, "That's lovely. Please tell me what kind of flowers have bloomed."

[1171] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1172] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1173] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1174] [Third embodiment]

[1175] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1176] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1177] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1178] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1179] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1180] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1181] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1182] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1183] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1184] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1185] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1186] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1187] The present invention uses a telephone terminal equipped with generative artificial intelligence to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. A specific embodiment of this system is described in detail below.

[1188] System configuration

[1189] The system is built around a telephone terminal with a built-in generative artificial intelligence module. The main components are:

[1190] 1. Telephone terminal:

[1191] Ringing function when receiving a call

[1192] Ability to record incoming calls in real time and send the audio data to an analysis module

[1193] 2. Generative artificial intelligence:

[1194] Natural language processing functions for user interaction

[1195] Automated response for fraudulent calls

[1196] Daily conversation function for elderly people living alone

[1197] 3. Vector Database:

[1198] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[1199] Program processing

[1200] The program for this system mainly executes processing in the following steps. A specific processing flow and operation example are shown below.

[1201] Receiving and analyzing calls

[1202] Device:

[1203] When an incoming call occurs, the device immediately rings and automatically answers after a specified number of rings.

[1204] As soon as the call begins, the audio data is recorded in real time and sent to the analysis module.

[1205] Identifying fraudulent calls

[1206] server:

[1207] Voice recognition technology is used to convert voice data sent from the terminal into text.

[1208] The text data is compared with a vector database to determine whether it matches the characteristic patterns of fraudulent calls.

[1209] If there is a high degree of match, the call is determined to be fraudulent and the response is handed over to generative artificial intelligence.

[1210] Response to fraudulent phone calls

[1211] Generative artificial intelligence:

[1212] If a call is determined to be fraudulent, the AI ​​automatically initiates a fraudulent call and response, generating responses to the fraudster's questions that will not affect the user and buy time.

[1213] Responses are flexibly changed to prevent elderly people from becoming victims by continuing the conversation with the scammer.

[1214] Normal call handling

[1215] Device:

[1216] If the call is determined not to be a scam, the device will transfer the call to the elderly person.

[1217] The senior citizen can continue the conversation by following normal call procedures.

[1218] Features for seniors living alone

[1219] User:

[1220] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[1221] Generative artificial intelligence:

[1222] The generative AI recognizes the command and automatically starts a conversation. The generative AI generates an appropriate response to the user's utterance and continues the conversation in a natural way.

[1223] Specific examples

[1224] Receiving and responding to fraudulent phone calls

[1225] Situation: An elderly person receives a fraudulent phone call at their home.

[1226] Device: The phone starts ringing. Because it is set to auto-answer mode, the call is automatically answered after three rings.

[1227] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[1228] Server: The call is determined to be fraudulent. The generative AI begins responding.

[1229] Generative AI: Buys time by responding to scammers with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[1230] Ability to talk to elderly people

[1231] Situation: An elderly person living alone feels lonely.

[1232] User: Say "Please talk to me" to the device.

[1233] Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[1234] User: "The flowers in my garden have been blooming beautifully lately."

[1235] Generator: "That's lovely. Can you tell me what kind of flowers bloomed?"

[1236] This invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI function for everyday conversations reduces the mental burden on elderly people living alone.

[1237] The processing flow will be explained below.

[1238] Step 1:

[1239] Device:

[1240] When powered on, the terminal boots the system.

[1241] During the startup process, a self-test is performed to verify that each module is operating correctly.

[1242] Loads the generative artificial intelligence module and establishes a connection with the built-in vector database.

[1243] Step 2:

[1244] Device:

[1245] When an incoming call occurs, a ring tone sounds to notify the user.

[1246] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[1247] As soon as the call begins, audio data recording begins and is sent to the analysis module in real time.

[1248] Step 3:

[1249] server:

[1250] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[1251] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[1252] If the score exceeds the threshold, the call is determined to be fraudulent.

[1253] Step 4:

[1254] Generative artificial intelligence:

[1255] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[1256] Generate questions and responses that will buy time for fraudsters without affecting the user.

[1257] Responses can be flexibly changed, and conversations with scammers are constantly recorded and analyzed.

[1258] Step 5:

[1259] Device:

[1260] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[1261] The normal call process will be resumed and the call will not be recorded.

[1262] Step 6:

[1263] User:

[1264] If an elderly person living alone feels lonely, the robot will issue a specific command (e.g., "talk to me") via voice command.

[1265] Device:

[1266] It recognizes commands and sends instructions to generative artificial intelligence.

[1267] Generative artificial intelligence:

[1268] It receives commands and starts a conversation with the user, using natural language processing to provide appropriate responses to what the user says.

[1269] The conversation content is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[1270] Step 7:

[1271] Device:

[1272] When the call or conversation ends, the communication records and analysis results are saved.

[1273] Safely shuts down the generative artificial intelligence module and puts it into standby mode until next use.

[1274] This system not only protects seniors from fraudulent phone calls, but also serves as a daily conversation partner, improving their sense of security and quality of life.

[1275] Example 1

[1276] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1277] The risk of elderly people becoming victims of fraudulent phone calls is increasing. In addition, elderly people who live alone often feel lonely because they have no one to talk to on a daily basis. To solve these issues, a system is needed that can identify fraudulent calls, automatically answer them, and provide elderly people living alone with someone to talk to on a daily basis.

[1278] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1279] In this invention, the server includes a means for ringing and switching to automatic answering when an incoming call is received, a means for recording voice data as soon as the call begins and sending it to an analysis module, a means for converting the received voice data into text and comparing it with fraudulent call characteristic patterns, a means for calculating the degree of match and automatically answering the call using a generative artificial intelligence if the call is determined to be fraudulent, a means for transferring the call to an elderly person if the call is not fraudulent, and a means for the generative artificial intelligence to converse when an elderly person living alone needs someone to talk to. This makes it possible to prevent damage caused by fraudulent calls and ensure the safety of the elderly. It also makes it possible to reduce the sense of loneliness felt by elderly people living alone.

[1280] "Means for ringing and switching to automatic answering when an incoming call occurs" refers to the function by which a telephone terminal detects an incoming call and automatically starts the call after ringing a predetermined number of times.

[1281] "Means for recording voice data as soon as a call starts and sending it to an analysis module" refers to a function for recording voice in real time as soon as a call starts and sending that data to a module for data analysis.

[1282] "Means for converting received voice data into text and comparing it with fraudulent call characteristic patterns" refers to a function that uses voice recognition technology to convert recorded voice data into text data and compares the text with predefined fraudulent call characteristic patterns.

[1283] "Means for calculating the degree of match and for the generative artificial intelligence to automatically respond if the call is determined to be fraudulent" refers to a function that evaluates the possibility of a fraudulent call based on the degree of match of the matching results, and for the generative artificial intelligence to automatically respond if the degree of match is high.

[1284] "Means of transferring calls to elderly people if they are not fraudulent calls" refers to a function that notifies elderly people of calls that are determined to be non-scam calls, allowing elderly people to actually make the calls.

[1285] "Means for generative AI to converse when an elderly person living alone wants someone to talk to" refers to the function of generative AI to engage in natural dialogue when an elderly person living alone utters a specific command.

[1286] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence to provide elderly people with everyday conversation partners while protecting them from fraudulent phone calls. The system's main components are a telephone terminal, a server, generative artificial intelligence, and a vector database.

[1287] Main components

[1288] 1. Telephone terminal

[1289] Device:

[1290] When an incoming call occurs, the device will ring and after a specified number of rings (e.g., three times) will switch to auto-answer mode.

[1291] As soon as the call begins, the audio data is recorded in real time and sent to the server.

[1292] 2. Server

[1293] server:

[1294] It receives voice data sent from the device and converts it into text using voice recognition technology (e.g., Google Cloud Speech-to-Text).

[1295] The text data is compared with characteristic patterns of fraudulent calls stored in a vector database (e.g., Elasticsearch) and the degree of match is calculated.

[1296] If the degree of match is high, the call is determined to be fraudulent and the generative artificial intelligence is notified.

[1297] 3. Generative artificial intelligence

[1298] Generative artificial intelligence:

[1299] If a call is determined to be fraudulent, the AI ​​will automatically respond to the call, generating responses to the fraudster's questions that will not affect the user and buy time.

[1300] When an elderly person living alone wants someone to talk to, the system provides natural dialogue in response to user instructions.

[1301] 4. Vector Database

[1302] Vector Database:

[1303] It stores characteristic patterns of fraudulent calls and compares them with text data sent from the server to calculate the degree of match.

[1304] Specific examples

[1305] Receiving and responding to fraudulent phone calls

[1306] Situation: An elderly person receives a fraudulent phone call at their home.

[1307] 1. Device: The phone starts ringing. After three rings, the phone automatically answers the call because it is set to auto-answer mode.

[1308] 2. Terminal: Records audio data in real time and sends it to the server.

[1309] 3. Server: The received voice data is converted into text using Google Cloud Speech-to-Text, and compared with fraudulent call patterns stored in Elasticsearch, a vector database.

[1310] 4. Server: Calculates the degree of match and determines whether the call is fraudulent. Notifies the generative AI.

[1311] 5. Generative AI: The system responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" and continues the conversation.

[1312] Ability to talk to elderly people

[1313] Situation: An elderly person living alone feels lonely.

[1314] 1. User: Speaks to the device, "Please talk to me."

[1315] 2. Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[1316] 3. User: "The flowers in my garden have been blooming beautifully lately."

[1317] 4. Generative AI: "That's lovely! Tell me what kind of flowers bloomed."

[1318] In this way, this invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI-based everyday conversation function reduces the mental burden on elderly people living alone.

[1319] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1320] Step 1:

[1321] Device:

[1322] Input: Incoming phone signal.

[1323] What it does: Rings when an incoming call comes in.

[1324] Output: The phone will start ringing. After the specified number of rings (e.g. 3), the phone will switch to auto-answer mode.

[1325] Specific operation: After three rings, the message "Now entering auto answer mode" will be played.

[1326] Step 2:

[1327] Device:

[1328] Input: Call initiation signal and audio data.

[1329] How it works: When a call starts, the audio data is recorded in real time and sent to the server.

[1330] Output: Recorded audio data.

[1331] Specific operation: Recording begins as soon as the call begins, and the voice data is divided into a certain number of data packets, encoded, and transferred to the server.

[1332] Step 3:

[1333] server:

[1334] Input: Audio data sent from the device.

[1335] How it works: Uses speech recognition technology (e.g., Google Cloud Speech-to-Text) to convert audio data into text.

[1336] Output: Textualized data.

[1337] Specific operation: The server receives the voice data and calls a speech recognition API to convert the data into text.

[1338] Step 4:

[1339] server:

[1340] Input: Textual data.

[1341] Operation: Compare the text data with the characteristic patterns of fraudulent calls stored in a vector database (e.g., Elasticsearch). Calculate the degree of match of the matching results.

[1342] Output: Matching result.

[1343] Specific operation: After the server receives the text data, it inputs the feature pattern vector into a matching algorithm, and if the degree of match is above a certain level, it generates a result that says "possibly a fraudulent call."

[1344] Step 5:

[1345] server:

[1346] Input: Matching result.

[1347] Operation: If there is a high match, the generative artificial intelligence is notified.

[1348] Output: Notification if the call is determined to be fraudulent.

[1349] Specific operation: The judgment result is sent to generative artificial intelligence, and a notification is sent stating that "there is a high possibility that the call is fraudulent."

[1350] Step 6:

[1351] Generative artificial intelligence:

[1352] Enter: Scam call notification.

[1353] How it works: If a call is determined to be fraudulent, the AI ​​automatically responds, generating a response that won't affect the user and buys time for the fraudster.

[1354] Output: Response to the scammer.

[1355] What it does: Generates enticing statements, such as "That's terrible! Which outstanding balance are you talking about?"

[1356] Step 7:

[1357] Device:

[1358] Input: If the server determines that the call is not a scam.

[1359] Operation: If the call is determined not to be a scam, the call is transferred to an elderly person.

[1360] Output: Call transfer.

[1361] Specific actions: The system conveys the message "This call is normal. Please continue speaking" to the elderly person and switches the call over to the elderly person.

[1362] Step 8:

[1363] User:

[1364] Enter: If you feel lonely.

[1365] Action: Say "Let me talk to you" to the device.

[1366] Output: The command passed.

[1367] Specific action: The user commands the device to "be a conversation partner."

[1368] Step 9:

[1369] Generative artificial intelligence:

[1370] Input: Commands from the user.

[1371] How it works: When an elderly person living alone needs someone to talk to, the AI ​​automatically starts a conversation.

[1372] Output: Natural dialogue.

[1373] Specific behavior: The generative AI responds, "Hello! What would you like to talk about today?" and provides an appropriate response to the user's statement, such as, "The flowers in the garden have been blooming beautifully recently," and continues, "That's wonderful. Please tell me what kind of flowers have bloomed."

[1374] This allows elderly people to use the telephone with peace of mind and reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud, and the everyday conversation function using generative artificial intelligence reduces the mental burden on elderly people living alone.

[1375] (Application example 1)

[1376] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1377] This invention relates to the provision of a system for protecting elderly people from fraud. Specifically, the objective is to protect elderly people from fraud by detecting possible fraud and automatically taking action, and to provide support for elderly people to enjoy shopping and staying in stores with peace of mind, thereby reducing the sense of loneliness felt by elderly people when they are alone.

[1378] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1379] In this invention, the server includes means for determining the possibility of fraud, means for a generative artificial intelligence to automatically respond to the determined fraud, means for transferring communication to an elderly person if fraud is not an attempt, means for the generative artificial intelligence to converse with an elderly person living alone when the elderly person wants someone to talk to, means for providing navigation and product explanations to elderly people visiting a store, means for analyzing conversations with suspicious people near the store and detecting fraud and means for issuing a warning based on the detection result, and means for the generative artificial intelligence to support the conversation when the elderly person is looking for someone to talk to. This protects elderly people from fraud, improves their shopping experience in stores, and reduces the sense of loneliness they feel when they are alone.

[1380] "Generative AI" is an AI technology that has the ability to generate new data and answers based on input data.

[1381] A "communication terminal" is an electronic device for sending and receiving voice and data.

[1382] "Fraud" is the act of deceiving others through dishonest means and stealing their property or information.

[1383] A "means of judgment" is a method or technique for making an appropriate judgment about an event or data.

[1384] "Means for automatic response" is a function that automatically generates a response to an external input without human intervention.

[1385] A "relaying means" is a method or device for relaying specific information or communication to a designated party.

[1386] "When someone wants someone to talk to" refers to a situation in which the other person wants to have a conversation.

[1387] "Navigation" is a function that provides guidance on the route to a destination.

[1388] "Product Description" is detailed information about the characteristics and usage of the product being offered.

[1389] "Means of analysis" refers to methods and techniques for analyzing input data in detail and extracting meaning and patterns.

[1390] "Warning means" means a method or technique for notifying people of a particular danger or abnormality.

[1391] "Means to support conversation" are auxiliary functions to facilitate smooth dialogue with the other person.

[1392] The system of the present invention uses a communication terminal equipped with generative artificial intelligence to protect elderly people from fraud and provide elderly people living alone with a daily conversation partner. It also provides support for elderly people to enjoy shopping and staying in physical stores with peace of mind. Specific embodiments of the present invention are described in detail below.

[1393] System configuration

[1394] The system is built around a communications terminal with a built-in generative artificial intelligence module, and its main components are as follows:

[1395] 1. Communication terminal:

[1396] Ability to receive communications and perform specified actions

[1397] A function that analyzes received voice data in real time and sends it to the server

[1398] 2. Generative artificial intelligence:

[1399] Natural language processing functions for user interaction

[1400] Automatic fraud detection function

[1401] Daily conversation function for elderly people living alone

[1402] Functions that provide in-store navigation and product information

[1403] 3. Server:

[1404] A function that stores characteristic patterns of fraudulent activity, converts voice data into text, and compares it with those patterns

[1405] 4. Warning system:

[1406] A function that analyzes interactions with suspicious people near stores, detects fraudulent activity, and issues a warning.

[1407] 5. Interface:

[1408] A user interface designed to make communication devices easier for the elderly to operate

[1409] Hardware and software used

[1410] Hardware:

[1411] Communication device: A smartphone with audio input (microphone) and audio output (speaker)

[1412] software:

[1413] Speech recognition: using HuggingFace's transformer pipeline

[1414] Natural Language Processing: Using spaCy

[1415] Generative AI: Using OpenAI's GPT-3.5-turbo model

[1416] Data processing and calculation

[1417] The server uses a speech recognition module to convert voice data into text data. It then uses a natural language processing module to analyze the text data and compare it with characteristic patterns of fraudulent activity. The generative artificial intelligence module generates natural dialogue and acts as a pseudo-conversational partner for the elderly. It also provides in-store navigation and product information to help seniors enjoy their shopping experience.

[1418] Specific examples

[1419] Dealing with fraud

[1420] Situation: An elderly person receives a fraudulent phone call at their home.

[1421] Communication terminal: When the call starts ringing, it will be set to auto-answer mode after the specified number of rings. The voice data will be analyzed in real time and sent to the server.

[1422] Server: Converts the received voice data into text and compares it with characteristic patterns of fraudulent activity.

[1423] Server: If fraudulent activity is determined, the generative artificial intelligence will automatically initiate a response, generating a response that will not have any impact on the fraudster.

[1424] In-store navigation and product explanations

[1425] Situation: An elderly person is searching for a product in a physical store.

[1426] User: Speak into the communication device, "Please tell me where I need navigation."

[1427] Generative AI: Asks, "Which section should I go to?" and provides navigation and product descriptions.

[1428] Prompt Sentence Examples

[1429] Dealing with fraud: "I was told I have outstanding bills. What should I do?"

[1430] In-store navigation: "Tell me where the milk is."

[1431] Conversation between an elderly person living alone: ​​"The flowers in my garden have been blooming beautifully recently."

[1432] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1433] Step 1:

[1434] The communication terminal receives a communication and rings. After a specified number of rings, it switches to an auto-answer mode, where the input includes the incoming voice and the output is that the auto-answer mode is activated.

[1435] Step 2:

[1436] The communication terminal records the received voice data in real time and transmits the data to the server. The input includes the received voice data, and the output includes the transmitted voice data.

[1437] Step 3:

[1438] The server uses a speech recognition module to convert the transmitted voice data into text data, where the input includes the voice data and the output is the text data.

[1439] Step 4:

[1440] The server uses a natural language processing module (e.g., spaCy) to analyze the generated text data and compare it with fraud feature patterns, where the input includes the text data and a fraud pattern database, and the output generates a match determination for fraud.

[1441] Step 5:

[1442] The server determines whether the fraudulent activity is occurring, and if a high degree of match is found, the generative artificial intelligence initiates an automatic response. The input includes the result of the match determination, and the output is an automatic response message.

[1443] Step 6:

[1444] The server uses generative artificial intelligence to generate an automatic response message and sends it back to the communication terminal. The input includes the match determination result and a prompt sentence, and the automatic response message is generated as the output.

[1445] Step 7:

[1446] The communication terminal plays the received automatic response message and makes an appropriate response to the fraudulent activity, where the input includes the automatic response message from the server and the output is the played response message.

[1447] Step 8:

[1448] If it is not fraudulent, the communication terminal transfers the call to the elderly person. The input includes the result of the match determination, and the output is that the call is transferred to the elderly person.

[1449] Step 9:

[1450] When an elderly person needs someone to talk to, they say "please talk to me" through the communication terminal and send it to the server. The input includes the elderly person's voice instruction, and the output is the transmission of voice data to the server.

[1451] Step 10:

[1452] The server uses generative artificial intelligence to analyze the user's instructions and initiate a conversation with the elderly, where the input includes voice instructions and prompts, and the output generates a conversational content.

[1453] Step 11:

[1454] The server sends the generated conversation content back to the communication terminal, which then plays it back. The input includes the generated conversation content, and the output is a conversation with the elderly person.

[1455] Step 12:

[1456] When an elderly person needs navigation or product explanations in a store, they can say "Tell me where I need navigation" through a communication terminal and send it to the server. The input includes the elderly person's voice instructions, and the output is the transmission of voice data to the server.

[1457] Step 13:

[1458] The server uses generative artificial intelligence to analyze the user's instructions and provide navigation and product descriptions, where inputs include voice instructions and store data, and outputs generate navigation and product descriptions.

[1459] Step 14:

[1460] The server sends the generated navigation content and product description back to the communication terminal, which then plays them. The input includes the generated navigation content and product description, and the output is a voice guide for the elderly.

[1461] Step 15:

[1462] When an elderly person is approached by a suspicious person near a store, the communication terminal receives the conversation and transmits it to the server in real time. The input includes the received voice data, and the output is the transmission of the voice data to the server.

[1463] Step 16:

[1464] The server analyzes the received voice data using a natural language processing module to determine the likelihood of fraud, where the input includes the voice data and a fraud pattern database, and the output generates a fraud match determination.

[1465] Step 17:

[1466] If the match is determined to be high, the server sends a warning message back to the communication terminal, which then plays it back. The input includes the match determination result and the warning message, and the output is the played warning message.

[1467] Step 18:

[1468] If the degree of match is determined to be low, the communication terminal maintains its normal state without issuing any particular warning. The input includes the result of the degree of match determination, and the output maintains the normal state without any warning.

[1469] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1470] The present invention uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a daily conversation partner. A specific embodiment of this system is described in detail below.

[1471] System configuration

[1472] The system is built around a phone terminal with a built-in generative artificial intelligence module and emotion engine. The main components are:

[1473] 1. Telephone terminal:

[1474] Ringing function when receiving a call

[1475] Ability to record incoming calls in real time and send the audio data to an analysis module

[1476] 2. Generative artificial intelligence:

[1477] Natural language processing functions for user interaction

[1478] Automated response for fraudulent calls

[1479] Daily conversation function for elderly people living alone

[1480] 3. Emotion Engine:

[1481] A function that analyzes emotions from user and fraudster voice data and optimizes generative AI responses

[1482] 4. Vector Database:

[1483] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[1484] Program processing

[1485] The program of this system executes the process in the following procedure.

[1486] Receiving and analyzing calls

[1487] Device:

[1488] When an incoming call occurs, a ring tone sounds to notify the user.

[1489] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[1490] As soon as a call is initiated, audio data is recorded in real time and sent to the analysis module.

[1491] Identifying fraudulent calls

[1492] server:

[1493] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[1494] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[1495] If the score exceeds the threshold, the call is determined to be fraudulent.

[1496] Response to fraudulent phone calls

[1497] Generative artificial intelligence:

[1498] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[1499] Generate questions and responses that will buy time for fraudsters without affecting the user.

[1500] The emotion engine analyzes emotions from the fraudster's voice and optimizes responses accordingly.

[1501] Normal call handling

[1502] Device:

[1503] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[1504] The normal call process will be resumed and the call will not be recorded.

[1505] Features for seniors living alone

[1506] User:

[1507] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[1508] Device:

[1509] It recognizes commands and sends instructions to generative artificial intelligence.

[1510] Generative artificial intelligence:

[1511] Receives commands and initiates a conversation with the user.

[1512] The emotion engine analyzes emotions from the user's voice and generates appropriate responses based on those emotions.

[1513] The content of the conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[1514] Specific examples

[1515] Receiving and responding to fraudulent phone calls

[1516] Situation: An elderly person receives a fraudulent phone call at their home.

[1517] Device: The phone starts ringing and switches to auto-answer mode after three rings.

[1518] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[1519] Server: The call is determined to be fraudulent. Generative AI begins responding.

[1520] Generative AI: Responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[1521] Ability to talk to elderly people

[1522] Situation: An elderly person living alone feels lonely.

[1523] User: Say "Talk to me" to the device.

[1524] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[1525] User: "The flowers in my garden have been blooming beautifully lately."

[1526] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[1527] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[1528] The processing flow will be explained below.

[1529] Step 1:

[1530] Device:

[1531] When powered on, the device boots up the system.

[1532] During the boot process, a self-diagnosis is performed to ensure that each module is working properly.

[1533] It loads a generative artificial intelligence module and an emotion engine, and establishes a connection with the built-in vector database.

[1534] Step 2:

[1535] Device:

[1536] When an incoming call occurs, the phone will ring to notify the user.

[1537] When the ring tone is repeated a specified number of times (e.g. 3 times), it will switch to auto answer mode.

[1538] As soon as the call begins, audio data begins recording and is sent to the analysis module in real time.

[1539] Step 3:

[1540] server:

[1541] It receives voice data sent from the device and converts it into text data using voice recognition technology.

[1542] The text data is compared with fraudulent call patterns registered in a vector database and the degree of match is scored.

[1543] If the score exceeds the threshold, the call is deemed to be fraudulent.

[1544] Step 4:

[1545] Generative artificial intelligence:

[1546] If the call is determined to be fraudulent, an automatic answering mode will be initiated.

[1547] The idea is to generate questions and responses that buy the scammers time without impacting the user.

[1548] The emotion engine analyzes the emotions in the scammer's voice and optimizes the response accordingly.

[1549] Step 5:

[1550] Device:

[1551] If the call is determined not to be a scam, the call will be transferred to an elderly user.

[1552] The normal call process will be used and the call will not be recorded.

[1553] Step 6:

[1554] User:

[1555] If an elderly person living alone feels lonely, they can issue a specific command (e.g., "talk to me") via voice command.

[1556] Device:

[1557] It recognizes commands and sends instructions to generative artificial intelligence.

[1558] Generative artificial intelligence:

[1559] It receives commands, initiates a conversation with the user, and uses natural language processing to provide appropriate responses to what the user says.

[1560] The emotion engine analyzes the emotions from the user's voice and generates appropriate responses based on those emotions.

[1561] The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[1562] Step 7:

[1563] Device:

[1564] Once the call or conversation ends, the communication records and analysis results are saved.

[1565] The generative artificial intelligence module will be safely shut down and put into standby mode until next use.

[1566] Specific examples

[1567] Receiving and responding to fraudulent phone calls

[1568] Situation: An elderly person receives a fraudulent phone call at their home.

[1569] Device: The phone will start ringing and will switch to auto-answer mode after three rings.

[1570] Server: Analyzes the received voice data in real time, converts it into text, and compares it with the characteristic patterns of fraudulent calls.

[1571] Server: The call is determined to be fraudulent. The generative AI begins responding.

[1572] Generative AI: Responding to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[1573] Ability to talk to elderly people

[1574] Situation: An elderly person living alone feels lonely.

[1575] User: Speak into the device, "Talk to me."

[1576] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[1577] User: "The flowers in my garden have been blooming beautifully lately."

[1578] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[1579] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[1580] Example 2

[1581] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1582] There is a need for a communication system that reduces the risk of elderly people becoming victims of fraudulent phone calls and allows elderly people living alone to live safely and comfortably without feeling lonely. Fraudulent phone calls are becoming increasingly sophisticated, making it extremely difficult for elderly people to identify them. Furthermore, elderly people living alone often have no one to talk to on a daily basis, which makes them prone to psychological loneliness. It is necessary to solve these problems simultaneously.

[1583] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1584] In this invention, the server includes means for ringing when a call is received, means for recording voice data in real time and sending it to an analysis module, means for determining whether a call is likely to be a fraudulent call, means for a generative artificial intelligence to automatically respond to the determined fraudulent call, means for analyzing emotions from the user's voice data and optimizing the response, means for transferring the call to the user if the call is not a fraudulent call, and means for the generative artificial intelligence to converse when a user living alone requests someone to talk to. This enables early detection and response of fraudulent calls, not only protecting the elderly from fraud but also providing communication that reduces their everyday feelings of loneliness.

[1585] The "means for ringing when a call is received" is a function that notifies the user by voice when a call is received.

[1586] The "means for recording voice data in real time and transmitting it to an analysis module" is a function for instantly recording voice during a call and transmitting the data for analysis.

[1587] The "means for determining the possibility of a fraudulent call" is a function that includes an algorithm that analyzes the content of a received call and determines whether it is a fraudulent call.

[1588] "Means for automated response by generative artificial intelligence" is a function that automatically generates appropriate responses to identified fraudulent phone calls and continues the dialogue with the fraudster.

[1589] "Means for analyzing emotions from user voice data and optimizing responses" is a function that analyzes the voice during a call to determine the user's emotional state and generates appropriate response content based on that.

[1590] The "means of transferring the call to the user if it is not a fraudulent call" is a function of directly connecting the call to the user if it is determined that the call is not a fraudulent call.

[1591] "A means for generative AI to converse when a user living alone wants someone to talk to" is a function that, when a user feels lonely, receives that command and the AI ​​starts a dialogue and communicates with the user.

[1592] "Means for converting voice data to text" refers to a function that includes an algorithm for converting the voice content of a call into text information.

[1593] The "means for comparing with fraudulent call patterns" is a function for comparing the converted text with characteristic patterns of fraudulent calls registered in advance and evaluating the degree of match.

[1594] The "feature vector database" is a database that stores vector data that quantifies the characteristics of fraudulent calls and is used for comparative analysis.

[1595] This invention is a system that protects elderly people from fraudulent phone calls and provides elderly people living alone with a regular conversation partner. A specific implementation is built around a telephone terminal equipped with a generative artificial intelligence module and an emotion engine. The system configuration is as follows:

[1596] System Components

[1597] 1. Telephone terminal

[1598] Hardware: Includes a speaker that rings when a call comes in, a microphone that records audio in real time, and a communication module that sends the recorded data to the analysis module.

[1599] Software: Equipped with automatic answering function and command recognition function (e.g. Amazon Alexa Voice Service).

[1600] 2. Generative artificial intelligence

[1601] Hardware: A server with a powerful processor and sufficient memory.

[1602] Software: AI modules with natural language processing and dialogue generation capabilities, such as automated answering of fraudulent phone calls and everyday conversations with the elderly (e.g., using models such as GPT-3).

[1603] 3. Emotion Engine

[1604] Software: Algorithms that analyze emotions from voice data and optimize generative AI responses (e.g., IBM Watson Tone Analyzer).

[1605] 4. Vector Database

[1606] Hardware: A database server with high-speed access.

[1607] Software: A database that quantifies and stores characteristic patterns of fraudulent calls. APIs (e.g., Cosine Similarity algorithms) that convert voice data into text and match it with the patterns.

[1608] How it works

[1609] Receiving and analyzing calls

[1610] When a call comes in, the device will ring to notify the user. If there is no answer after three rings, the device will switch to auto-answer mode. Once the call is initiated, the device will record audio data in real time and send it to the analysis module.

[1611] Identifying fraudulent calls

[1612] The server receives the voice data sent from the device and converts it into text data using speech recognition technology. The converted text data is compared with fraudulent call patterns registered in a vector database and a score is assigned to determine the degree of match. If the score exceeds a threshold, the call is determined to be fraudulent.

[1613] Responding to fraudulent phone calls

[1614] If the generative AI determines that the call is a scam, it will initiate an automatic response mode, generating questions and responses to buy the scammer time without impacting the user. The emotion engine analyzes the emotions in the scammer's voice and optimizes the response based on that information.

[1615] Normal call handling

[1616] If the device determines that the call is not a scam, it will transfer the call to the user. The call will not be recorded.

[1617] Conversation starter for elderly people living alone

[1618] If the user feels lonely, they can say to the terminal, "Please talk to me."

[1619] The terminal recognizes this command and sends instructions to the generative artificial intelligence.

[1620] The generative AI receives instructions and begins a conversation with the user. The emotion engine analyzes the user's voice and generates appropriate responses based on their emotions. The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[1621] Specific examples

[1622] Receiving and responding to fraudulent phone calls

[1623] Situation: An elderly person receives a fraudulent phone call at their home.

[1624] Device: The phone starts ringing. After three rings, it switches to auto-answer mode. During this time, the device's display panel will show "Incoming call" and the LED will flash.

[1625] Server: Processes the received voice data in real time and converts it into text. It compares it with the characteristic patterns of fraudulent calls. As a result, it determines that "this is likely to be a fraud."

[1626] Generative AI: The system responds to the fraudster by saying, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the fraudster's emotions. For example, if the fraudster is frustrated, the system responds in a calming tone.

[1627] Ability to talk to elderly people

[1628] Situation: An elderly person living alone feels lonely.

[1629] User: Say "Talk to me" to the device.

[1630] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[1631] User: "The flowers in my garden have been blooming beautifully lately."

[1632] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[1633] Example of input prompt for the generative AI model to be used

[1634] Prompt 1: Analyze the characteristics of the scam call and inform the user that it is a scam.

[1635] Prompt 2: Please start a fun conversation with an elderly person who lives alone and feels lonely.

[1636] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1637] Program processing steps

[1638] Step 1:

[1639] Input: When a call comes in, a signal is sent to the telephone terminal.

[1640] Terminal: When a call comes in, the terminal will ring to notify the user, and the display panel will show "Incoming call" and the LED will flash.

[1641] Output: The user hears the phone ringing or sees the device's display panel and a flashing LED.

[1642] Step 2:

[1643] Input: After three rings or when the user is determined not to answer the call.

[1644] Terminal: If there is no answer after three rings, the device will switch to auto-answer mode. As soon as the call starts, it will start recording the audio data in real time. The recorded audio data will be immediately sent to the analysis module.

[1645] Output: The recorded audio data is sent to the analysis module.

[1646] Step 3:

[1647] Input: Audio data sent from the device.

[1648] Server: The server converts the received voice data into text data using voice recognition technology (e.g., Google Speech-to-Text API).

[1649] Output: Text data is generated.

[1650] Step 4:

[1651] Input: Text data sent from the server.

[1652] Server: Converts the text data into vector format and matches it with a vector database containing patterns of fraudulent calls, scoring the match using, for example, the Cosine Similarity algorithm.

[1653] Output: A match score is generated and a scam call determination is given.

[1654] Step 5:

[1655] Input: Match score and fraud call determination result.

[1656] Generative AI: If the score exceeds a threshold, it is determined to be a fraudulent call and an automatic response mode is initiated. Questions and responses are generated for the fraudster that will buy time without affecting the user. An emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the emotions in the fraudster's voice and optimizes the response based on that information. For example, it generates a response such as, "That's terrible. Can you tell me more about which outstanding balance you're referring to?"

[1657] Output: An automated response to the scammer is generated and sent.

[1658] Step 6:

[1659] Inputs: Match score and if the call was determined not to be fraudulent.

[1660] Terminal: If the call is not a fraudulent call, the terminal will transfer the call to the user, notifying the user of the call via voice guidance and a display panel.

[1661] Output: The call is transferred to the user, who prepares to answer the call.

[1662] Step 7:

[1663] Input: If the user feels lonely, they can speak a voice command to the device saying, "Talk to me."

[1664] Terminal: Recognizes voice commands and sends instructions to the generative artificial intelligence.

[1665] Output: The recognition results of the voice command are sent to the generative artificial intelligence.

[1666] Step 8:

[1667] Input: The recognition result of the voice command.

[1668] Generative AI: Initiates a conversation with the user upon receiving instructions. The emotion engine analyzes the user's voice and generates an appropriate response based on the user's emotion. For example, "Hello! What would you like to talk about today?" It also responds to user comments with, "That's lovely. Tell me what kind of flowers have bloomed."

[1669] Output: A personalized conversation is provided to the user.

[1670] Specific examples

[1671] Prompt 1: "Analyze the characteristics of scam calls and notify the user that they are scams."

[1672] Prompt 2: "Please start a fun conversation with an elderly person who lives alone and feels lonely."

[1673] (Application example 2)

[1674] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1675] The number of victims of fraudulent phone calls targeting elderly people is increasing, and as a countermeasure, a system that can identify fraudulent calls and automatically respond to them is needed. Furthermore, to reduce the sense of loneliness felt by elderly people living alone, a system that can provide daily conversation partners is also needed. This invention aims to protect elderly people from fraudulent phone calls and provide them with daily conversation partners by utilizing generative artificial intelligence and an emotion engine.

[1676] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for determining whether the call is a fraudulent call, means for the generative artificial intelligence to automatically respond to the determined fraudulent call, means for transferring the call to an elderly person if the call is not a fraudulent call, means for the generative artificial intelligence to converse with an elderly person living alone when they need someone to talk to, means for analyzing voice data and detecting emotions, and means for optimizing the response content according to the detected emotions. This allows elderly people to avoid falling victim to fraudulent calls, further reduce their sense of loneliness, and live their lives with peace of mind.

[1677] "Generative AI" is AI that has the ability to generate language and content based on user input.

[1678] A "telephone terminal" is a device with a calling function that supports calls with elderly people.

[1679] "Fraud call detection" is the process of determining whether a call is likely to be fraudulent based on the received voice data.

[1680] "Means for automatic response" refers to a system function in which artificial intelligence automatically generates and responds to identified fraudulent calls.

[1681] "Means of transferring calls to elderly people" is a function that connects normal calls that are determined not to be fraudulent calls to elderly people.

[1682] "Means for generative AI to converse when someone wants someone to talk to" refers to a function that allows AI to automatically start a conversation and respond when an elderly person requests a conversation.

[1683] The "means for analyzing voice data" is a function for analyzing received voice data and converting it into text.

[1684] "Means for detecting emotions" refers to a function that reads emotions from the content and tone of analyzed voice data.

[1685] "Means for optimizing response content" is a function that generates and provides the optimal response based on the detected emotion.

[1686] "Fraud call patterns" are a collection of data summarizing the characteristics and commonalities of past fraud calls.

[1687] A "vector database" is a database that stores and manages data in vector format and allows for comparison of similarities and features.

[1688] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. The specific system configuration and various means for realizing this are described in detail below.

[1689] System configuration

[1690] The system consists of the following main components:

[1691] Telephone terminal: A device with a calling function. Suitable for use by the elderly.

[1692] Generative artificial intelligence (AI): An AI module with natural language processing capabilities.

[1693] Emotion engine: An engine that analyzes emotions from voice data and optimizes responses.

[1694] Vector database: A database that stores patterns of fraudulent calls.

[1695] Program processing

[1696] The program of this system is realized using the following hardware and software.

[1697] Hardware:

[1698] Smartphone

[1699] IoT device with speakerphone functionality

[1700] software:

[1701] Speech recognition module (speech_recognition library)

[1702] Natural language generation module (transformers library)

[1703] Sentiment analysis module (pipeline library)

[1704] Database management system (sqlite3)

[1705] The server first detects an incoming call from the user and acquires the voice data. Next, it analyzes this voice data, converts it into text, and compares it with a vector database to determine whether it is likely to be fraud. If it is determined to be fraud, the generative AI automatically responds. On the other hand, if it is determined not to be fraud, the call can be transferred directly to the elderly person.

[1706] Furthermore, the system's emotion engine analyzes emotions from voice data and optimizes responses based on those emotions. If an elderly person living alone says, "I want someone to talk to," the system recognizes this and initiates a conversation with the generative AI, generating an appropriate response.

[1707] Specific examples

[1708] Receiving and responding to fraudulent phone calls

[1709] Situation: An elderly person receives a fraudulent phone call at their home.

[1710] User: Import audio into the system as "input_audio.wav".

[1711] Server: Analyzes received voice data in real time and converts it into text.

[1712] Server: Compares with the characteristic patterns of fraudulent calls and determines that it is a fraudulent call.

[1713] Generative AI: Responds to the scammer with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[1714] Ability to talk to elderly people

[1715] Situation: An elderly person living alone feels lonely and wants someone to talk to.

[1716] User: Say "Talk to me" to the device.

[1717] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[1718] User: "The flowers in my garden have been blooming beautifully lately."

[1719] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[1720] Examples of specific prompts include:

[1721] "Scam call voice: Mom, it's me and I want to talk to you about an unpaid bill."

[1722] The present invention enables elderly people to live without becoming victims of fraudulent phone calls, and further reduces their sense of loneliness, allowing them to live with peace of mind.

[1723] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1724] Step 1:

[1725] Terminal: The phone starts ringing. It notifies the user of the incoming call and plays the ringtone. The input is the incoming call signal and the output is the ringtone.

[1726] Step 2:

[1727] Terminal: Switch to auto-answer mode after the specified number of rings (e.g. 3 times). Input is the number of rings, output is to switch to auto-answer mode.

[1728] Step 3:

[1729] Terminal: When a call is started, the terminal records the voice data in real time and sends it to the server. The input is the call start signal, and the output is the recorded voice data.

[1730] Step 4:

[1731] Server: Converts received voice data into text using a speech recognition module (speech_recognition library). The input is recorded voice data, and the output is text data. Specifically, the audio file is analyzed and the voice is converted into text.

[1732] Step 5:

[1733] Server: Compares the text data with a vector database to determine the likelihood of a fraudulent call. The input is text data, and the output is the result of the fraudulent call determination. Specifically, it compares the data with fraudulent call patterns registered in the vector database and scores the degree of match.

[1734] Step 6:

[1735] Server: If the call is determined to be fraudulent, an automatic response is generated using generative artificial intelligence (the transformers library). The input is the fraudulent call determination result and text data, and the output is the generated response. Specifically, questions and responses are generated for the fraudster.

[1736] Step 7:

[1737] Server: Sends the generated response to the device and plays it back to the fraudster. The input is the generated response, and the output is the played response. Specifically, the emotion engine analyzes the emotion from the fraudster's voice and optimizes the response based on the results.

[1738] Step 8:

[1739] Server: If the call is determined not to be a scam, the server transfers the call to the elderly person. The input is the result of the scam call determination, and the output is the call transfer.

[1740] Step 9:

[1741] Terminal: The elderly person utters a specific command, such as "Please be my conversation partner." The input is the command voice, and the output is command recognition.

[1742] Step 10:

[1743] Generative AI: Recognizes commands and starts a conversation. The input is a spoken command, and the output is a generated conversation starter. Specifically, it responds, "Hello! What would you like to talk about today?"

[1744] Step 11:

[1745] Terminal and Generative AI: The content of what the user says is analyzed in real time, and the emotion engine analyzes the emotion. The input is the user's voice, and the output is the result of the emotion analysis. Specifically, in response to the statement, "The flowers in the garden have been blooming beautifully recently," the response generated is, "That's lovely. Please tell me what kind of flowers have bloomed."

[1746] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1747] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1748] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1749] [Fourth embodiment]

[1750] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1751] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1752] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1753] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1754] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1755] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1756] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1757] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1758] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1759] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1760] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1761] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1762] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1763] The present invention uses a telephone terminal equipped with generative artificial intelligence to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. A specific embodiment of this system is described in detail below.

[1764] System configuration

[1765] The system is built around a telephone terminal with a built-in generative artificial intelligence module. The main components are:

[1766] 1. Telephone terminal:

[1767] Ringing function when receiving a call

[1768] Ability to record incoming calls in real time and send the audio data to an analysis module

[1769] 2. Generative artificial intelligence:

[1770] Natural language processing functions for user interaction

[1771] Automated response for fraudulent calls

[1772] Daily conversation function for elderly people living alone

[1773] 3. Vector Database:

[1774] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[1775] Program processing

[1776] The program for this system mainly executes processing in the following steps. A specific processing flow and operation example are shown below.

[1777] Receiving and analyzing calls

[1778] Device:

[1779] When an incoming call occurs, the device immediately rings and automatically answers after a specified number of rings.

[1780] As soon as the call begins, the audio data is recorded in real time and sent to the analysis module.

[1781] Identifying fraudulent calls

[1782] server:

[1783] Voice recognition technology is used to convert voice data sent from the terminal into text.

[1784] The text data is compared with a vector database to determine whether it matches the characteristic patterns of fraudulent calls.

[1785] If there is a high degree of match, the call is determined to be fraudulent and the response is handed over to generative artificial intelligence.

[1786] Response to fraudulent phone calls

[1787] Generative artificial intelligence:

[1788] If a call is determined to be fraudulent, the AI ​​automatically initiates a fraudulent call and response, generating responses to the fraudster's questions that will not affect the user and buy time.

[1789] Responses are flexibly changed to prevent elderly people from becoming victims by continuing the conversation with the scammer.

[1790] Normal call handling

[1791] Device:

[1792] If the call is determined not to be a scam, the device will transfer the call to the elderly person.

[1793] The senior citizen can continue the conversation by following normal call procedures.

[1794] Features for seniors living alone

[1795] User:

[1796] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[1797] Generative artificial intelligence:

[1798] The generative AI recognizes the command and automatically starts a conversation. The generative AI generates an appropriate response to the user's utterance and continues the conversation in a natural way.

[1799] Specific examples

[1800] Receiving and responding to fraudulent phone calls

[1801] Situation: An elderly person receives a fraudulent phone call at their home.

[1802] Device: The phone starts ringing. Because it is set to auto-answer mode, the call is automatically answered after three rings.

[1803] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[1804] Server: The call is determined to be fraudulent. The generative AI begins responding.

[1805] Generative AI: Buys time by responding to scammers with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[1806] Ability to talk to elderly people

[1807] Situation: An elderly person living alone feels lonely.

[1808] User: Say "Please talk to me" to the device.

[1809] Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[1810] User: "The flowers in my garden have been blooming beautifully lately."

[1811] Generator: "That's lovely. Can you tell me what kind of flowers bloomed?"

[1812] This invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI function for everyday conversations reduces the mental burden on elderly people living alone.

[1813] The processing flow will be explained below.

[1814] Step 1:

[1815] Device:

[1816] When powered on, the terminal boots the system.

[1817] During the startup process, a self-test is performed to verify that each module is operating correctly.

[1818] Loads the generative artificial intelligence module and establishes a connection with the built-in vector database.

[1819] Step 2:

[1820] Device:

[1821] When an incoming call occurs, a ring tone sounds to notify the user.

[1822] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[1823] As soon as the call begins, audio data recording begins and is sent to the analysis module in real time.

[1824] Step 3:

[1825] server:

[1826] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[1827] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[1828] If the score exceeds the threshold, the call is determined to be fraudulent.

[1829] Step 4:

[1830] Generative artificial intelligence:

[1831] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[1832] Generate questions and responses that will buy time for fraudsters without affecting the user.

[1833] Responses can be flexibly changed, and conversations with scammers are constantly recorded and analyzed.

[1834] Step 5:

[1835] Device:

[1836] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[1837] The normal call process will be resumed and the call will not be recorded.

[1838] Step 6:

[1839] User:

[1840] If an elderly person living alone feels lonely, the robot will issue a specific command (e.g., "talk to me") via voice command.

[1841] Device:

[1842] It recognizes commands and sends instructions to generative artificial intelligence.

[1843] Generative artificial intelligence:

[1844] It receives commands and starts a conversation with the user, using natural language processing to provide appropriate responses to what the user says.

[1845] The conversation content is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[1846] Step 7:

[1847] Device:

[1848] When the call or conversation ends, the communication records and analysis results are saved.

[1849] Safely shuts down the generative artificial intelligence module and puts it into standby mode until next use.

[1850] This system not only protects seniors from fraudulent phone calls, but also serves as a daily conversation partner, improving their sense of security and quality of life.

[1851] Example 1

[1852] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1853] The risk of elderly people becoming victims of fraudulent phone calls is increasing. In addition, elderly people who live alone often feel lonely because they have no one to talk to on a daily basis. To solve these issues, a system is needed that can identify fraudulent calls, automatically answer them, and provide elderly people living alone with someone to talk to on a daily basis.

[1854] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1855] In this invention, the server includes a means for ringing and switching to automatic answering when an incoming call is received, a means for recording voice data as soon as the call begins and sending it to an analysis module, a means for converting the received voice data into text and comparing it with fraudulent call characteristic patterns, a means for calculating the degree of match and automatically answering the call using a generative artificial intelligence if the call is determined to be fraudulent, a means for transferring the call to an elderly person if the call is not fraudulent, and a means for the generative artificial intelligence to converse when an elderly person living alone needs someone to talk to. This makes it possible to prevent damage caused by fraudulent calls and ensure the safety of the elderly. It also makes it possible to reduce the sense of loneliness felt by elderly people living alone.

[1856] "Means for ringing and switching to automatic answering when an incoming call occurs" refers to the function by which a telephone terminal detects an incoming call and automatically starts the call after ringing a predetermined number of times.

[1857] "Means for recording voice data as soon as a call starts and sending it to an analysis module" refers to a function for recording voice in real time as soon as a call starts and sending that data to a module for data analysis.

[1858] "Means for converting received voice data into text and comparing it with fraudulent call characteristic patterns" refers to a function that uses voice recognition technology to convert recorded voice data into text data and compares the text with predefined fraudulent call characteristic patterns.

[1859] "Means for calculating the degree of match and for the generative artificial intelligence to automatically respond if the call is determined to be fraudulent" refers to a function that evaluates the possibility of a fraudulent call based on the degree of match of the matching results, and for the generative artificial intelligence to automatically respond if the degree of match is high.

[1860] "Means of transferring calls to elderly people if they are not fraudulent calls" refers to a function that notifies elderly people of calls that are determined to be non-scam calls, allowing elderly people to actually make the calls.

[1861] "Means for generative AI to converse when an elderly person living alone wants someone to talk to" refers to the function of generative AI to engage in natural dialogue when an elderly person living alone utters a specific command.

[1862] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence to provide elderly people with everyday conversation partners while protecting them from fraudulent phone calls. The system's main components are a telephone terminal, a server, generative artificial intelligence, and a vector database.

[1863] Main components

[1864] 1. Telephone terminal

[1865] Device:

[1866] When an incoming call occurs, the device will ring and after a specified number of rings (e.g., three times) will switch to auto-answer mode.

[1867] As soon as the call begins, the audio data is recorded in real time and sent to the server.

[1868] 2. Server

[1869] server:

[1870] It receives voice data sent from the device and converts it into text using voice recognition technology (e.g., Google Cloud Speech-to-Text).

[1871] The text data is compared with characteristic patterns of fraudulent calls stored in a vector database (e.g., Elasticsearch) and the degree of match is calculated.

[1872] If the degree of match is high, the call is determined to be fraudulent and the generative artificial intelligence is notified.

[1873] 3. Generative artificial intelligence

[1874] Generative artificial intelligence:

[1875] If a call is determined to be fraudulent, the AI ​​will automatically respond to the call, generating responses to the fraudster's questions that will not affect the user and buy time.

[1876] When an elderly person living alone wants someone to talk to, the system provides natural dialogue in response to user instructions.

[1877] 4. Vector Database

[1878] Vector Database:

[1879] It stores characteristic patterns of fraudulent calls and compares them with text data sent from the server to calculate the degree of match.

[1880] Specific examples

[1881] Receiving and responding to fraudulent phone calls

[1882] Situation: An elderly person receives a fraudulent phone call at their home.

[1883] 1. Device: The phone starts ringing. After three rings, the phone automatically answers the call because it is set to auto-answer mode.

[1884] 2. Terminal: Records audio data in real time and sends it to the server.

[1885] 3. Server: The received voice data is converted into text using Google Cloud Speech-to-Text, and compared with fraudulent call patterns stored in Elasticsearch, a vector database.

[1886] 4. Server: Calculates the degree of match and determines whether the call is fraudulent. Notifies the generative AI.

[1887] 5. Generative AI: The system responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" and continues the conversation.

[1888] Ability to talk to elderly people

[1889] Situation: An elderly person living alone feels lonely.

[1890] 1. User: Speaks to the device, "Please talk to me."

[1891] 2. Generative AI: Starts with "Hello! What would you like to talk about today?" and continues the conversation with the user.

[1892] 3. User: "The flowers in my garden have been blooming beautifully lately."

[1893] 4. Generative AI: "That's lovely! Tell me what kind of flowers bloomed."

[1894] In this way, this invention not only allows elderly people to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud. Furthermore, the generative AI-based everyday conversation function reduces the mental burden on elderly people living alone.

[1895] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1896] Step 1:

[1897] Device:

[1898] Input: Incoming phone signal.

[1899] What it does: Rings when an incoming call comes in.

[1900] Output: The phone will start ringing. After the specified number of rings (e.g. 3), the phone will switch to auto-answer mode.

[1901] Specific operation: After three rings, the message "Now entering auto answer mode" will be played.

[1902] Step 2:

[1903] Device:

[1904] Input: Call initiation signal and audio data.

[1905] How it works: When a call starts, the audio data is recorded in real time and sent to the server.

[1906] Output: Recorded audio data.

[1907] Specific operation: Recording begins as soon as the call begins, and the voice data is divided into a certain number of data packets, encoded, and transferred to the server.

[1908] Step 3:

[1909] server:

[1910] Input: Audio data sent from the device.

[1911] How it works: Uses speech recognition technology (e.g., Google Cloud Speech-to-Text) to convert audio data into text.

[1912] Output: Textualized data.

[1913] Specific operation: The server receives the voice data and calls a speech recognition API to convert the data into text.

[1914] Step 4:

[1915] server:

[1916] Input: Textual data.

[1917] Operation: Compare the text data with the characteristic patterns of fraudulent calls stored in a vector database (e.g., Elasticsearch). Calculate the degree of match of the matching results.

[1918] Output: Matching result.

[1919] Specific operation: After the server receives the text data, it inputs the feature pattern vector into a matching algorithm, and if the degree of match is above a certain level, it generates a result that says "possibly a fraudulent call."

[1920] Step 5:

[1921] server:

[1922] Input: Matching result.

[1923] Operation: If there is a high match, the generative artificial intelligence is notified.

[1924] Output: Notification if the call is determined to be fraudulent.

[1925] Specific operation: The judgment result is sent to generative artificial intelligence, and a notification is sent stating that "there is a high possibility that the call is fraudulent."

[1926] Step 6:

[1927] Generative artificial intelligence:

[1928] Enter: Scam call notification.

[1929] How it works: If a call is determined to be fraudulent, the AI ​​automatically responds, generating a response that won't affect the user and buys time for the fraudster.

[1930] Output: Response to the scammer.

[1931] What it does: Generates enticing statements, such as "That's terrible! Which outstanding balance are you talking about?"

[1932] Step 7:

[1933] Device:

[1934] Input: If the server determines that the call is not a scam.

[1935] Operation: If the call is determined not to be a scam, the call is transferred to an elderly person.

[1936] Output: Call transfer.

[1937] Specific actions: The system conveys the message "This call is normal. Please continue speaking" to the elderly person and switches the call over to the elderly person.

[1938] Step 8:

[1939] User:

[1940] Enter: If you feel lonely.

[1941] Action: Say "Let me talk to you" to the device.

[1942] Output: The command passed.

[1943] Specific action: The user commands the device to "be a conversation partner."

[1944] Step 9:

[1945] Generative artificial intelligence:

[1946] Input: Commands from the user.

[1947] How it works: When an elderly person living alone needs someone to talk to, the AI ​​automatically starts a conversation.

[1948] Output: Natural dialogue.

[1949] Specific behavior: The generative AI responds, "Hello! What would you like to talk about today?" and provides an appropriate response to the user's statement, such as, "The flowers in the garden have been blooming beautifully recently," and continues, "That's wonderful. Please tell me what kind of flowers have bloomed."

[1950] This allows elderly people to use the telephone with peace of mind and reduces feelings of loneliness. The automated answering function for fraudulent calls protects elderly people from becoming victims of fraud, and the everyday conversation function using generative artificial intelligence reduces the mental burden on elderly people living alone.

[1951] (Application example 1)

[1952] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1953] This invention relates to the provision of a system for protecting elderly people from fraud. Specifically, the objective is to protect elderly people from fraud by detecting possible fraud and automatically taking action, and to provide support for elderly people to enjoy shopping and staying in stores with peace of mind, thereby reducing the sense of loneliness felt by elderly people when they are alone.

[1954] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1955] In this invention, the server includes means for determining the possibility of fraud, means for a generative artificial intelligence to automatically respond to the determined fraud, means for transferring communication to an elderly person if fraud is not an attempt, means for the generative artificial intelligence to converse with an elderly person living alone when the elderly person wants someone to talk to, means for providing navigation and product explanations to elderly people visiting a store, means for analyzing conversations with suspicious people near the store and detecting fraud and means for issuing a warning based on the detection result, and means for the generative artificial intelligence to support the conversation when the elderly person is looking for someone to talk to. This protects elderly people from fraud, improves their shopping experience in stores, and reduces the sense of loneliness they feel when they are alone.

[1956] "Generative AI" is an AI technology that has the ability to generate new data and answers based on input data.

[1957] A "communication terminal" is an electronic device for sending and receiving voice and data.

[1958] "Fraud" is the act of deceiving others through dishonest means and stealing their property or information.

[1959] A "means of judgment" is a method or technique for making an appropriate judgment about an event or data.

[1960] "Means for automatic response" is a function that automatically generates a response to an external input without human intervention.

[1961] A "relaying means" is a method or device for relaying specific information or communication to a designated party.

[1962] "When someone wants someone to talk to" refers to a situation in which the other person wants to have a conversation.

[1963] "Navigation" is a function that provides guidance on the route to a destination.

[1964] "Product Description" is detailed information about the characteristics and usage of the product being offered.

[1965] "Means of analysis" refers to methods and techniques for analyzing input data in detail and extracting meaning and patterns.

[1966] "Warning means" means a method or technique for notifying people of a particular danger or abnormality.

[1967] "Means to support conversation" are auxiliary functions to facilitate smooth dialogue with the other person.

[1968] The system of the present invention uses a communication terminal equipped with generative artificial intelligence to protect elderly people from fraud and provide elderly people living alone with a daily conversation partner. It also provides support for elderly people to enjoy shopping and staying in physical stores with peace of mind. Specific embodiments of the present invention are described in detail below.

[1969] System configuration

[1970] The system is built around a communications terminal with a built-in generative artificial intelligence module, and its main components are as follows:

[1971] 1. Communication terminal:

[1972] Ability to receive communications and perform specified actions

[1973] A function that analyzes received voice data in real time and sends it to the server

[1974] 2. Generative artificial intelligence:

[1975] Natural language processing functions for user interaction

[1976] Automatic fraud detection function

[1977] Daily conversation function for elderly people living alone

[1978] Functions that provide in-store navigation and product information

[1979] 3. Server:

[1980] A function that stores characteristic patterns of fraudulent activity, converts voice data into text, and compares it with those patterns

[1981] 4. Warning system:

[1982] A function that analyzes interactions with suspicious people near stores, detects fraudulent activity, and issues a warning.

[1983] 5. Interface:

[1984] A user interface designed to make communication devices easier for the elderly to operate

[1985] Hardware and software used

[1986] Hardware:

[1987] Communication device: A smartphone with audio input (microphone) and audio output (speaker)

[1988] software:

[1989] Speech recognition: using HuggingFace's transformer pipeline

[1990] Natural Language Processing: Using spaCy

[1991] Generative AI: Using OpenAI's GPT-3.5-turbo model

[1992] Data processing and calculation

[1993] The server uses a speech recognition module to convert voice data into text data. It then uses a natural language processing module to analyze the text data and compare it with characteristic patterns of fraudulent activity. The generative artificial intelligence module generates natural dialogue and acts as a pseudo-conversational partner for the elderly. It also provides in-store navigation and product information to help seniors enjoy their shopping experience.

[1994] Specific examples

[1995] Dealing with fraud

[1996] Situation: An elderly person receives a fraudulent phone call at their home.

[1997] Communication terminal: When the call starts ringing, it will be set to auto-answer mode after the specified number of rings. The voice data will be analyzed in real time and sent to the server.

[1998] Server: Converts the received voice data into text and compares it with characteristic patterns of fraudulent activity.

[1999] Server: If fraudulent activity is determined, the generative artificial intelligence will automatically initiate a response, generating a response that will not have any impact on the fraudster.

[2000] In-store navigation and product explanations

[2001] Situation: An elderly person is searching for a product in a physical store.

[2002] User: Speak into the communication device, "Please tell me where I need navigation."

[2003] Generative AI: Asks, "Which section should I go to?" and provides navigation and product descriptions.

[2004] Prompt Sentence Examples

[2005] Dealing with fraud: "I was told I have outstanding bills. What should I do?"

[2006] In-store navigation: "Tell me where the milk is."

[2007] Conversation between an elderly person living alone: ​​"The flowers in my garden have been blooming beautifully recently."

[2008] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2009] Step 1:

[2010] The communication terminal receives a communication and rings. After a specified number of rings, it switches to an auto-answer mode, where the input includes the incoming voice and the output is that the auto-answer mode is activated.

[2011] Step 2:

[2012] The communication terminal records the received voice data in real time and transmits the data to the server. The input includes the received voice data, and the output includes the transmitted voice data.

[2013] Step 3:

[2014] The server uses a speech recognition module to convert the transmitted voice data into text data, where the input includes the voice data and the output is the text data.

[2015] Step 4:

[2016] The server uses a natural language processing module (e.g., spaCy) to analyze the generated text data and compare it with fraud feature patterns, where the input includes the text data and a fraud pattern database, and the output generates a match determination for fraud.

[2017] Step 5:

[2018] The server determines whether the fraudulent activity is occurring, and if a high degree of match is found, the generative artificial intelligence initiates an automatic response. The input includes the result of the match determination, and the output is an automatic response message.

[2019] Step 6:

[2020] The server uses generative artificial intelligence to generate an automatic response message and sends it back to the communication terminal. The input includes the match determination result and a prompt sentence, and the automatic response message is generated as the output.

[2021] Step 7:

[2022] The communication terminal plays the received automatic response message and makes an appropriate response to the fraudulent activity, where the input includes the automatic response message from the server and the output is the played response message.

[2023] Step 8:

[2024] If it is not fraudulent, the communication terminal transfers the call to the elderly person. The input includes the result of the match determination, and the output is that the call is transferred to the elderly person.

[2025] Step 9:

[2026] When an elderly person needs someone to talk to, they say "please talk to me" through the communication terminal and send it to the server. The input includes the elderly person's voice instruction, and the output is the transmission of voice data to the server.

[2027] Step 10:

[2028] The server uses generative artificial intelligence to analyze the user's instructions and initiate a conversation with the elderly, where the input includes voice instructions and prompts, and the output generates a conversational content.

[2029] Step 11:

[2030] The server sends the generated conversation content back to the communication terminal, which then plays it back. The input includes the generated conversation content, and the output is a conversation with the elderly person.

[2031] Step 12:

[2032] When an elderly person needs navigation or product explanations in a store, they can say "Tell me where I need navigation" through a communication terminal and send it to the server. The input includes the elderly person's voice instructions, and the output is the transmission of voice data to the server.

[2033] Step 13:

[2034] The server uses generative artificial intelligence to analyze the user's instructions and provide navigation and product descriptions, where inputs include voice instructions and store data, and outputs generate navigation and product descriptions.

[2035] Step 14:

[2036] The server sends the generated navigation content and product description back to the communication terminal, which then plays them. The input includes the generated navigation content and product description, and the output is a voice guide for the elderly.

[2037] Step 15:

[2038] When an elderly person is approached by a suspicious person near a store, the communication terminal receives the conversation and transmits it to the server in real time. The input includes the received voice data, and the output is the transmission of the voice data to the server.

[2039] Step 16:

[2040] The server analyzes the received voice data using a natural language processing module to determine the likelihood of fraud, where the input includes the voice data and a fraud pattern database, and the output generates a fraud match determination.

[2041] Step 17:

[2042] If the match is determined to be high, the server sends a warning message back to the communication terminal, which then plays it back. The input includes the match determination result and the warning message, and the output is the played warning message.

[2043] Step 18:

[2044] If the degree of match is determined to be low, the communication terminal maintains its normal state without issuing any particular warning. The input includes the result of the degree of match determination, and the output maintains the normal state without any warning.

[2045] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2046] The present invention uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a daily conversation partner. A specific embodiment of this system is described in detail below.

[2047] System configuration

[2048] The system is built around a phone terminal with a built-in generative artificial intelligence module and emotion engine. The main components are:

[2049] 1. Telephone terminal:

[2050] Ringing function when receiving a call

[2051] Ability to record incoming calls in real time and send the audio data to an analysis module

[2052] 2. Generative artificial intelligence:

[2053] Natural language processing functions for user interaction

[2054] Automated response for fraudulent calls

[2055] Daily conversation function for elderly people living alone

[2056] 3. Emotion Engine:

[2057] A function that analyzes emotions from user and fraudster voice data and optimizes generative AI responses

[2058] 4. Vector Database:

[2059] A function that stores characteristic patterns of fraudulent calls, converts voice data into text, and compares it with those patterns

[2060] Program processing

[2061] The program of this system executes the process in the following procedure.

[2062] Receiving and analyzing calls

[2063] Device:

[2064] When an incoming call occurs, a ring tone sounds to notify the user.

[2065] When the ring tone is repeated a specified number of times (e.g. 3 times), the phone will switch to auto answer mode.

[2066] As soon as a call is initiated, audio data is recorded in real time and sent to the analysis module.

[2067] Identifying fraudulent calls

[2068] server:

[2069] Voice data sent from the terminal is received and converted into text data using voice recognition technology.

[2070] The text data is compared with fraudulent call patterns registered in a vector database, and the degree of match is scored.

[2071] If the score exceeds the threshold, the call is determined to be fraudulent.

[2072] Response to fraudulent phone calls

[2073] Generative artificial intelligence:

[2074] If the call is determined to be fraudulent, an automatic response mode will be initiated.

[2075] Generate questions and responses that will buy time for fraudsters without affecting the user.

[2076] The emotion engine analyzes emotions from the fraudster's voice and optimizes responses accordingly.

[2077] Normal call handling

[2078] Device:

[2079] If it is determined that the call is not a scam, the call is transferred to an elderly user.

[2080] The normal call process will be resumed and the call will not be recorded.

[2081] Features for seniors living alone

[2082] User:

[2083] When a user feels lonely, they speak a specific command to the device (e.g., "talk to me").

[2084] Device:

[2085] It recognizes commands and sends instructions to generative artificial intelligence.

[2086] Generative artificial intelligence:

[2087] Receives commands and initiates a conversation with the user.

[2088] The emotion engine analyzes emotions from the user's voice and generates appropriate responses based on those emotions.

[2089] The content of the conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[2090] Specific examples

[2091] Receiving and responding to fraudulent phone calls

[2092] Situation: An elderly person receives a fraudulent phone call at their home.

[2093] Device: The phone starts ringing and switches to auto-answer mode after three rings.

[2094] Server: Analyzes received voice data in real time, converts it into text, and compares it with characteristic patterns of fraudulent calls.

[2095] Server: The call is determined to be fraudulent. Generative AI begins responding.

[2096] Generative AI: Responds to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[2097] Ability to talk to elderly people

[2098] Situation: An elderly person living alone feels lonely.

[2099] User: Say "Talk to me" to the device.

[2100] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[2101] User: "The flowers in my garden have been blooming beautifully lately."

[2102] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[2103] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[2104] The processing flow will be explained below.

[2105] Step 1:

[2106] Device:

[2107] When powered on, the device boots up the system.

[2108] During the boot process, a self-diagnosis is performed to ensure that each module is working properly.

[2109] It loads a generative artificial intelligence module and an emotion engine, and establishes a connection with the built-in vector database.

[2110] Step 2:

[2111] Device:

[2112] When an incoming call occurs, the phone will ring to notify the user.

[2113] When the ring tone is repeated a specified number of times (e.g. 3 times), it will switch to auto answer mode.

[2114] As soon as the call begins, audio data begins recording and is sent to the analysis module in real time.

[2115] Step 3:

[2116] server:

[2117] It receives voice data sent from the device and converts it into text data using voice recognition technology.

[2118] The text data is compared with fraudulent call patterns registered in a vector database and the degree of match is scored.

[2119] If the score exceeds the threshold, the call is deemed to be fraudulent.

[2120] Step 4:

[2121] Generative artificial intelligence:

[2122] If the call is determined to be fraudulent, an automatic answering mode will be initiated.

[2123] The idea is to generate questions and responses that buy the scammers time without impacting the user.

[2124] The emotion engine analyzes the emotions in the scammer's voice and optimizes the response accordingly.

[2125] Step 5:

[2126] Device:

[2127] If the call is determined not to be a scam, the call will be transferred to an elderly user.

[2128] The normal call process will be used and the call will not be recorded.

[2129] Step 6:

[2130] User:

[2131] If an elderly person living alone feels lonely, they can issue a specific command (e.g., "talk to me") via voice command.

[2132] Device:

[2133] It recognizes commands and sends instructions to generative artificial intelligence.

[2134] Generative artificial intelligence:

[2135] It receives commands, initiates a conversation with the user, and uses natural language processing to provide appropriate responses to what the user says.

[2136] The emotion engine analyzes the emotions from the user's voice and generates appropriate responses based on those emotions.

[2137] The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[2138] Step 7:

[2139] Device:

[2140] Once the call or conversation ends, the communication records and analysis results are saved.

[2141] The generative artificial intelligence module will be safely shut down and put into standby mode until next use.

[2142] Specific examples

[2143] Receiving and responding to fraudulent phone calls

[2144] Situation: An elderly person receives a fraudulent phone call at their home.

[2145] Device: The phone will start ringing and will switch to auto-answer mode after three rings.

[2146] Server: Analyzes the received voice data in real time, converts it into text, and compares it with the characteristic patterns of fraudulent calls.

[2147] Server: The call is determined to be fraudulent. The generative AI begins responding.

[2148] Generative AI: Responding to the scammer with something like, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the scammer's emotions and optimizes the response based on those results.

[2149] Ability to talk to elderly people

[2150] Situation: An elderly person living alone feels lonely.

[2151] User: Speak into the device, "Talk to me."

[2152] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[2153] User: "The flowers in my garden have been blooming beautifully lately."

[2154] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[2155] This invention not only allows seniors to use the telephone with peace of mind, but also reduces feelings of loneliness. The automated answering function for fraudulent calls protects seniors from becoming victims of fraud. Furthermore, the everyday conversation function using generative artificial intelligence and an emotion engine reduces the mental burden on seniors living alone.

[2156] Example 2

[2157] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2158] There is a need for a communication system that reduces the risk of elderly people becoming victims of fraudulent phone calls and allows elderly people living alone to live safely and comfortably without feeling lonely. Fraudulent phone calls are becoming increasingly sophisticated, making it extremely difficult for elderly people to identify them. Furthermore, elderly people living alone often have no one to talk to on a daily basis, which makes them prone to psychological loneliness. It is necessary to solve these problems simultaneously.

[2159] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2160] In this invention, the server includes means for ringing when a call is received, means for recording voice data in real time and sending it to an analysis module, means for determining whether a call is likely to be a fraudulent call, means for a generative artificial intelligence to automatically respond to the determined fraudulent call, means for analyzing emotions from the user's voice data and optimizing the response, means for transferring the call to the user if the call is not a fraudulent call, and means for the generative artificial intelligence to converse when a user living alone requests someone to talk to. This enables early detection and response of fraudulent calls, not only protecting the elderly from fraud but also providing communication that reduces their everyday feelings of loneliness.

[2161] The "means for ringing when a call is received" is a function that notifies the user by voice when a call is received.

[2162] The "means for recording voice data in real time and transmitting it to an analysis module" is a function for instantly recording voice during a call and transmitting the data for analysis.

[2163] The "means for determining the possibility of a fraudulent call" is a function that includes an algorithm that analyzes the content of a received call and determines whether it is a fraudulent call.

[2164] "Means for automated response by generative artificial intelligence" is a function that automatically generates appropriate responses to identified fraudulent phone calls and continues the dialogue with the fraudster.

[2165] "Means for analyzing emotions from user voice data and optimizing responses" is a function that analyzes the voice during a call to determine the user's emotional state and generates appropriate response content based on that.

[2166] The "means of transferring the call to the user if it is not a fraudulent call" is a function of directly connecting the call to the user if it is determined that the call is not a fraudulent call.

[2167] "A means for generative AI to converse when a user living alone wants someone to talk to" is a function that, when a user feels lonely, receives that command and the AI ​​starts a dialogue and communicates with the user.

[2168] "Means for converting voice data to text" refers to a function that includes an algorithm for converting the voice content of a call into text information.

[2169] The "means for comparing with fraudulent call patterns" is a function for comparing the converted text with characteristic patterns of fraudulent calls registered in advance and evaluating the degree of match.

[2170] The "feature vector database" is a database that stores vector data that quantifies the characteristics of fraudulent calls and is used for comparative analysis.

[2171] This invention is a system that protects elderly people from fraudulent phone calls and provides elderly people living alone with a regular conversation partner. A specific implementation is built around a telephone terminal equipped with a generative artificial intelligence module and an emotion engine. The system configuration is as follows:

[2172] System Components

[2173] 1. Telephone terminal

[2174] Hardware: Includes a speaker that rings when a call comes in, a microphone that records audio in real time, and a communication module that sends the recorded data to the analysis module.

[2175] Software: Equipped with automatic answering function and command recognition function (e.g. Amazon Alexa Voice Service).

[2176] 2. Generative artificial intelligence

[2177] Hardware: A server with a powerful processor and sufficient memory.

[2178] Software: AI modules with natural language processing and dialogue generation capabilities, such as automated answering of fraudulent phone calls and everyday conversations with the elderly (e.g., using models such as GPT-3).

[2179] 3. Emotion Engine

[2180] Software: Algorithms that analyze emotions from voice data and optimize generative AI responses (e.g., IBM Watson Tone Analyzer).

[2181] 4. Vector Database

[2182] Hardware: A database server with high-speed access.

[2183] Software: A database that quantifies and stores characteristic patterns of fraudulent calls. APIs (e.g., Cosine Similarity algorithms) that convert voice data into text and match it with the patterns.

[2184] How it works

[2185] Receiving and analyzing calls

[2186] When a call comes in, the device will ring to notify the user. If there is no answer after three rings, the device will switch to auto-answer mode. Once the call is initiated, the device will record audio data in real time and send it to the analysis module.

[2187] Identifying fraudulent calls

[2188] The server receives the voice data sent from the device and converts it into text data using speech recognition technology. The converted text data is compared with fraudulent call patterns registered in a vector database and a score is assigned to determine the degree of match. If the score exceeds a threshold, the call is determined to be fraudulent.

[2189] Responding to fraudulent phone calls

[2190] If the generative AI determines that the call is a scam, it will initiate an automatic response mode, generating questions and responses to buy the scammer time without impacting the user. The emotion engine analyzes the emotions in the scammer's voice and optimizes the response based on that information.

[2191] Normal call handling

[2192] If the device determines that the call is not a scam, it will transfer the call to the user. The call will not be recorded.

[2193] Conversation starter for elderly people living alone

[2194] If the user feels lonely, they can say to the terminal, "Please talk to me."

[2195] The terminal recognizes this command and sends instructions to the generative artificial intelligence.

[2196] The generative AI receives instructions and begins a conversation with the user. The emotion engine analyzes the user's voice and generates appropriate responses based on their emotions. The conversation is recorded on the device, and the device references the user's past comments and conversation history to provide more natural and personalized responses.

[2197] Specific examples

[2198] Receiving and responding to fraudulent phone calls

[2199] Situation: An elderly person receives a fraudulent phone call at their home.

[2200] Device: The phone starts ringing. After three rings, it switches to auto-answer mode. During this time, the device's display panel will show "Incoming call" and the LED will flash.

[2201] Server: Processes the received voice data in real time and converts it into text. It compares it with the characteristic patterns of fraudulent calls. As a result, it determines that "this is likely to be a fraud."

[2202] Generative AI: The system responds to the fraudster by saying, "That's terrible. Can you tell me more about which outstanding balance you're referring to?" The emotion engine analyzes the fraudster's emotions. For example, if the fraudster is frustrated, the system responds in a calming tone.

[2203] Ability to talk to elderly people

[2204] Situation: An elderly person living alone feels lonely.

[2205] User: Say "Talk to me" to the device.

[2206] Generative AI: Recognizes commands and starts a conversation by saying, "Hello! What would you like to talk about today?"

[2207] User: "The flowers in my garden have been blooming beautifully lately."

[2208] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[2209] Example of input prompt for the generative AI model to be used

[2210] Prompt 1: Analyze the characteristics of the scam call and inform the user that it is a scam.

[2211] Prompt 2: Please start a fun conversation with an elderly person who lives alone and feels lonely.

[2212] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2213] Program processing steps

[2214] Step 1:

[2215] Input: When a call comes in, a signal is sent to the telephone terminal.

[2216] Terminal: When a call comes in, the terminal will ring to notify the user, and the display panel will show "Incoming call" and the LED will flash.

[2217] Output: The user hears the phone ringing or sees the device's display panel and a flashing LED.

[2218] Step 2:

[2219] Input: After three rings or when the user is determined not to answer the call.

[2220] Terminal: If there is no answer after three rings, the device will switch to auto-answer mode. As soon as the call starts, it will start recording the audio data in real time. The recorded audio data will be immediately sent to the analysis module.

[2221] Output: The recorded audio data is sent to the analysis module.

[2222] Step 3:

[2223] Input: Audio data sent from the device.

[2224] Server: The server converts the received voice data into text data using voice recognition technology (e.g., Google Speech-to-Text API).

[2225] Output: Text data is generated.

[2226] Step 4:

[2227] Input: Text data sent from the server.

[2228] Server: Converts the text data into vector format and matches it with a vector database containing patterns of fraudulent calls, scoring the match using, for example, the Cosine Similarity algorithm.

[2229] Output: A match score is generated and a scam call determination is given.

[2230] Step 5:

[2231] Input: Match score and fraud call determination result.

[2232] Generative AI: If the score exceeds a threshold, it is determined to be a fraudulent call and an automatic response mode is initiated. Questions and responses are generated for the fraudster that will buy time without affecting the user. An emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the emotions in the fraudster's voice and optimizes the response based on that information. For example, it generates a response such as, "That's terrible. Can you tell me more about which outstanding balance you're referring to?"

[2233] Output: An automated response to the scammer is generated and sent.

[2234] Step 6:

[2235] Inputs: Match score and if the call was determined not to be fraudulent.

[2236] Terminal: If the call is not a fraudulent call, the terminal will transfer the call to the user, notifying the user of the call via voice guidance and a display panel.

[2237] Output: The call is transferred to the user, who prepares to answer the call.

[2238] Step 7:

[2239] Input: If the user feels lonely, they can speak a voice command to the device saying, "Talk to me."

[2240] Terminal: Recognizes voice commands and sends instructions to the generative artificial intelligence.

[2241] Output: The recognition results of the voice command are sent to the generative artificial intelligence.

[2242] Step 8:

[2243] Input: The recognition result of the voice command.

[2244] Generative AI: Initiates a conversation with the user upon receiving instructions. The emotion engine analyzes the user's voice and generates an appropriate response based on the user's emotion. For example, "Hello! What would you like to talk about today?" It also responds to user comments with, "That's lovely. Tell me what kind of flowers have bloomed."

[2245] Output: A personalized conversation is provided to the user.

[2246] Specific examples

[2247] Prompt 1: "Analyze the characteristics of scam calls and notify the user that they are scams."

[2248] Prompt 2: "Please start a fun conversation with an elderly person who lives alone and feels lonely."

[2249] (Application example 2)

[2250] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2251] The number of victims of fraudulent phone calls targeting elderly people is increasing, and as a countermeasure, a system that can identify fraudulent calls and automatically respond to them is needed. Furthermore, to reduce the sense of loneliness felt by elderly people living alone, a system that can provide daily conversation partners is also needed. This invention aims to protect elderly people from fraudulent phone calls and provide them with daily conversation partners by utilizing generative artificial intelligence and an emotion engine.

[2252] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for determining whether the call is a fraudulent call, means for the generative artificial intelligence to automatically respond to the determined fraudulent call, means for transferring the call to an elderly person if the call is not a fraudulent call, means for the generative artificial intelligence to converse with an elderly person living alone when they need someone to talk to, means for analyzing voice data and detecting emotions, and means for optimizing the response content according to the detected emotions. This allows elderly people to avoid falling victim to fraudulent calls, further reduce their sense of loneliness, and live their lives with peace of mind.

[2253] "Generative AI" is AI that has the ability to generate language and content based on user input.

[2254] A "telephone terminal" is a device with a calling function that supports calls with elderly people.

[2255] "Fraud call detection" is the process of determining whether a call is likely to be fraudulent based on the received voice data.

[2256] "Means for automatic response" refers to a system function in which artificial intelligence automatically generates and responds to identified fraudulent calls.

[2257] "Means of transferring calls to elderly people" is a function that connects normal calls that are determined not to be fraudulent calls to elderly people.

[2258] "Means for generative AI to converse when someone wants someone to talk to" refers to a function that allows AI to automatically start a conversation and respond when an elderly person requests a conversation.

[2259] The "means for analyzing voice data" is a function for analyzing received voice data and converting it into text.

[2260] "Means for detecting emotions" refers to a function that reads emotions from the content and tone of analyzed voice data.

[2261] "Means for optimizing response content" is a function that generates and provides the optimal response based on the detected emotion.

[2262] "Fraud call patterns" are a collection of data summarizing the characteristics and commonalities of past fraud calls.

[2263] A "vector database" is a database that stores and manages data in vector format and allows for comparison of similarities and features.

[2264] This invention is a system that uses a telephone terminal equipped with generative artificial intelligence and an emotion engine to protect elderly people from fraudulent phone calls and provide elderly people living alone with a regular conversation partner. The specific system configuration and various means for realizing this are described in detail below.

[2265] System configuration

[2266] The system consists of the following main components:

[2267] Telephone terminal: A device with a calling function. Suitable for use by the elderly.

[2268] Generative artificial intelligence (AI): An AI module with natural language processing capabilities.

[2269] Emotion engine: An engine that analyzes emotions from voice data and optimizes responses.

[2270] Vector database: A database that stores patterns of fraudulent calls.

[2271] Program processing

[2272] The program of this system is realized using the following hardware and software.

[2273] Hardware:

[2274] Smartphone

[2275] IoT device with speakerphone functionality

[2276] software:

[2277] Speech recognition module (speech_recognition library)

[2278] Natural language generation module (transformers library)

[2279] Sentiment analysis module (pipeline library)

[2280] Database management system (sqlite3)

[2281] The server first detects an incoming call from the user and acquires the voice data. Next, it analyzes this voice data, converts it into text, and compares it with a vector database to determine whether it is likely to be fraud. If it is determined to be fraud, the generative AI automatically responds. On the other hand, if it is determined not to be fraud, the call can be transferred directly to the elderly person.

[2282] Furthermore, the system's emotion engine analyzes emotions from voice data and optimizes responses based on those emotions. If an elderly person living alone says, "I want someone to talk to," the system recognizes this and initiates a conversation with the generative AI, generating an appropriate response.

[2283] Specific examples

[2284] Receiving and responding to fraudulent phone calls

[2285] Situation: An elderly person receives a fraudulent phone call at their home.

[2286] User: Import audio into the system as "input_audio.wav".

[2287] Server: Analyzes received voice data in real time and converts it into text.

[2288] Server: Compares with the characteristic patterns of fraudulent calls and determines that it is a fraudulent call.

[2289] Generative AI: Responds to the scammer with something like, "That's a pity. Can you tell me more about which outstanding balance you're referring to?"

[2290] Ability to talk to elderly people

[2291] Situation: An elderly person living alone feels lonely and wants someone to talk to.

[2292] User: Say "Talk to me" to the device.

[2293] Generative AI: Recognizes commands and begins a conversation by saying, "Hello! What would you like to talk about today?"

[2294] User: "The flowers in my garden have been blooming beautifully lately."

[2295] Generative AI: The emotion engine analyzes the user's emotions and generates an appropriate response, such as, "That's lovely. Tell me what kind of flowers have bloomed."

[2296] Examples of specific prompts include:

[2297] "Scam call voice: Mom, it's me and I want to talk to you about an unpaid bill."

[2298] The present invention enables elderly people to live without becoming victims of fraudulent phone calls, and further reduces their sense of loneliness, allowing them to live with peace of mind.

[2299] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2300] Step 1:

[2301] Terminal: The phone starts ringing. It notifies the user of the incoming call and plays the ringtone. The input is the incoming call signal and the output is the ringtone.

[2302] Step 2:

[2303] Terminal: Switch to auto-answer mode after the specified number of rings (e.g. 3 times). Input is the number of rings, output is to switch to auto-answer mode.

[2304] Step 3:

[2305] Terminal: When a call is started, the terminal records the voice data in real time and sends it to the server. The input is the call start signal, and the output is the recorded voice data.

[2306] Step 4:

[2307] Server: Converts received voice data into text using a speech recognition module (speech_recognition library). The input is recorded voice data, and the output is text data. Specifically, the audio file is analyzed and the voice is converted into text.

[2308] Step 5:

[2309] Server: Compares the text data with a vector database to determine the likelihood of a fraudulent call. The input is text data, and the output is the result of the fraudulent call determination. Specifically, it compares the data with fraudulent call patterns registered in the vector database and scores the degree of match.

[2310] Step 6:

[2311] Server: If the call is determined to be fraudulent, an automatic response is generated using generative artificial intelligence (the transformers library). The input is the fraudulent call determination result and text data, and the output is the generated response. Specifically, questions and responses are generated for the fraudster.

[2312] Step 7:

[2313] Server: Sends the generated response to the device and plays it back to the fraudster. The input is the generated response, and the output is the played response. Specifically, the emotion engine analyzes the emotion from the fraudster's voice and optimizes the response based on the results.

[2314] Step 8:

[2315] Server: If the call is determined not to be a scam, the server transfers the call to the elderly person. The input is the result of the scam call determination, and the output is the call transfer.

[2316] Step 9:

[2317] Terminal: The elderly person utters a specific command, such as "Please be my conversation partner." The input is the command voice, and the output is command recognition.

[2318] Step 10:

[2319] Generative AI: Recognizes commands and starts a conversation. The input is a spoken command, and the output is a generated conversation starter. Specifically, it responds, "Hello! What would you like to talk about today?"

[2320] Step 11:

[2321] Terminal and Generative AI: The content of what the user says is analyzed in real time, and the emotion engine analyzes the emotion. The input is the user's voice, and the output is the result of the emotion analysis. Specifically, in response to the statement, "The flowers in the garden have been blooming beautifully recently," the response generated is, "That's lovely. Please tell me what kind of flowers have bloomed."

[2322] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2323] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2324] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2325] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2326] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotio...

Claims

1. A telephone terminal equipped with generative artificial intelligence, a means for determining whether a call is fraudulent; A means for the generative artificial intelligence to automatically respond to the determined fraudulent call; If the call is not a scam, how can we transfer the call to an elderly person? A system that includes a means for generative artificial intelligence to converse with elderly people living alone who want someone to talk to.

2. 2. The system of claim 1, further comprising means for converting voice data to text and comparing said text to fraudulent call patterns to determine the likelihood of a fraudulent call.

3. 10. The system of claim 1, further comprising means for using a vector database to determine fraudulent calls.

Images