System

Abstract

A system is provided.SOLUTION: A fraud prevention system for an elderly person or a information poor, comprising: means for recording and registering a voice of a family member or a relative by a user; means for analyzing a received call or mail by using a generated AI model; and means for displaying a warning notification to the user when the received call or mail is determined to be suspicious based on an analysis result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Special frauds targeting the elderly and those with limited information, particularly "It's me" frauds, have become a major social problem. Fraudsters use sophisticated methods to target the elderly and those with limited information, using telephones and emails, making it difficult for victims to identify frauds on their own. Furthermore, many victims are confident that they will not fall victim to fraud, making them unable to make calm decisions. This calls for a method to detect and respond to fraud without relying on third-party observation. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides a fraud prevention system for the elderly and those with limited information, which includes the following means:

[0006] 1. A means for users to record and register the voices of their family and relatives.

[0007] 2. A means of analyzing incoming call and email data using generative AI models.

[0008] 3. A means of displaying a warning notification to the user if the received call or email is deemed suspicious based on the analysis results.

[0009] Furthermore, it also includes a means to recommend users to contact their family or public institutions, and a means to regularly update the generative AI model, which will enable effective detection and response to fraud, and will enable the elderly and those with limited information to prevent fraud from occurring.

[0010] The term "elderly" generally refers to older people, especially those aged 65 and over.

[0011] "Informationally disadvantaged" refers to people who lack the ability to properly understand and utilize information.

[0012] "Fraud" refers to the criminal act of deceiving others to obtain property or profits illegally.

[0013] "It's me" fraud refers to a fraudulent method in which a person pretends to be a family member or acquaintance and defrauds the victim of money over the phone or other means.

[0014] A "system" refers to a set of means in which multiple elements are combined to achieve a specific function.

[0015] A "generative AI model" refers to an artificial intelligence model that analyzes input data and automatically generates meaningful output.

[0016] "User" refers to a person who uses this system.

[0017] "Audio data" refers to information that records sound in digital form.

[0018] "Warning notification" refers to a message that warns the user of danger or caution.

[0019] "Public institutions" refer to institutions that provide public services, such as city halls and police stations. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, and 5) model update function.

[0042] Voice registration function

[0043] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[0044] Call and email analysis function

[0045] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[0046] Warning notification function

[0047] If the server determines that a received call or email is suspicious, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[0048] Countermeasure recommendation function

[0049] At the same time as receiving the warning, the device will also suggest countermeasures to the user. The message will say, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[0050] Model update function

[0051] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[0052] Specific examples

[0053] Specific examples of calls

[0054] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0055] Specific examples of emails

[0056] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0057] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[0058] The processing flow will be explained below.

[0059] Voice registration function processing steps

[0060] Step 1:

[0061] The user installs and launches the app.

[0062] Step 2:

[0063] The device prompts the user to agree to the terms of use.

[0064] Step 3:

[0065] The user agrees to the terms of use.

[0066] Step 4:

[0067] The device prompts the user to record the voices of family members and relatives.

[0068] Step 5:

[0069] The user records the voices of family members or relatives.

[0070] Step 6:

[0071] The device sends the recorded audio data to the server.

[0072] Step 7:

[0073] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[0074] Call analysis processing steps

[0075] Step 1:

[0076] The device receives the call.

[0077] Step 2:

[0078] The voice data received by the terminal is transmitted to the server in real time.

[0079] Step 3:

[0080] The server analyzes the transmitted voice data using a generative AI model.

[0081] Step 4:

[0082] The server compares the voice characteristics with the voice data of registered family members and relatives.

[0083] Step 5:

[0084] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[0085] Step 6:

[0086] The server transmits the determination result to the terminal.

[0087] Step 7:

[0088] The device displays a warning to the user saying, "This call may be suspicious. Please verify."

[0089] Step 8:

[0090] The user sees the warning notification and chooses whether to end the call or continue.

[0091] Processing steps of email analysis function

[0092] Step 1:

[0093] Your device receives a new email.

[0094] Step 2:

[0095] The terminal sends the contents of the received email to the server.

[0096] Step 3:

[0097] The server analyzes the email content using a generative AI model.

[0098] Step 4:

[0099] The server checks the sender information and content to determine if there is any suspicion of fraud.

[0100] Step 5:

[0101] The server transmits the determination result to the terminal.

[0102] Step 6:

[0103] The device will display a warning to the user saying, "This email may be fraudulent. Please delete it."

[0104] Step 7:

[0105] The user reviews the warning notification and chooses whether to delete the email.

[0106] Steps for recommending measures

[0107] Step 1:

[0108] The device will notify the user of the warning and provide countermeasures at the same time.

[0109] Step 2:

[0110] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[0111] Step 3:

[0112] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[0113] Step 4:

[0114] The device will display appropriate contact information and instructions based on the user's selection.

[0115] Model Update Function Processing Steps

[0116] Step 1:

[0117] The server periodically updates the generative AI model.

[0118] Step 2:

[0119] The server learns about new fraud techniques and updates the model.

[0120] Step 3:

[0121] The server sends the update information to the device.

[0122] Step 4:

[0123] The terminal receives the update information and notifies the user.

[0124] Step 5:

[0125] The user updates the app and installs the latest generative AI model.

[0126] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud.

[0127] Example 1

[0128] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] The elderly and those with limited information are easy targets for fraud, and the damage they cause is serious. To prevent these types of fraud, a system is needed that can properly analyze the content of received calls and emails, quickly warn users if fraud is suspected, and suggest countermeasures. Furthermore, because fraud methods are evolving daily, the accuracy of the system must be kept up to date.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0131] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for presenting countermeasures to the user simultaneously with the warning notification, and a means for regularly updating the generative AI model. This makes it possible to quickly and appropriately determine the risk of fraud and present effective countermeasures to the user. In addition, by constantly updating the generative AI model to keep up with the latest fraudulent techniques, the effectiveness of the system can be maintained.

[0132] "Users" refers to the elderly and information-poor people who use the system.

[0133] "Means for recording and registering voice" refers to the functionality that allows users to record the voices of their family members or relatives and store them in the system's database.

[0134] "Means for analyzing received call and email data using a generative AI model" refers to a function that allows the system to analyze received call and email data using a generative AI model and evaluate its content.

[0135] "Means for displaying a warning notification" refers to a function for displaying a warning message to the user when a suspicious call or email is identified based on the analysis results.

[0136] "Means for suggesting countermeasures" refers to a function for presenting specific countermeasures to the user at the same time as issuing a warning notification.

[0137] "Means for regularly updating the generative AI model" refers to the system's ability to regularly update the generative AI model to respond to new fraudulent methods.

[0138] "Means for notifying users that an updated generative AI model is available" refers to a function for notifying users that an updated generative AI model is available.

[0139] "Means to recommend contacting family members or public institutions" refers to a function that recommends that users contact family members or public institutions when a suspicious call or email is identified.

[0140] The system of this invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: voice registration function, call and email analysis function, warning notification function, countermeasure recommendation function, and model update function.

[0141] Voice registration function

[0142] First, the user installs the system's application on their smartphone and launches the app. Next, they acquire voice data by recording the voices of their family and relatives. The device then sends the recorded voice data to the server. The server then analyzes the received voice data using a generative AI model, extracts voice characteristics, and stores them in a database. This voice registration function makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[0143] Call and email analysis function

[0144] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with pre-registered voices. Similarly, when the device receives an email, the content of that email is also sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud based on the analysis results.

[0145] Warning notification function

[0146] If the server determines that the received call or email is suspicious based on the analysis results, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[0147] Countermeasure recommendation function

[0148] At the same time as the warning notification, the device will also suggest specific countermeasures to the user. For example, a message will be displayed saying, "This call / email may be suspicious. Please take action from the options below." The user will be able to choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[0149] Model update function

[0150] The server periodically updates the generative AI model and learns about new fraud techniques. When the latest generative AI model becomes available, the server sends this information to the device. The device then sends an update notification to the user, displaying the message "The latest generative AI model is available. Please update your app." When the user updates the app, the latest generative AI model is installed.

[0151] Specific examples

[0152] Specific examples of calls

[0153] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0154] Specific examples of emails

[0155] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0156] Prompt Sentence Examples

[0157] 1. "Determine if this audio belongs to a specific family member"

[0158] 2. "Evaluate the likelihood that this email is fraudulent."

[0159] 3. "Analyze whether the caller matches the recorded voice."

[0160] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[0161] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0162] Step 1:

[0163] Installing and launching the app

[0164] A user installs and launches a fraud prevention app on their smartphone. The app is installed from the app store, and upon launch, an initial setup screen appears.

[0165] Input: Smartphone operation.

[0166] Output: The app is installed and launched.

[0167] Step 2:

[0168] Voice recording and transmission

[0169] Users can use the app's "audio registration" function to record the voices of their family and relatives, and also enter their names when recording.

[0170] The device compresses the recorded audio data and sends it to the server using a security protocol such as SSL / TLS.

[0171] Input: Audio data recorded by the user.

[0172] Output: The audio data is sent to the server and stored.

[0173] Step 3:

[0174] Audio analysis and storage

[0175] The server analyzes the received voice data using a generative AI model to extract voice characteristics, specifically, multiple parameters such as voice frequency characteristics, pitch, and tempo.

[0176] The server registers the extracted feature data and voice data in a database.

[0177] Input: Audio data sent from the device.

[0178] Output: The extracted speech features are stored in a database.

[0179] Step 4:

[0180] Receiving and sending calls

[0181] When the device receives a call, it automatically starts the process of recording the audio data in real time and sending it to the server.

[0182] The device encrypts the voice data using SSL / TLS and sends it to the server in real time.

[0183] Input: Audio data from the received call.

[0184] Output: Audio data is sent to the server in real time.

[0185] Step 5:

[0186] Voice analysis and judgment

[0187] The server instantly analyzes the transmitted voice data using a generative AI model, which involves comparing the voiceprint with registered voice data.

[0188] The server uses the analysis results to determine whether there is a possibility of fraud.

[0189] Input: Audio data from the received call.

[0190] Output: Determination result regarding likelihood of fraud.

[0191] Step 6:

[0192] Receiving and sending emails

[0193] When the terminal receives an email, the content data is automatically sent to the server.

[0194] The device converts the text content of the email into a format that is easy to parse and then executes the sending process.

[0195] Input: Content data of received email.

[0196] Output: The email content is sent to the server.

[0197] Step 7:

[0198] Email analysis and judgment

[0199] The server analyzes the content of the received email using a generative AI model, specifically assessing the likelihood of fraud based on the content's grammar, keywords, sender address, etc.

[0200] The server uses the analysis results to determine whether the email is likely to be fraudulent.

[0201] Input: Email content data sent from the terminal.

[0202] Output: Determination result regarding likelihood of fraud.

[0203] Step 8:

[0204] Warning notice

[0205] The server sends the analysis results to the terminal.

[0206] Based on the results of the judgment, the device displays a warning notification to the user saying, "This call / email may be suspicious. Please check."

[0207] Input: Verification result from the server.

[0208] Output: A warning notice to the user.

[0209] Step 9:

[0210] Countermeasures

[0211] Along with the warning notification, the device will also present the user with suggestions for countermeasures, displaying the message "This call / email may be suspicious. Please take action from the options below."

[0212] The user selects countermeasures such as "check with family" or "contact public institutions."

[0213] Input: Verification result and countermeasures from the server.

[0214] Output: Proposal of countermeasures to the user.

[0215] Step 10:

[0216] Model Update

[0217] The server periodically updates the generative AI model to learn about new fraud schemes, and this updating happens in the background.

[0218] The server sends information about the new model to the device.

[0219] The device sends a notification to the user saying, "A newer generative AI model is available. Please update your app."

[0220] Input: Data on new scam schemes.

[0221] Output: Latest generative AI model update and user notification.

[0222] (Application example 1)

[0223] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] It is important to help the elderly and those with limited information to avoid falling victim to fraud. However, frauds perpetrated via telephone and email are becoming increasingly sophisticated, making it difficult for the elderly and those with limited information to accurately assess these threats on their own. Furthermore, if the devices they use are inconvenient in terms of operability and the way information is presented, a fundamental solution will not be reached. Therefore, there is a need for a system that provides visual information, identifies fraud in real time, and suggests appropriate countermeasures.

[0225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0226] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This enables elderly people and those with limited information to receive warnings about suspected fraudulent calls and emails in real time and take appropriate measures promptly.

[0227] "Users" refers to the elderly and information-poor people who use the system.

[0228] "Voice of family and relatives" refers to voice data of the user's family and relatives registered in the system.

[0229] "Means for recording and registering" refers to a mechanism that allows users to record the voices of their family members and relatives and register that voice data in the system.

[0230] "Received call and email data" refers to data including the contents of call audio and emails received on the user's smart device.

[0231] "Generative AI model" refers to a machine learning model that analyzes collected voice and email data to determine suspected fraud.

[0232] "Means of analysis" refers to the mechanism for analyzing received call and email data using a generative AI model.

[0233] "Means for displaying a warning notice" refers to a function that notifies the user of suspected fraud based on the analysis results.

[0234] "Means for displaying a notification on the smart glasses" refers to a mechanism for visually displaying an alert notification to a user using the smart glasses.

[0235] "Means for recommending contacting public institutions" refers to a function that recommends to the user to contact family members or public institutions as necessary.

[0236] "Means for regularly updating the generative AI model" refers to the ability to regularly update the generative AI model to respond to the latest fraud techniques.

[0237] This invention is a fraud prevention system using smart glasses for the elderly and those with limited information. The system includes a means for users to record and register the voices of their family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is deemed suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This allows the elderly and those with limited information to receive warnings in real time about calls or emails that may be fraudulent and take appropriate measures promptly.

[0238] System configuration

[0239] Voice registration function

[0240] When a user first uses the system, they record the voice of their family or relatives using a microphone connected to the smart glasses. The recorded voice data is sent to a server, which analyzes it using a generative AI model to extract and store features that enable speaker recognition during later call analysis.

[0241] Call and email analysis function

[0242] When a user's smart glasses receive a call, the voice data is sent to the server in real time. The server uses a generative AI model to analyze the voice data and compare it with the voice data of pre-registered family members and relatives. Similarly, the content of emails received by the user is sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[0243] Warning notification function

[0244] If the server determines that a received call or email is suspicious, the result of the determination is sent to the smart glasses. The smart glasses then display a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[0245] Countermeasure recommendation function

[0246] At the same time as sending the warning, the smart glasses also suggest countermeasures to the user. The message "This call / email may be suspicious. Please take action from the options below" is displayed, and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This further reduces the risk of fraud.

[0247] Model update function

[0248] The server periodically updates the generative AI model to learn about new fraud techniques. The server sends the updated information to the smart glasses, which then notify the user. When the user updates the app, the latest generative AI model is installed.

[0249] Hardware and software used

[0250] Smart glasses: Requires a microphone for voice input and a display for notifications. Examples include Google Glass and Microsoft HoloLens.

[0251] Server: A high-performance server is required, and analysis processing is performed on the server.

[0252] Python: The main logic of the program uses Python.

[0253] speech_recognition library: Use this library for speech recognition.

[0254] Generative AI model: Uses deep learning frameworks (e.g., TensorFlow, PyTorch) for voice analysis and judgment.

[0255] Specific examples

[0256] When a user receives a call from someone claiming to be their "son," the smart glasses' microphone picks up the call and sends it to the server in real time. The server uses a generative AI model to compare the voice data with the registered "son's" voice, and if it determines there is a mismatch, the smart glasses' display displays a warning saying, "This call may be suspicious. Please check." This allows the user to end the call and check with their actual family member.

[0257] Prompt Sentence Examples

[0258] Design a smart glasses application that analyzes the call in real time when an elderly person receives a fraudulent phone call, and displays a warning message if the voice does not match that of a registered family member. Specifically, please explain the functions of the smart glasses, including voice recognition, real-time analysis, and warning display functions.

[0259] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0260] Step 1:

[0261] User voice recording and registration

[0262] Users use a microphone connected to the smart glasses to record the voices of their family and relatives. The recorded voice data is sent from the device to a server. The server then analyzes the received voice data using a generative AI model, extracts features, and stores them. This provides the reference data needed for subsequent call analysis.

[0263] Input: User-recorded voice data of family and relatives

[0264] Data processing: speech analysis and feature extraction using generative AI models

[0265] Output: Analyzed audio data

[0266] Step 2:

[0267] Real-time analysis of call data

[0268] When the device receives a call, the microphone in the smart glasses picks up the call and the device transmits the audio data in real time to the server. The server then uses a generative AI model to analyze the received audio data and compare it with the voices of registered family members and relatives. Based on the comparison results, it determines whether the voices match.

[0269] Input: Audio data from the received call

[0270] Data processing: Voice analysis and matching with registered voice using a generative AI model

[0271] Output: Matching result (match / mismatch)

[0272] Step 3:

[0273] Email data analysis

[0274] When a device receives an email, it sends the email content to a server. The server uses a generative AI model to analyze the content of the received email and determine whether it is suspected of being fraudulent. Based on the analysis results, the likelihood of fraud is assessed.

[0275] Input: Text data of received email

[0276] Data processing: Text analysis with generative AI models

[0277] Output: Analysis result (suspected of fraud / not suspected)

[0278] Step 4:

[0279] Displaying warning notifications

[0280] Based on the analysis of calls and emails, the server generates a warning notification if fraud is suspected and sends it to the device, which then displays a warning on the smart glasses display saying, "This call / email may be suspicious. Please check."

[0281] Input: Call and email matching or analysis results

[0282] Data processing: Generate warning messages

[0283] Output: Displaying a warning notification

[0284] Step 5:

[0285] Providing recommendations for countermeasures

[0286] The smart glasses display will show a warning message along with suggested actions to take. The message will read, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Ask a family member" or "Contact the nearest public agency." This will allow the user to take appropriate action.

[0287] Input: Warning Notification

[0288] Data manipulation: Providing countermeasure options

[0289] Output: User's selected action

[0290] Step 6:

[0291] Regular updates to generative AI models

[0292] The server periodically updates the generative AI model, learning about new fraud techniques and building the latest model. The server then sends the update information to the device, which then displays a notification on the smart glasses, prompting the user to update the app. When the user updates, the latest generative AI model is installed.

[0293] Input: New dataset, trained generative AI model

[0294] Data processing: Retraining and updating generative AI models

[0295] Output: Install the latest generative AI model

[0296] Through these processing steps, the system helps the elderly and the information-poor to protect themselves from the risk of fraud, and the visual warning notification via the smart glasses allows users to instantly understand the situation and take appropriate measures.

[0297] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0298] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following components: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[0299] Voice registration function

[0300] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[0301] Call and email analysis function

[0302] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[0303] Emotion engine function

[0304] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[0305] Warning notification function

[0306] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[0307] Countermeasure recommendation function

[0308] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[0309] Model update function

[0310] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[0311] Specific examples

[0312] Specific examples of calls

[0313] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of its judgment to the device, and at the same time, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0314] Specific examples of emails

[0315] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0316] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[0317] The processing flow will be explained below.

[0318] Voice registration function processing steps

[0319] Step 1:

[0320] The user installs and launches the app.

[0321] Step 2:

[0322] The device prompts the user to agree to the terms of use.

[0323] Step 3:

[0324] The user agrees to the terms of use.

[0325] Step 4:

[0326] The device prompts the user to record the voices of family members and relatives.

[0327] Step 5:

[0328] The user records the voices of family members or relatives.

[0329] Step 6:

[0330] The device sends the recorded audio data to the server.

[0331] Step 7:

[0332] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[0333] Call analysis processing steps

[0334] Step 1:

[0335] The device receives the call.

[0336] Step 2:

[0337] The terminal transmits the voice data of the received call to the server in real time.

[0338] Step 3:

[0339] The server analyzes the transmitted voice data using a generative AI model.

[0340] Step 4:

[0341] The server compares the voice characteristics with the voice data of registered family members and relatives.

[0342] Step 5:

[0343] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[0344] Step 6:

[0345] The server sends the discrimination result and the user's emotional state to the terminal.

[0346] Step 7:

[0347] The device will display a warning to the user saying, "This call may be suspicious. Please check." The emotion engine will adjust the strength of the warning and the message.

[0348] Step 8:

[0349] The user sees the warning notification and chooses whether to end the call or continue.

[0350] Processing steps of email analysis function

[0351] Step 1:

[0352] Your device receives a new email.

[0353] Step 2:

[0354] The terminal sends the contents of the received email to the server.

[0355] Step 3:

[0356] The server analyzes the email content using a generative AI model.

[0357] Step 4:

[0358] The server checks the sender information and content to determine if there is any suspicion of fraud.

[0359] Step 5:

[0360] The server sends the discrimination result and the user's emotional state to the terminal.

[0361] Step 6:

[0362] The device displays a warning to the user saying, "This email may be fraudulent. Please delete it." The emotion engine adjusts the strength of the warning and the message.

[0363] Step 7:

[0364] The user reviews the warning notification and chooses whether to delete the email.

[0365] Emotion Engine Function Processing Steps

[0366] Step 1:

[0367] The device activates the camera and microphone to analyze the user's facial expressions and voice.

[0368] Step 2:

[0369] The device analyzes the user's facial expressions and tone of voice in real time to determine their emotional state.

[0370] Step 3:

[0371] The device transmits the emotion data to the server.

[0372] Step 4:

[0373] The server analyzes the emotional data and correlates it with calls and emails that appear to be fraudulent.

[0374] Steps for recommending measures

[0375] Step 1:

[0376] The device will notify the user of the warning and provide countermeasures at the same time.

[0377] Step 2:

[0378] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[0379] Step 3:

[0380] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[0381] Step 4:

[0382] The device will display appropriate contact information and procedures based on the user's choices, and an emotion engine will also display messages encouraging users to remain calm based on their emotional state.

[0383] Model Update Function Processing Steps

[0384] Step 1:

[0385] The server periodically updates the generative AI model.

[0386] Step 2:

[0387] The server learns about new fraud techniques and updates the model.

[0388] Step 3:

[0389] The server sends the update information to the device.

[0390] Step 4:

[0391] The terminal receives the update information and notifies the user.

[0392] Step 5:

[0393] The user updates the app and installs the latest generative AI model.

[0394] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud. In particular, the emotion engine function enables appropriate responses according to the user's emotional state, thereby enhancing the effectiveness of fraud prevention.

[0395] Example 2

[0396] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0397] Elderly people and those with limited information are highly vulnerable to fraud, particularly fraud via telephone and email. This often results in financial loss and mental stress. Current fraud prevention systems are not sufficient to protect against this problem, and more effective measures are needed.

[0398] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0399] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, and a means for displaying a warning to the user if the received call or email is deemed suspicious based on the analysis results, thereby enabling the elderly and those with limited information to protect themselves from fraud in real time.

[0400] A "generative AI model" is an artificial intelligence model that analyzes voice and text data and detects patterns based on its features.

[0401] The "voice registration means" is a function that allows the user to record the voices of family members and relatives and transmit the data to the server.

[0402] The "call analysis means" is a function that sends the voice data of the call received by the terminal to a server and analyzes it using a generative AI model.

[0403] The "email analysis means" is a function that sends data of emails received by the terminal to a server and analyzes them using a generative AI model.

[0404] The "warning notification means" is a function that issues a warning to the user if a received call or email is determined to be suspicious based on the analysis results.

[0405] The "emotion engine" is a system for determining a user's emotional state from their facial expressions and tone of voice.

[0406] "Emotion recognition means" is a function that enables the terminal to detect the user's emotions in real time and transmit that data to the server.

[0407] "Update methods" are functions that periodically train the generative AI model to respond to new fraudulent methods.

[0408] "Recommended measures" is a function that suggests appropriate measures to the user when they receive a suspicious call or email.

[0409] The present invention provides a smartphone application for fraud prevention targeted at the elderly and those with limited information access. Specific embodiments of the application are described below.

[0410] System Configuration

[0411] This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[0412] Voice registration function

[0413] When a user installs and launches the app on their smartphone, the system first displays a screen for recording and registering the voices of family members and relatives. The user records the voices of their family members and relatives, and the device sends the recording data to the server. The server then analyzes this voice data using a generative AI model, extracting and saving features. This makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[0414] Call and email analysis function

[0415] When the device receives a call, the voice data is sent to the server in real time. The server then uses a generative AI model to analyze the voice data and compare it with the voices of registered family members and relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[0416] Emotion engine function

[0417] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[0418] Warning notification function

[0419] If the server determines that the content of the call or email is suspicious and that the user's emotional state is unstable, the server sends the result of the analysis to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning is adjusted by the emotion engine depending on the user's emotional state.

[0420] Countermeasure recommendation function

[0421] At the same time as issuing the warning notification, the device will also suggest specific countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below," and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The emotion engine will also display response procedures based on the user's emotional state. For example, if the user is panicking, a message urging them to respond calmly will be displayed.

[0422] Model update function

[0423] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then sends a notification to the user. When the user updates the app, the latest generative AI model is installed, keeping the entire system up to date.

[0424] Specific examples

[0425] Specific examples of calls

[0426] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. When the server sends the results of its assessment to the device, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0427] Specific examples of emails

[0428] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0429] Examples of prompt statements

[0430] "Describe a smartphone app that helps seniors avoid fraud."

[0431] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0432] Step 1:

[0433] The user installs the app on their smartphone and launches it. When the app is launched for the first time, a screen prompting the user to register their voice is displayed. This screen displays instructions such as, "Please record the voices of your family and relatives."

[0434] Step 2:

[0435] The user records the voice of a family member or relative. The user records a voice such as "Hello, I'm Grandpa." This provides voice data (input).

[0436] Step 3:

[0437] The device sends the recorded voice data to the server. The data is sent using a secure protocol (e.g. HTTPS). This action passes the recorded voice data (input) to the server.

[0438] Step 4:

[0439] The server analyzes the received voice data using a generative AI model to extract voice features (data processing and data calculation). For example, it calculates features such as voice tone and speaking rate. The analysis results (output) are used in the next step.

[0440] Step 5:

[0441] The server stores the extracted voice features in a database, which is used for later analysis of calls and emails (data accumulation).

[0442] Step 6:

[0443] When a device receives a call, it sends audio data to the server in real time. At the start of a call, the device captures the audio stream and sends it to the server (input).

[0444] Step 7:

[0445] The server analyzes the received voice data using a generative AI model and compares it with the recorded voices of family members and relatives (data calculation). It checks whether there is a match and passes the result (output) to the next step.

[0446] Step 8:

[0447] The terminal sends the contents of the received email to the server. The email data (input) is transferred to the server.

[0448] Step 9:

[0449] The server analyzes the email data using a generative AI model to determine whether there is any suspicion of fraud (data calculation). If there is a possibility of fraud, the analysis result (output) is passed to the next step.

[0450] Step 10:

[0451] The device recognizes the user's emotions in real time and sends the data to the server. Emotional data (input) is collected through facial recognition and voice tone analysis.

[0452] Step 11:

[0453] The server analyzes the received emotion data and determines the user's emotional state (data calculation). For example, emotions such as anxiety or impatience are detected. The analysis results (output) are used in the next step.

[0454] Step 12:

[0455] The server generates a warning notification based on the results of the call and email analysis and the user's emotional state. If a suspicious call or email is detected and the emotional state is recognized as unstable, a warning is generated (data generation).

[0456] Step 13:

[0457] The server sends a warning notification to the device, which includes the analysis results and a message according to the emotional state (output).

[0458] Step 14:

[0459] The device displays a warning notification to the user. A warning such as "This call / email may be suspicious. Please check it" is displayed. The user considers how to respond based on the displayed warning (output).

[0460] Step 15:

[0461] The server proposes appropriate measures to the user, providing options such as "check with family" or "contact the nearest public institution" (data generation).

[0462] Step 16:

[0463] The server customizes countermeasure procedures based on the user's emotional state. It uses an emotion engine to add messages encouraging users to remain calm (data generation).

[0464] Step 17:

[0465] The server periodically updates the generated AI model. The model learns about new fraud methods and sends updated information to the device (data processing and data calculation).

[0466] Step 18:

[0467] The device notifies the user of update information. When the user updates the app, the latest generative AI model is installed, allowing the entire system to respond to the latest fraudulent techniques (data accumulation).

[0468] (Application example 2)

[0469] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0470] Currently, the elderly and those with limited information are vulnerable to fraud, and it is difficult to prevent such damage. Virtual stores also face the same risk of fraud, making real-time fraud prevention particularly necessary. Another issue is the lack of systems that provide warnings and countermeasures that take into account the user's mental state.

[0471] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for analyzing and monitoring the user's emotional state, and a means for adjusting the content of the notification if the user shows anxiety or impatience. This reduces the risk of the user being scammed in a virtual store and makes it possible to provide appropriate warnings and countermeasures that take into account the user's mental state.

[0472] "Elderly" refers to individuals who are aging, generally aged 65 or older, and who, for social and physical reasons, are more likely to be targeted by fraud.

[0473] "Informationally vulnerable" refers to individuals who have little knowledge or experience regarding the Internet or digital devices and therefore have a low resistance to fraud.

[0474] A "fraud prevention system" is a system that includes technical means and methods to prevent users from becoming victims of fraud.

[0475] "Voice recording and registration means" refers to technology or devices that store the voice of a person designated by the user as digital data and make it available within the system.

[0476] "Means of analyzing call and email data using generative AI models" refers to technologies and methods that use artificial intelligence technology to analyze received voice and text data and understand its context and characteristics.

[0477] "Means for displaying a warning notice" refers to a device or technology for visually or audibly conveying a warning message to a user based on the analysis results.

[0478] "Means for analyzing and monitoring emotional state" refers to technology that analyzes a user's voice, facial expressions, etc. in real time to estimate their emotions and psychological state.

[0479] "Means for adjusting notification content" refers to techniques or methods for changing the intensity of the notification content or the content of the message depending on the emotional state of the user.

[0480] A "virtual store" refers to a shopping environment or platform built on the Internet, rather than a physical store.

[0481] This system is a fraud prevention system aimed at the elderly and those with limited information, and operates primarily on mobile devices such as smartphones and tablets. The system consists of the following main components: voice registration, call and email analysis, emotion engine, warning notification, countermeasure recommendation, and model update. This system can also be applied to fraud prevention in virtual stores.

[0482] Voice registration function

[0483] When a user installs and launches the app, they are first provided with a means to record and register the voices of their family and relatives. The user records the voices of each family member and relative, and the device sends the recordings to the server. The server then analyzes the voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[0484] Call and email analysis function

[0485] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[0486] Emotion engine function

[0487] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[0488] Warning notification function

[0489] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[0490] Countermeasure recommendation function

[0491] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[0492] Model update function

[0493] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[0494] Examples and prompts

[0495] Example

[0496] Example of a call: When a user receives a call from someone claiming to be their "grandchild," the device analyzes the call in real time and sends the voice data to the server. The server uses a generative AI model to analyze the voice data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, and the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0497] Example of email: When a user receives a fraudulent email purporting to be from a different sender, the device sends the received email to a server, which analyzes the email content using a generative AI model. If it determines that the email is suspected to be fraudulent, the server sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message will be displayed saying, "Please remain calm and respond accordingly." The user will then see the warning and delete the email, avoiding opening any suspicious links or attachments.

[0498] Prompt Sentence Examples

[0499] "Analyze the given audio data for known voice features and determine if it matches any registered voices. If a match is not found or the user appears anxious, send a cautionary notification to the user's device."

[0500] "Analyze the given message data to detect any fraudulent content using the AI ​​model. If a fraud is detected, notify the user with a warning message."

[0501] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[0502] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0503] Step 1:

[0504] (Audio recording and registration)

[0505] When a user installs and launches the app, the server provides the user with a screen for recording and registering the voices of their family and relatives.

[0506] Input: User-recorded voice data of family and relatives

[0507] Processing: The device sends the recorded voice data to the server, which uses a generative AI model to extract voice features.

[0508] Output: Family and relatives' characteristics data stored on the server

[0509] How it works: The server analyzes the feature data and stores it in a database as a registered voice profile.

[0510] Step 2:

[0511] (Real-time analysis of call data)

[0512] When the terminal receives a call, the call data is sent to the server in real time.

[0513] Input: Audio data from the received call

[0514] Processing: The server uses the generative AI model to analyze the voice data and match it with the registered voice profile.

[0515] Output: Analysis results (whether or not there is suspicion of fraud)

[0516] Operation: The server performs real-time matching and sends the results to the device.

[0517] Step 3:

[0518] (Email data analysis)

[0519] The terminal sends the contents of the received email to the server.

[0520] Input: Received email data

[0521] Processing: The server uses the generative AI model to analyze the text of the email and determine whether it is likely to be fraudulent.

[0522] Output: Analysis results (whether or not there is suspicion of fraud)

[0523] How it works: Based on the analysis results, the server determines whether the email is likely to be fraudulent and sends the result to the device.

[0524] Step 4:

[0525] (Monitoring emotional state)

[0526] The terminal monitors the user's emotional state.

[0527] Input: User's voice tone and facial expression data

[0528] Processing: The server uses the emotion engine to analyze the user's emotional state.

[0529] Output: Emotional state judgment result

[0530] Operation: The server analyzes the user's emotional state and sends the analysis results to the device.

[0531] Step 5:

[0532] (Display warning notification)

[0533] The server displays a warning notification to the user based on the analysis of calls and emails and the user's emotional state.

[0534] Input: Analysis results of calls and emails, emotional state determination results

[0535] Processing: The server generates the warning content and sends it to the terminal.

[0536] Output: A warning notice that is displayed to the user

[0537] What it does: The device displays a warning notification to the user saying "This call / email may be suspicious."

[0538] Step 6:

[0539] (Recommended measures)

[0540] The device recommends countermeasures to the user.

[0541] Input: Warning notification content

[0542] Processing: The server generates appropriate countermeasures and sends them to the terminal.

[0543] Output: Recommended measures

[0544] What it does: The device prompts the user with options such as "Check with a family member" or "Contact the nearest public agency."

[0545] Step 7:

[0546] (Regular model updates)

[0547] The server periodically updates the generative AI model.

[0548] Input: Latest scam information

[0549] Processing: The server trains the generated AI model on new fraud techniques.

[0550] Output: Updated generative AI model

[0551] How it works: The server sends update information to the device and notifies the user. When the user updates the app, the latest generative AI model is installed.

[0552] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0553] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0554] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0555] [Second embodiment]

[0556] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0557] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0558] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0559] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0560] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0561] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0562] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0563] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0564] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0565] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0566] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0567] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0568] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, and 5) model update function.

[0569] Voice registration function

[0570] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[0571] Call and email analysis function

[0572] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[0573] Warning notification function

[0574] If the server determines that a received call or email is suspicious, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[0575] Countermeasure recommendation function

[0576] At the same time as receiving the warning, the device will also suggest countermeasures to the user. The message will say, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[0577] Model update function

[0578] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[0579] Specific examples

[0580] Specific examples of calls

[0581] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0582] Specific examples of emails

[0583] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0584] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[0585] The processing flow will be explained below.

[0586] Voice registration function processing steps

[0587] Step 1:

[0588] The user installs and launches the app.

[0589] Step 2:

[0590] The device prompts the user to agree to the terms of use.

[0591] Step 3:

[0592] The user agrees to the terms of use.

[0593] Step 4:

[0594] The device prompts the user to record the voices of family members and relatives.

[0595] Step 5:

[0596] The user records the voices of family members or relatives.

[0597] Step 6:

[0598] The device sends the recorded audio data to the server.

[0599] Step 7:

[0600] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[0601] Call analysis processing steps

[0602] Step 1:

[0603] The device receives the call.

[0604] Step 2:

[0605] The voice data received by the terminal is transmitted to the server in real time.

[0606] Step 3:

[0607] The server analyzes the transmitted voice data using a generative AI model.

[0608] Step 4:

[0609] The server compares the voice characteristics with the voice data of registered family members and relatives.

[0610] Step 5:

[0611] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[0612] Step 6:

[0613] The server transmits the determination result to the terminal.

[0614] Step 7:

[0615] The device displays a warning to the user saying, "This call may be suspicious. Please verify."

[0616] Step 8:

[0617] The user sees the warning notification and chooses whether to end the call or continue.

[0618] Processing steps of email analysis function

[0619] Step 1:

[0620] Your device receives a new email.

[0621] Step 2:

[0622] The terminal sends the contents of the received email to the server.

[0623] Step 3:

[0624] The server analyzes the email content using a generative AI model.

[0625] Step 4:

[0626] The server checks the sender information and content to determine if there is any suspicion of fraud.

[0627] Step 5:

[0628] The server transmits the determination result to the terminal.

[0629] Step 6:

[0630] The device will display a warning to the user saying, "This email may be fraudulent. Please delete it."

[0631] Step 7:

[0632] The user reviews the warning notification and chooses whether to delete the email.

[0633] Steps for recommending measures

[0634] Step 1:

[0635] The device will notify the user of the warning and provide countermeasures at the same time.

[0636] Step 2:

[0637] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[0638] Step 3:

[0639] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[0640] Step 4:

[0641] The device will display appropriate contact information and instructions based on the user's selection.

[0642] Model Update Function Processing Steps

[0643] Step 1:

[0644] The server periodically updates the generative AI model.

[0645] Step 2:

[0646] The server learns about new fraud techniques and updates the model.

[0647] Step 3:

[0648] The server sends the update information to the device.

[0649] Step 4:

[0650] The terminal receives the update information and notifies the user.

[0651] Step 5:

[0652] The user updates the app and installs the latest generative AI model.

[0653] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud.

[0654] Example 1

[0655] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] The elderly and those with limited information are easy targets for fraud, and the damage they cause is serious. To prevent these types of fraud, a system is needed that can properly analyze the content of received calls and emails, quickly warn users if fraud is suspected, and suggest countermeasures. Furthermore, because fraud methods are evolving daily, the accuracy of the system must be kept up to date.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0658] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for presenting countermeasures to the user simultaneously with the warning notification, and a means for regularly updating the generative AI model. This makes it possible to quickly and appropriately determine the risk of fraud and present effective countermeasures to the user. In addition, by constantly updating the generative AI model to keep up with the latest fraudulent techniques, the effectiveness of the system can be maintained.

[0659] "Users" refers to the elderly and information-poor people who use the system.

[0660] "Means for recording and registering voice" refers to the functionality that allows users to record the voices of their family members or relatives and store them in the system's database.

[0661] "Means for analyzing received call and email data using a generative AI model" refers to a function that allows the system to analyze received call and email data using a generative AI model and evaluate its content.

[0662] "Means for displaying a warning notification" refers to a function for displaying a warning message to the user when a suspicious call or email is identified based on the analysis results.

[0663] "Means for suggesting countermeasures" refers to a function for presenting specific countermeasures to the user at the same time as issuing a warning notification.

[0664] "Means for regularly updating the generative AI model" refers to the system's ability to regularly update the generative AI model to respond to new fraudulent methods.

[0665] "Means for notifying users that an updated generative AI model is available" refers to a function for notifying users that an updated generative AI model is available.

[0666] "Means to recommend contacting family members or public institutions" refers to a function that recommends that users contact family members or public institutions when a suspicious call or email is identified.

[0667] The system of this invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: voice registration function, call and email analysis function, warning notification function, countermeasure recommendation function, and model update function.

[0668] Voice registration function

[0669] First, the user installs the system's application on their smartphone and launches the app. Next, they acquire voice data by recording the voices of their family and relatives. The device then sends the recorded voice data to the server. The server then analyzes the received voice data using a generative AI model, extracts voice characteristics, and stores them in a database. This voice registration function makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[0670] Call and email analysis function

[0671] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with pre-registered voices. Similarly, when the device receives an email, the content of that email is also sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud based on the analysis results.

[0672] Warning notification function

[0673] If the server determines that the received call or email is suspicious based on the analysis results, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[0674] Countermeasure recommendation function

[0675] At the same time as the warning notification, the device will also suggest specific countermeasures to the user. For example, a message will be displayed saying, "This call / email may be suspicious. Please take action from the options below." The user will be able to choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[0676] Model update function

[0677] The server periodically updates the generative AI model and learns about new fraud techniques. When the latest generative AI model becomes available, the server sends this information to the device. The device then sends an update notification to the user, displaying the message "The latest generative AI model is available. Please update your app." When the user updates the app, the latest generative AI model is installed.

[0678] Specific examples

[0679] Specific examples of calls

[0680] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0681] Specific examples of emails

[0682] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0683] Prompt Sentence Examples

[0684] 1. "Determine if this audio belongs to a specific family member"

[0685] 2. "Evaluate the likelihood that this email is fraudulent."

[0686] 3. "Analyze whether the caller matches the recorded voice."

[0687] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[0688] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0689] Step 1:

[0690] Installing and launching the app

[0691] A user installs and launches a fraud prevention app on their smartphone. The app is installed from the app store, and upon launch, an initial setup screen appears.

[0692] Input: Smartphone operation.

[0693] Output: The app is installed and launched.

[0694] Step 2:

[0695] Voice recording and transmission

[0696] Users can use the app's "audio registration" function to record the voices of their family and relatives, and also enter their names when recording.

[0697] The device compresses the recorded audio data and sends it to the server using a security protocol such as SSL / TLS.

[0698] Input: Audio data recorded by the user.

[0699] Output: The audio data is sent to the server and stored.

[0700] Step 3:

[0701] Audio analysis and storage

[0702] The server analyzes the received voice data using a generative AI model to extract voice characteristics, specifically, multiple parameters such as voice frequency characteristics, pitch, and tempo.

[0703] The server registers the extracted feature data and voice data in a database.

[0704] Input: Audio data sent from the device.

[0705] Output: The extracted speech features are stored in a database.

[0706] Step 4:

[0707] Receiving and sending calls

[0708] When the device receives a call, it automatically starts the process of recording the audio data in real time and sending it to the server.

[0709] The device encrypts the voice data using SSL / TLS and sends it to the server in real time.

[0710] Input: Audio data from the received call.

[0711] Output: Audio data is sent to the server in real time.

[0712] Step 5:

[0713] Voice analysis and judgment

[0714] The server instantly analyzes the transmitted voice data using a generative AI model, which involves comparing the voiceprint with registered voice data.

[0715] The server uses the analysis results to determine whether there is a possibility of fraud.

[0716] Input: Audio data from the received call.

[0717] Output: Determination result regarding likelihood of fraud.

[0718] Step 6:

[0719] Receiving and sending emails

[0720] When the terminal receives an email, the content data is automatically sent to the server.

[0721] The device converts the text content of the email into a format that is easy to parse and then executes the sending process.

[0722] Input: Content data of received email.

[0723] Output: The email content is sent to the server.

[0724] Step 7:

[0725] Email analysis and judgment

[0726] The server analyzes the content of the received email using a generative AI model, specifically assessing the likelihood of fraud based on the content's grammar, keywords, sender address, etc.

[0727] The server uses the analysis results to determine whether the email is likely to be fraudulent.

[0728] Input: Email content data sent from the terminal.

[0729] Output: Determination result regarding likelihood of fraud.

[0730] Step 8:

[0731] Warning notice

[0732] The server sends the analysis results to the terminal.

[0733] Based on the results of the judgment, the device displays a warning notification to the user saying, "This call / email may be suspicious. Please check."

[0734] Input: Verification result from the server.

[0735] Output: A warning notice to the user.

[0736] Step 9:

[0737] Countermeasures

[0738] Along with the warning notification, the device will also present the user with suggestions for countermeasures, displaying the message "This call / email may be suspicious. Please take action from the options below."

[0739] The user selects countermeasures such as "check with family" or "contact public institutions."

[0740] Input: Verification result and countermeasures from the server.

[0741] Output: Proposal of countermeasures to the user.

[0742] Step 10:

[0743] Model Update

[0744] The server periodically updates the generative AI model to learn about new fraud schemes, and this updating happens in the background.

[0745] The server sends information about the new model to the device.

[0746] The device sends a notification to the user saying, "A newer generative AI model is available. Please update your app."

[0747] Input: Data on new scam schemes.

[0748] Output: Latest generative AI model update and user notification.

[0749] (Application example 1)

[0750] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0751] It is important to help the elderly and those with limited information to avoid falling victim to fraud. However, frauds perpetrated via telephone and email are becoming increasingly sophisticated, making it difficult for the elderly and those with limited information to accurately assess these threats on their own. Furthermore, if the devices they use are inconvenient in terms of operability and the way information is presented, a fundamental solution will not be reached. Therefore, there is a need for a system that provides visual information, identifies fraud in real time, and suggests appropriate countermeasures.

[0752] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0753] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This enables elderly people and those with limited information to receive warnings about suspected fraudulent calls and emails in real time and take appropriate measures promptly.

[0754] "Users" refers to the elderly and information-poor people who use the system.

[0755] "Voice of family and relatives" refers to voice data of the user's family and relatives registered in the system.

[0756] "Means for recording and registering" refers to a mechanism that allows users to record the voices of their family members and relatives and register that voice data in the system.

[0757] "Received call and email data" refers to data including the contents of call audio and emails received on the user's smart device.

[0758] "Generative AI model" refers to a machine learning model that analyzes collected voice and email data to determine suspected fraud.

[0759] "Means of analysis" refers to the mechanism for analyzing received call and email data using a generative AI model.

[0760] "Means for displaying a warning notice" refers to a function that notifies the user of suspected fraud based on the analysis results.

[0761] "Means for displaying a notification on the smart glasses" refers to a mechanism for visually displaying an alert notification to a user using the smart glasses.

[0762] "Means for recommending contacting public institutions" refers to a function that recommends to the user to contact family members or public institutions as necessary.

[0763] "Means for regularly updating the generative AI model" refers to the ability to regularly update the generative AI model to respond to the latest fraud techniques.

[0764] This invention is a fraud prevention system using smart glasses for the elderly and those with limited information. The system includes a means for users to record and register the voices of their family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is deemed suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This allows the elderly and those with limited information to receive warnings in real time about calls or emails that may be fraudulent and take appropriate measures promptly.

[0765] System configuration

[0766] Voice registration function

[0767] When a user first uses the system, they record the voice of their family or relatives using a microphone connected to the smart glasses. The recorded voice data is sent to a server, which analyzes it using a generative AI model to extract and store features that enable speaker recognition during later call analysis.

[0768] Call and email analysis function

[0769] When a user's smart glasses receive a call, the voice data is sent to the server in real time. The server uses a generative AI model to analyze the voice data and compare it with the voice data of pre-registered family members and relatives. Similarly, the content of emails received by the user is sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[0770] Warning notification function

[0771] If the server determines that a received call or email is suspicious, the result of the determination is sent to the smart glasses. The smart glasses then display a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[0772] Countermeasure recommendation function

[0773] At the same time as sending the warning, the smart glasses also suggest countermeasures to the user. The message "This call / email may be suspicious. Please take action from the options below" is displayed, and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This further reduces the risk of fraud.

[0774] Model update function

[0775] The server periodically updates the generative AI model to learn about new fraud techniques. The server sends the updated information to the smart glasses, which then notify the user. When the user updates the app, the latest generative AI model is installed.

[0776] Hardware and software used

[0777] Smart glasses: Requires a microphone for voice input and a display for notifications. Examples include Google Glass and Microsoft HoloLens.

[0778] Server: A high-performance server is required, and analysis processing is performed on the server.

[0779] Python: The main logic of the program uses Python.

[0780] speech_recognition library: Use this library for speech recognition.

[0781] Generative AI model: Uses deep learning frameworks (e.g., TensorFlow, PyTorch) for voice analysis and judgment.

[0782] Specific examples

[0783] When a user receives a call from someone claiming to be their "son," the smart glasses' microphone picks up the call and sends it to the server in real time. The server uses a generative AI model to compare the voice data with the registered "son's" voice, and if it determines there is a mismatch, the smart glasses' display displays a warning saying, "This call may be suspicious. Please check." This allows the user to end the call and check with their actual family member.

[0784] Prompt Sentence Examples

[0785] Design a smart glasses application that analyzes the call in real time when an elderly person receives a fraudulent phone call, and displays a warning message if the voice does not match that of a registered family member. Specifically, please explain the functions of the smart glasses, including voice recognition, real-time analysis, and warning display functions.

[0786] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0787] Step 1:

[0788] User voice recording and registration

[0789] Users use a microphone connected to the smart glasses to record the voices of their family and relatives. The recorded voice data is sent from the device to a server. The server then analyzes the received voice data using a generative AI model, extracts features, and stores them. This provides the reference data needed for subsequent call analysis.

[0790] Input: User-recorded voice data of family and relatives

[0791] Data processing: speech analysis and feature extraction using generative AI models

[0792] Output: Analyzed audio data

[0793] Step 2:

[0794] Real-time analysis of call data

[0795] When the device receives a call, the microphone in the smart glasses picks up the call and the device transmits the audio data in real time to the server. The server then uses a generative AI model to analyze the received audio data and compare it with the voices of registered family members and relatives. Based on the comparison results, it determines whether the voices match.

[0796] Input: Audio data from the received call

[0797] Data processing: Voice analysis and matching with registered voice using a generative AI model

[0798] Output: Matching result (match / mismatch)

[0799] Step 3:

[0800] Email data analysis

[0801] When a device receives an email, it sends the email content to a server. The server uses a generative AI model to analyze the content of the received email and determine whether it is suspected of being fraudulent. Based on the analysis results, the likelihood of fraud is assessed.

[0802] Input: Text data of received email

[0803] Data processing: Text analysis with generative AI models

[0804] Output: Analysis result (suspected of fraud / not suspected)

[0805] Step 4:

[0806] Displaying warning notifications

[0807] Based on the analysis of calls and emails, the server generates a warning notification if fraud is suspected and sends it to the device, which then displays a warning on the smart glasses display saying, "This call / email may be suspicious. Please check."

[0808] Input: Call and email matching or analysis results

[0809] Data processing: Generate warning messages

[0810] Output: Displaying a warning notification

[0811] Step 5:

[0812] Providing recommendations for countermeasures

[0813] The smart glasses display will show a warning message along with suggested actions to take. The message will read, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Ask a family member" or "Contact the nearest public agency." This will allow the user to take appropriate action.

[0814] Input: Warning Notification

[0815] Data manipulation: Providing countermeasure options

[0816] Output: User's selected action

[0817] Step 6:

[0818] Regular updates to generative AI models

[0819] The server periodically updates the generative AI model, learning about new fraud techniques and building the latest model. The server then sends the update information to the device, which then displays a notification on the smart glasses, prompting the user to update the app. When the user updates, the latest generative AI model is installed.

[0820] Input: New dataset, trained generative AI model

[0821] Data processing: Retraining and updating generative AI models

[0822] Output: Install the latest generative AI model

[0823] Through these processing steps, the system helps the elderly and the information-poor to protect themselves from the risk of fraud, and the visual warning notification via the smart glasses allows users to instantly understand the situation and take appropriate measures.

[0824] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0825] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following components: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[0826] Voice registration function

[0827] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[0828] Call and email analysis function

[0829] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[0830] Emotion engine function

[0831] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[0832] Warning notification function

[0833] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[0834] Countermeasure recommendation function

[0835] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[0836] Model update function

[0837] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[0838] Specific examples

[0839] Specific examples of calls

[0840] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of its judgment to the device, and at the same time, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0841] Specific examples of emails

[0842] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0843] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[0844] The processing flow will be explained below.

[0845] Voice registration function processing steps

[0846] Step 1:

[0847] The user installs and launches the app.

[0848] Step 2:

[0849] The device prompts the user to agree to the terms of use.

[0850] Step 3:

[0851] The user agrees to the terms of use.

[0852] Step 4:

[0853] The device prompts the user to record the voices of family members and relatives.

[0854] Step 5:

[0855] The user records the voices of family members or relatives.

[0856] Step 6:

[0857] The device sends the recorded audio data to the server.

[0858] Step 7:

[0859] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[0860] Call analysis processing steps

[0861] Step 1:

[0862] The device receives the call.

[0863] Step 2:

[0864] The terminal transmits the voice data of the received call to the server in real time.

[0865] Step 3:

[0866] The server analyzes the transmitted voice data using a generative AI model.

[0867] Step 4:

[0868] The server compares the voice characteristics with the voice data of registered family members and relatives.

[0869] Step 5:

[0870] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[0871] Step 6:

[0872] The server sends the discrimination result and the user's emotional state to the terminal.

[0873] Step 7:

[0874] The device will display a warning to the user saying, "This call may be suspicious. Please check." The emotion engine will adjust the strength of the warning and the message.

[0875] Step 8:

[0876] The user sees the warning notification and chooses whether to end the call or continue.

[0877] Processing steps of email analysis function

[0878] Step 1:

[0879] Your device receives a new email.

[0880] Step 2:

[0881] The terminal sends the contents of the received email to the server.

[0882] Step 3:

[0883] The server analyzes the email content using a generative AI model.

[0884] Step 4:

[0885] The server checks the sender information and content to determine if there is any suspicion of fraud.

[0886] Step 5:

[0887] The server sends the discrimination result and the user's emotional state to the terminal.

[0888] Step 6:

[0889] The device displays a warning to the user saying, "This email may be fraudulent. Please delete it." The emotion engine adjusts the strength of the warning and the message.

[0890] Step 7:

[0891] The user reviews the warning notification and chooses whether to delete the email.

[0892] Emotion Engine Function Processing Steps

[0893] Step 1:

[0894] The device activates the camera and microphone to analyze the user's facial expressions and voice.

[0895] Step 2:

[0896] The device analyzes the user's facial expressions and tone of voice in real time to determine their emotional state.

[0897] Step 3:

[0898] The device transmits the emotion data to the server.

[0899] Step 4:

[0900] The server analyzes the emotional data and correlates it with calls and emails that appear to be fraudulent.

[0901] Steps for recommending measures

[0902] Step 1:

[0903] The device will notify the user of the warning and provide countermeasures at the same time.

[0904] Step 2:

[0905] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[0906] Step 3:

[0907] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[0908] Step 4:

[0909] The device will display appropriate contact information and procedures based on the user's choices, and an emotion engine will also display messages encouraging users to remain calm based on their emotional state.

[0910] Model Update Function Processing Steps

[0911] Step 1:

[0912] The server periodically updates the generative AI model.

[0913] Step 2:

[0914] The server learns about new fraud techniques and updates the model.

[0915] Step 3:

[0916] The server sends the update information to the device.

[0917] Step 4:

[0918] The terminal receives the update information and notifies the user.

[0919] Step 5:

[0920] The user updates the app and installs the latest generative AI model.

[0921] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud. In particular, the emotion engine function enables appropriate responses according to the user's emotional state, thereby enhancing the effectiveness of fraud prevention.

[0922] Example 2

[0923] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0924] Elderly people and those with limited information are highly vulnerable to fraud, particularly fraud via telephone and email. This often results in financial loss and mental stress. Current fraud prevention systems are not sufficient to protect against this problem, and more effective measures are needed.

[0925] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0926] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, and a means for displaying a warning to the user if the received call or email is deemed suspicious based on the analysis results, thereby enabling the elderly and those with limited information to protect themselves from fraud in real time.

[0927] A "generative AI model" is an artificial intelligence model that analyzes voice and text data and detects patterns based on its features.

[0928] The "voice registration means" is a function that allows the user to record the voices of family members and relatives and transmit the data to the server.

[0929] The "call analysis means" is a function that sends the voice data of the call received by the terminal to a server and analyzes it using a generative AI model.

[0930] The "email analysis means" is a function that sends data of emails received by the terminal to a server and analyzes them using a generative AI model.

[0931] The "warning notification means" is a function that issues a warning to the user if a received call or email is determined to be suspicious based on the analysis results.

[0932] The "emotion engine" is a system for determining a user's emotional state from their facial expressions and tone of voice.

[0933] "Emotion recognition means" is a function that enables the terminal to detect the user's emotions in real time and transmit that data to the server.

[0934] "Update methods" are functions that periodically train the generative AI model to respond to new fraudulent methods.

[0935] "Recommended measures" is a function that suggests appropriate measures to the user when they receive a suspicious call or email.

[0936] The present invention provides a smartphone application for fraud prevention targeted at the elderly and those with limited information access. Specific embodiments of the application are described below.

[0937] System Configuration

[0938] This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[0939] Voice registration function

[0940] When a user installs and launches the app on their smartphone, the system first displays a screen for recording and registering the voices of family members and relatives. The user records the voices of their family members and relatives, and the device sends the recording data to the server. The server then analyzes this voice data using a generative AI model, extracting and saving features. This makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[0941] Call and email analysis function

[0942] When the device receives a call, the voice data is sent to the server in real time. The server then uses a generative AI model to analyze the voice data and compare it with the voices of registered family members and relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[0943] Emotion engine function

[0944] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[0945] Warning notification function

[0946] If the server determines that the content of the call or email is suspicious and that the user's emotional state is unstable, the server sends the result of the analysis to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning is adjusted by the emotion engine depending on the user's emotional state.

[0947] Countermeasure recommendation function

[0948] At the same time as issuing the warning notification, the device will also suggest specific countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below," and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The emotion engine will also display response procedures based on the user's emotional state. For example, if the user is panicking, a message urging them to respond calmly will be displayed.

[0949] Model update function

[0950] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then sends a notification to the user. When the user updates the app, the latest generative AI model is installed, keeping the entire system up to date.

[0951] Specific examples

[0952] Specific examples of calls

[0953] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. When the server sends the results of its assessment to the device, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[0954] Specific examples of emails

[0955] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[0956] Examples of prompt statements

[0957] "Describe a smartphone app that helps seniors avoid fraud."

[0958] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0959] Step 1:

[0960] The user installs the app on their smartphone and launches it. When the app is launched for the first time, a screen prompting the user to register their voice is displayed. This screen displays instructions such as, "Please record the voices of your family and relatives."

[0961] Step 2:

[0962] The user records the voice of a family member or relative. The user records a voice such as "Hello, I'm Grandpa." This provides voice data (input).

[0963] Step 3:

[0964] The device sends the recorded voice data to the server. The data is sent using a secure protocol (e.g. HTTPS). This action passes the recorded voice data (input) to the server.

[0965] Step 4:

[0966] The server analyzes the received voice data using a generative AI model to extract voice features (data processing and data calculation). For example, it calculates features such as voice tone and speaking rate. The analysis results (output) are used in the next step.

[0967] Step 5:

[0968] The server stores the extracted voice features in a database, which is used for later analysis of calls and emails (data accumulation).

[0969] Step 6:

[0970] When a device receives a call, it sends audio data to the server in real time. At the start of a call, the device captures the audio stream and sends it to the server (input).

[0971] Step 7:

[0972] The server analyzes the received voice data using a generative AI model and compares it with the recorded voices of family members and relatives (data calculation). It checks whether there is a match and passes the result (output) to the next step.

[0973] Step 8:

[0974] The terminal sends the contents of the received email to the server. The email data (input) is transferred to the server.

[0975] Step 9:

[0976] The server analyzes the email data using a generative AI model to determine whether there is any suspicion of fraud (data calculation). If there is a possibility of fraud, the analysis result (output) is passed to the next step.

[0977] Step 10:

[0978] The device recognizes the user's emotions in real time and sends the data to the server. Emotional data (input) is collected through facial recognition and voice tone analysis.

[0979] Step 11:

[0980] The server analyzes the received emotion data and determines the user's emotional state (data calculation). For example, emotions such as anxiety or impatience are detected. The analysis results (output) are used in the next step.

[0981] Step 12:

[0982] The server generates a warning notification based on the results of the call and email analysis and the user's emotional state. If a suspicious call or email is detected and the emotional state is recognized as unstable, a warning is generated (data generation).

[0983] Step 13:

[0984] The server sends a warning notification to the device, which includes the analysis results and a message according to the emotional state (output).

[0985] Step 14:

[0986] The device displays a warning notification to the user. A warning such as "This call / email may be suspicious. Please check it" is displayed. The user considers how to respond based on the displayed warning (output).

[0987] Step 15:

[0988] The server proposes appropriate measures to the user, providing options such as "check with family" or "contact the nearest public institution" (data generation).

[0989] Step 16:

[0990] The server customizes countermeasure procedures based on the user's emotional state. It uses an emotion engine to add messages encouraging users to remain calm (data generation).

[0991] Step 17:

[0992] The server periodically updates the generated AI model. The model learns about new fraud methods and sends updated information to the device (data processing and data calculation).

[0993] Step 18:

[0994] The device notifies the user of update information. When the user updates the app, the latest generative AI model is installed, allowing the entire system to respond to the latest fraudulent techniques (data accumulation).

[0995] (Application example 2)

[0996] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0997] Currently, the elderly and those with limited information are vulnerable to fraud, and it is difficult to prevent such damage. Virtual stores also face the same risk of fraud, making real-time fraud prevention particularly necessary. Another issue is the lack of systems that provide warnings and countermeasures that take into account the user's mental state.

[0998] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for analyzing and monitoring the user's emotional state, and a means for adjusting the content of the notification if the user shows anxiety or impatience. This reduces the risk of the user being scammed in a virtual store and makes it possible to provide appropriate warnings and countermeasures that take into account the user's mental state.

[0999] "Elderly" refers to individuals who are aging, generally aged 65 or older, and who, for social and physical reasons, are more likely to be targeted by fraud.

[1000] "Informationally vulnerable" refers to individuals who have little knowledge or experience regarding the Internet or digital devices and therefore have a low resistance to fraud.

[1001] A "fraud prevention system" is a system that includes technical means and methods to prevent users from becoming victims of fraud.

[1002] "Voice recording and registration means" refers to technology or devices that store the voice of a person designated by the user as digital data and make it available within the system.

[1003] "Means of analyzing call and email data using generative AI models" refers to technologies and methods that use artificial intelligence technology to analyze received voice and text data and understand its context and characteristics.

[1004] "Means for displaying a warning notice" refers to a device or technology for visually or audibly conveying a warning message to a user based on the analysis results.

[1005] "Means for analyzing and monitoring emotional state" refers to technology that analyzes a user's voice, facial expressions, etc. in real time to estimate their emotions and psychological state.

[1006] "Means for adjusting notification content" refers to techniques or methods for changing the intensity of the notification content or the content of the message depending on the emotional state of the user.

[1007] A "virtual store" refers to a shopping environment or platform built on the Internet, rather than a physical store.

[1008] This system is a fraud prevention system aimed at the elderly and those with limited information, and operates primarily on mobile devices such as smartphones and tablets. The system consists of the following main components: voice registration, call and email analysis, emotion engine, warning notification, countermeasure recommendation, and model update. This system can also be applied to fraud prevention in virtual stores.

[1009] Voice registration function

[1010] When a user installs and launches the app, they are first provided with a means to record and register the voices of their family and relatives. The user records the voices of each family member and relative, and the device sends the recordings to the server. The server then analyzes the voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[1011] Call and email analysis function

[1012] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[1013] Emotion engine function

[1014] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[1015] Warning notification function

[1016] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[1017] Countermeasure recommendation function

[1018] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[1019] Model update function

[1020] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[1021] Examples and prompts

[1022] Example

[1023] Example of a call: When a user receives a call from someone claiming to be their "grandchild," the device analyzes the call in real time and sends the voice data to the server. The server uses a generative AI model to analyze the voice data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, and the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1024] Example of email: When a user receives a fraudulent email purporting to be from a different sender, the device sends the received email to a server, which analyzes the email content using a generative AI model. If it determines that the email is suspected to be fraudulent, the server sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message will be displayed saying, "Please remain calm and respond accordingly." The user will then see the warning and delete the email, avoiding opening any suspicious links or attachments.

[1025] Prompt Sentence Examples

[1026] "Analyze the given audio data for known voice features and determine if it matches any registered voices. If a match is not found or the user appears anxious, send a cautionary notification to the user's device."

[1027] "Analyze the given message data to detect any fraudulent content using the AI ​​model. If a fraud is detected, notify the user with a warning message."

[1028] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[1029] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1030] Step 1:

[1031] (Audio recording and registration)

[1032] When a user installs and launches the app, the server provides the user with a screen for recording and registering the voices of their family and relatives.

[1033] Input: User-recorded voice data of family and relatives

[1034] Processing: The device sends the recorded voice data to the server, which uses a generative AI model to extract voice features.

[1035] Output: Family and relatives' characteristics data stored on the server

[1036] How it works: The server analyzes the feature data and stores it in a database as a registered voice profile.

[1037] Step 2:

[1038] (Real-time analysis of call data)

[1039] When the terminal receives a call, the call data is sent to the server in real time.

[1040] Input: Audio data from the received call

[1041] Processing: The server uses the generative AI model to analyze the voice data and match it with the registered voice profile.

[1042] Output: Analysis results (whether or not there is suspicion of fraud)

[1043] Operation: The server performs real-time matching and sends the results to the device.

[1044] Step 3:

[1045] (Email data analysis)

[1046] The terminal sends the contents of the received email to the server.

[1047] Input: Received email data

[1048] Processing: The server uses the generative AI model to analyze the text of the email and determine whether it is likely to be fraudulent.

[1049] Output: Analysis results (whether or not there is suspicion of fraud)

[1050] How it works: Based on the analysis results, the server determines whether the email is likely to be fraudulent and sends the result to the device.

[1051] Step 4:

[1052] (Monitoring emotional state)

[1053] The terminal monitors the user's emotional state.

[1054] Input: User's voice tone and facial expression data

[1055] Processing: The server uses the emotion engine to analyze the user's emotional state.

[1056] Output: Emotional state judgment result

[1057] Operation: The server analyzes the user's emotional state and sends the analysis results to the device.

[1058] Step 5:

[1059] (Display warning notification)

[1060] The server displays a warning notification to the user based on the analysis of calls and emails and the user's emotional state.

[1061] Input: Analysis results of calls and emails, emotional state determination results

[1062] Processing: The server generates the warning content and sends it to the terminal.

[1063] Output: A warning notice that is displayed to the user

[1064] What it does: The device displays a warning notification to the user saying "This call / email may be suspicious."

[1065] Step 6:

[1066] (Recommended measures)

[1067] The device recommends countermeasures to the user.

[1068] Input: Warning notification content

[1069] Processing: The server generates appropriate countermeasures and sends them to the terminal.

[1070] Output: Recommended measures

[1071] What it does: The device prompts the user with options such as "Check with a family member" or "Contact the nearest public agency."

[1072] Step 7:

[1073] (Regular model updates)

[1074] The server periodically updates the generative AI model.

[1075] Input: Latest scam information

[1076] Processing: The server trains the generated AI model on new fraud techniques.

[1077] Output: Updated generative AI model

[1078] How it works: The server sends update information to the device and notifies the user. When the user updates the app, the latest generative AI model is installed.

[1079] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1080] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1081] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1082] [Third embodiment]

[1083] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1084] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1085] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1086] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1087] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1088] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1089] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1090] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1091] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1092] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1093] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1094] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1095] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, and 5) model update function.

[1096] Voice registration function

[1097] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[1098] Call and email analysis function

[1099] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[1100] Warning notification function

[1101] If the server determines that a received call or email is suspicious, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[1102] Countermeasure recommendation function

[1103] At the same time as receiving the warning, the device will also suggest countermeasures to the user. The message will say, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[1104] Model update function

[1105] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[1106] Specific examples

[1107] Specific examples of calls

[1108] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1109] Specific examples of emails

[1110] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[1111] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[1112] The processing flow will be explained below.

[1113] Voice registration function processing steps

[1114] Step 1:

[1115] The user installs and launches the app.

[1116] Step 2:

[1117] The device prompts the user to agree to the terms of use.

[1118] Step 3:

[1119] The user agrees to the terms of use.

[1120] Step 4:

[1121] The device prompts the user to record the voices of family members and relatives.

[1122] Step 5:

[1123] The user records the voices of family members or relatives.

[1124] Step 6:

[1125] The device sends the recorded audio data to the server.

[1126] Step 7:

[1127] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[1128] Processing steps of the call analysis function

[1129] Step 1:

[1130] The device receives the call.

[1131] Step 2:

[1132] The voice data received by the terminal is transmitted to the server in real time.

[1133] Step 3:

[1134] The server analyzes the transmitted voice data using a generative AI model.

[1135] Step 4:

[1136] The server compares the voice characteristics with the voice data of registered family members and relatives.

[1137] Step 5:

[1138] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[1139] Step 6:

[1140] The server transmits the determination result to the terminal.

[1141] Step 7:

[1142] The device displays a warning to the user saying, "This call may be suspicious. Please verify."

[1143] Step 8:

[1144] The user sees the warning notification and chooses whether to end the call or continue.

[1145] Processing steps of email analysis function

[1146] Step 1:

[1147] Your device receives a new email.

[1148] Step 2:

[1149] The terminal sends the contents of the received email to the server.

[1150] Step 3:

[1151] The server analyzes the email content using a generative AI model.

[1152] Step 4:

[1153] The server checks the sender information and content to determine if there is any suspicion of fraud.

[1154] Step 5:

[1155] The server transmits the determination result to the terminal.

[1156] Step 6:

[1157] The device will display a warning to the user saying, "This email may be fraudulent. Please delete it."

[1158] Step 7:

[1159] The user reviews the warning notification and chooses whether to delete the email.

[1160] Steps for recommending measures

[1161] Step 1:

[1162] The device will notify the user of the warning and provide countermeasures at the same time.

[1163] Step 2:

[1164] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[1165] Step 3:

[1166] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[1167] Step 4:

[1168] The device will display appropriate contact information and instructions based on the user's selection.

[1169] Model Update Function Processing Steps

[1170] Step 1:

[1171] The server periodically updates the generative AI model.

[1172] Step 2:

[1173] The server learns about new fraud techniques and updates the model.

[1174] Step 3:

[1175] The server sends the update information to the device.

[1176] Step 4:

[1177] The terminal receives the update information and notifies the user.

[1178] Step 5:

[1179] The user updates the app and installs the latest generative AI model.

[1180] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud.

[1181] Example 1

[1182] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1183] The elderly and those with limited information are easy targets for fraud, and the damage they cause is serious. To prevent these types of fraud, a system is needed that can properly analyze the content of received calls and emails, quickly warn users if fraud is suspected, and suggest countermeasures. Furthermore, because fraud methods are evolving daily, the accuracy of the system must be kept up to date.

[1184] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1185] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for presenting countermeasures to the user simultaneously with the warning notification, and a means for regularly updating the generative AI model. This makes it possible to quickly and appropriately determine the risk of fraud and present effective countermeasures to the user. In addition, by constantly updating the generative AI model to keep up with the latest fraudulent techniques, the effectiveness of the system can be maintained.

[1186] "Users" refers to the elderly and information-poor people who use the system.

[1187] "Means for recording and registering voice" refers to the functionality that allows users to record the voices of their family members or relatives and store them in the system's database.

[1188] "Means for analyzing received call and email data using a generative AI model" refers to a function that allows the system to analyze received call and email data using a generative AI model and evaluate its content.

[1189] "Means for displaying a warning notification" refers to a function for displaying a warning message to the user when a suspicious call or email is identified based on the analysis results.

[1190] "Means for suggesting countermeasures" refers to a function for presenting specific countermeasures to the user at the same time as issuing a warning notification.

[1191] "Means for regularly updating the generative AI model" refers to the system's ability to regularly update the generative AI model to respond to new fraudulent methods.

[1192] "Means for notifying users that an updated generative AI model is available" refers to a function for notifying users that an updated generative AI model is available.

[1193] "Means to recommend contacting family members or public institutions" refers to a function that recommends that users contact family members or public institutions when a suspicious call or email is identified.

[1194] The system of this invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: voice registration function, call and email analysis function, warning notification function, countermeasure recommendation function, and model update function.

[1195] Voice registration function

[1196] First, the user installs the system's application on their smartphone and launches the app. Next, they acquire voice data by recording the voices of their family and relatives. The device then sends the recorded voice data to the server. The server then analyzes the received voice data using a generative AI model, extracts voice characteristics, and stores them in a database. This voice registration function makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[1197] Call and email analysis function

[1198] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with pre-registered voices. Similarly, when the device receives an email, the content of that email is also sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud based on the analysis results.

[1199] Warning notification function

[1200] If the server determines that the received call or email is suspicious based on the analysis results, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[1201] Countermeasure recommendation function

[1202] At the same time as the warning notification, the device will also suggest specific countermeasures to the user. For example, a message will be displayed saying, "This call / email may be suspicious. Please take action from the options below." The user will be able to choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[1203] Model update function

[1204] The server periodically updates the generative AI model and learns about new fraud techniques. When the latest generative AI model becomes available, the server sends this information to the device. The device then sends an update notification to the user, displaying the message "The latest generative AI model is available. Please update your app." When the user updates the app, the latest generative AI model is installed.

[1205] Specific examples

[1206] Specific examples of calls

[1207] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1208] Specific examples of emails

[1209] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[1210] Prompt Sentence Examples

[1211] 1. "Determine if this audio belongs to a specific family member"

[1212] 2. "Evaluate the likelihood that this email is fraudulent."

[1213] 3. "Analyze whether the caller matches the recorded voice."

[1214] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[1215] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1216] Step 1:

[1217] Installing and launching the app

[1218] A user installs and launches a fraud prevention app on their smartphone. The app is installed from the app store, and upon launch, an initial setup screen appears.

[1219] Input: Smartphone operation.

[1220] Output: The app is installed and launched.

[1221] Step 2:

[1222] Voice recording and transmission

[1223] Users can use the app's "audio registration" function to record the voices of their family and relatives, and also enter their names when recording.

[1224] The device compresses the recorded audio data and sends it to the server using a security protocol such as SSL / TLS.

[1225] Input: Audio data recorded by the user.

[1226] Output: The audio data is sent to the server and stored.

[1227] Step 3:

[1228] Audio analysis and storage

[1229] The server analyzes the received voice data using a generative AI model to extract voice characteristics, specifically, multiple parameters such as voice frequency characteristics, pitch, and tempo.

[1230] The server registers the extracted feature data and voice data in a database.

[1231] Input: Audio data sent from the device.

[1232] Output: The extracted speech features are stored in a database.

[1233] Step 4:

[1234] Receiving and sending calls

[1235] When the device receives a call, it automatically starts the process of recording the audio data in real time and sending it to the server.

[1236] The device encrypts the voice data using SSL / TLS and sends it to the server in real time.

[1237] Input: Audio data from the received call.

[1238] Output: Audio data is sent to the server in real time.

[1239] Step 5:

[1240] Voice analysis and judgment

[1241] The server instantly analyzes the transmitted voice data using a generative AI model, which involves comparing the voiceprint with registered voice data.

[1242] The server uses the analysis results to determine whether there is a possibility of fraud.

[1243] Input: Audio data from the received call.

[1244] Output: Determination result regarding likelihood of fraud.

[1245] Step 6:

[1246] Receiving and sending emails

[1247] When the terminal receives an email, the content data is automatically sent to the server.

[1248] The device converts the text content of the email into a format that is easy to parse and then executes the sending process.

[1249] Input: Content data of received email.

[1250] Output: The email content is sent to the server.

[1251] Step 7:

[1252] Email analysis and judgment

[1253] The server analyzes the content of the received email using a generative AI model, specifically assessing the likelihood of fraud based on the content's grammar, keywords, sender address, etc.

[1254] The server uses the analysis results to determine whether the email is likely to be fraudulent.

[1255] Input: Email content data sent from the terminal.

[1256] Output: Determination result regarding likelihood of fraud.

[1257] Step 8:

[1258] Warning notice

[1259] The server sends the analysis results to the terminal.

[1260] Based on the results of the judgment, the device displays a warning notification to the user saying, "This call / email may be suspicious. Please check."

[1261] Input: Verification result from the server.

[1262] Output: A warning notice to the user.

[1263] Step 9:

[1264] Countermeasures

[1265] Along with the warning notification, the device will also present the user with suggestions for countermeasures, displaying the message "This call / email may be suspicious. Please take action from the options below."

[1266] The user selects countermeasures such as "check with family" or "contact public institutions."

[1267] Input: Verification result and countermeasures from the server.

[1268] Output: Proposal of countermeasures to the user.

[1269] Step 10:

[1270] Model Update

[1271] The server periodically updates the generative AI model to learn about new fraud schemes, and this updating happens in the background.

[1272] The server sends information about the new model to the device.

[1273] The device sends a notification to the user saying, "A newer generative AI model is available. Please update your app."

[1274] Input: Data on new scam schemes.

[1275] Output: Latest generative AI model update and user notification.

[1276] (Application example 1)

[1277] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1278] It is important to help the elderly and those with limited information to avoid falling victim to fraud. However, frauds perpetrated via telephone and email are becoming increasingly sophisticated, making it difficult for the elderly and those with limited information to accurately assess these threats on their own. Furthermore, if the devices they use are inconvenient in terms of operability and the way information is presented, a fundamental solution will not be reached. Therefore, there is a need for a system that provides visual information, identifies fraud in real time, and suggests appropriate countermeasures.

[1279] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1280] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This enables elderly people and those with limited information to receive warnings about suspected fraudulent calls and emails in real time and take appropriate measures promptly.

[1281] "Users" refers to the elderly and information-poor people who use the system.

[1282] "Voice of family and relatives" refers to voice data of the user's family and relatives registered in the system.

[1283] "Means for recording and registering" refers to a mechanism that allows users to record the voices of their family members and relatives and register that voice data in the system.

[1284] "Received call and email data" refers to data including the contents of call audio and emails received on the user's smart device.

[1285] "Generative AI model" refers to a machine learning model that analyzes collected voice and email data to determine suspected fraud.

[1286] "Means of analysis" refers to the mechanism for analyzing received call and email data using a generative AI model.

[1287] "Means for displaying a warning notice" refers to a function that notifies the user of suspected fraud based on the analysis results.

[1288] "Means for displaying a notification on the smart glasses" refers to a mechanism for visually displaying an alert notification to a user using the smart glasses.

[1289] "Means for recommending contacting public institutions" refers to a function that recommends to the user to contact family members or public institutions as necessary.

[1290] "Means for regularly updating the generative AI model" refers to the ability to regularly update the generative AI model to respond to the latest fraud techniques.

[1291] This invention is a fraud prevention system using smart glasses for the elderly and those with limited information. The system includes a means for users to record and register the voices of their family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is deemed suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This allows the elderly and those with limited information to receive warnings in real time about calls or emails that may be fraudulent and take appropriate measures promptly.

[1292] System configuration

[1293] Voice registration function

[1294] When a user first uses the system, they record the voice of their family or relatives using a microphone connected to the smart glasses. The recorded voice data is sent to a server, which analyzes it using a generative AI model to extract and store features that enable speaker recognition during later call analysis.

[1295] Call and email analysis function

[1296] When a user's smart glasses receive a call, the voice data is sent to the server in real time. The server uses a generative AI model to analyze the voice data and compare it with the voice data of pre-registered family members and relatives. Similarly, the content of emails received by the user is sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[1297] Warning notification function

[1298] If the server determines that a received call or email is suspicious, the result of the determination is sent to the smart glasses. The smart glasses then display a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[1299] Countermeasure recommendation function

[1300] At the same time as sending the warning, the smart glasses also suggest countermeasures to the user. The message "This call / email may be suspicious. Please take action from the options below" is displayed, and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This further reduces the risk of fraud.

[1301] Model update function

[1302] The server periodically updates the generative AI model to learn about new fraud techniques. The server sends the updated information to the smart glasses, which then notify the user. When the user updates the app, the latest generative AI model is installed.

[1303] Hardware and software used

[1304] Smart glasses: Requires a microphone for voice input and a display for notifications. Examples include Google Glass and Microsoft HoloLens.

[1305] Server: A high-performance server is required, and analysis processing is performed on the server.

[1306] Python: The main logic of the program uses Python.

[1307] speech_recognition library: Use this library for speech recognition.

[1308] Generative AI model: Uses deep learning frameworks (e.g., TensorFlow, PyTorch) for voice analysis and judgment.

[1309] Specific examples

[1310] When a user receives a call from someone claiming to be their "son," the smart glasses' microphone picks up the call and sends it to the server in real time. The server uses a generative AI model to compare the voice data with the registered "son's" voice, and if it determines there is a mismatch, the smart glasses' display displays a warning saying, "This call may be suspicious. Please check." This allows the user to end the call and check with their actual family member.

[1311] Prompt Sentence Examples

[1312] Design a smart glasses application that analyzes the call in real time when an elderly person receives a fraudulent phone call, and displays a warning message if the voice does not match that of a registered family member. Specifically, please explain the functions of the smart glasses, including voice recognition, real-time analysis, and warning display functions.

[1313] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1314] Step 1:

[1315] User voice recording and registration

[1316] Users use a microphone connected to the smart glasses to record the voices of their family and relatives. The recorded voice data is sent from the device to a server. The server then analyzes the received voice data using a generative AI model, extracts features, and stores them. This provides the reference data needed for subsequent call analysis.

[1317] Input: User-recorded voice data of family and relatives

[1318] Data processing: speech analysis and feature extraction using generative AI models

[1319] Output: Analyzed audio data

[1320] Step 2:

[1321] Real-time analysis of call data

[1322] When the device receives a call, the microphone in the smart glasses picks up the call and the device transmits the audio data in real time to the server. The server then uses a generative AI model to analyze the received audio data and compare it with the voices of registered family members and relatives. Based on the comparison results, it determines whether the voices match.

[1323] Input: Audio data from the received call

[1324] Data processing: Voice analysis and matching with registered voice using a generative AI model

[1325] Output: Matching result (match / mismatch)

[1326] Step 3:

[1327] Email data analysis

[1328] When a device receives an email, it sends the email content to a server. The server uses a generative AI model to analyze the content of the received email and determine whether it is suspected of being fraudulent. Based on the analysis results, the likelihood of fraud is assessed.

[1329] Input: Text data of received email

[1330] Data processing: Text analysis with generative AI models

[1331] Output: Analysis result (suspected of fraud / not suspected)

[1332] Step 4:

[1333] Displaying warning notifications

[1334] Based on the analysis of calls and emails, the server generates a warning notification if fraud is suspected and sends it to the device, which then displays a warning on the smart glasses display saying, "This call / email may be suspicious. Please check."

[1335] Input: Call and email matching or analysis results

[1336] Data processing: Generate warning messages

[1337] Output: Displaying a warning notification

[1338] Step 5:

[1339] Providing recommendations for countermeasures

[1340] The smart glasses display will show a warning message along with suggested actions to take. The message will read, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Ask a family member" or "Contact the nearest public agency." This will allow the user to take appropriate action.

[1341] Input: Warning Notification

[1342] Data manipulation: Providing countermeasure options

[1343] Output: User's selected action

[1344] Step 6:

[1345] Regular updates to generative AI models

[1346] The server periodically updates the generative AI model, learning about new fraud techniques and building the latest model. The server then sends the update information to the device, which then displays a notification on the smart glasses, prompting the user to update the app. When the user updates, the latest generative AI model is installed.

[1347] Input: New dataset, trained generative AI model

[1348] Data processing: Retraining and updating generative AI models

[1349] Output: Install the latest generative AI model

[1350] Through these processing steps, the system helps the elderly and the information-poor to protect themselves from the risk of fraud, and the visual warning notification via the smart glasses allows users to instantly understand the situation and take appropriate measures.

[1351] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1352] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following components: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[1353] Voice registration function

[1354] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[1355] Call and email analysis function

[1356] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[1357] Emotion engine function

[1358] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[1359] Warning notification function

[1360] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[1361] Countermeasure recommendation function

[1362] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[1363] Model update function

[1364] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[1365] Specific examples

[1366] Specific examples of calls

[1367] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of its judgment to the device, and at the same time, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1368] Specific examples of emails

[1369] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[1370] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[1371] The processing flow will be explained below.

[1372] Voice registration function processing steps

[1373] Step 1:

[1374] The user installs and launches the app.

[1375] Step 2:

[1376] The device prompts the user to agree to the terms of use.

[1377] Step 3:

[1378] The user agrees to the terms of use.

[1379] Step 4:

[1380] The device prompts the user to record the voices of family members and relatives.

[1381] Step 5:

[1382] The user records the voices of family members or relatives.

[1383] Step 6:

[1384] The device sends the recorded audio data to the server.

[1385] Step 7:

[1386] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[1387] Processing steps of the call analysis function

[1388] Step 1:

[1389] The device receives the call.

[1390] Step 2:

[1391] The terminal transmits the voice data of the received call to the server in real time.

[1392] Step 3:

[1393] The server analyzes the transmitted voice data using a generative AI model.

[1394] Step 4:

[1395] The server compares the voice characteristics with the voice data of registered family members and relatives.

[1396] Step 5:

[1397] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[1398] Step 6:

[1399] The server sends the discrimination result and the user's emotional state to the terminal.

[1400] Step 7:

[1401] The device will display a warning to the user saying, "This call may be suspicious. Please check." The emotion engine will adjust the strength of the warning and the message.

[1402] Step 8:

[1403] The user sees the warning notification and chooses whether to end the call or continue.

[1404] Processing steps of email analysis function

[1405] Step 1:

[1406] Your device receives a new email.

[1407] Step 2:

[1408] The terminal sends the contents of the received email to the server.

[1409] Step 3:

[1410] The server analyzes the email content using a generative AI model.

[1411] Step 4:

[1412] The server checks the sender information and content to determine if there is any suspicion of fraud.

[1413] Step 5:

[1414] The server sends the discrimination result and the user's emotional state to the terminal.

[1415] Step 6:

[1416] The device displays a warning to the user saying, "This email may be fraudulent. Please delete it." The emotion engine adjusts the strength of the warning and the message.

[1417] Step 7:

[1418] The user reviews the warning notification and chooses whether to delete the email.

[1419] Emotion Engine Function Processing Steps

[1420] Step 1:

[1421] The device activates the camera and microphone to analyze the user's facial expressions and voice.

[1422] Step 2:

[1423] The device analyzes the user's facial expressions and tone of voice in real time to determine their emotional state.

[1424] Step 3:

[1425] The device transmits the emotion data to the server.

[1426] Step 4:

[1427] The server analyzes the emotional data and correlates it with calls and emails that appear to be fraudulent.

[1428] Steps for recommending measures

[1429] Step 1:

[1430] The device will notify the user of the warning and provide countermeasures at the same time.

[1431] Step 2:

[1432] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[1433] Step 3:

[1434] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[1435] Step 4:

[1436] The device will display appropriate contact information and procedures based on the user's selection, and an emotion engine will also display messages encouraging users to remain calm based on their emotional state.

[1437] Model Update Function Processing Steps

[1438] Step 1:

[1439] The server periodically updates the generative AI model.

[1440] Step 2:

[1441] The server learns about new fraud techniques and updates the model.

[1442] Step 3:

[1443] The server sends the update information to the device.

[1444] Step 4:

[1445] The terminal receives the update information and notifies the user.

[1446] Step 5:

[1447] The user updates the app and installs the latest generative AI model.

[1448] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud. In particular, the emotion engine function enables appropriate responses according to the user's emotional state, thereby enhancing the effectiveness of fraud prevention.

[1449] Example 2

[1450] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1451] The elderly and those with limited information are highly vulnerable to fraud, particularly fraud via telephone and email. This often results in financial loss and mental stress. Current fraud prevention systems are not sufficient to protect against this problem, and more effective measures are needed.

[1452] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1453] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, and a means for displaying a warning to the user if the received call or email is determined to be suspicious based on the analysis results, thereby enabling elderly people and those with limited information to be protected from fraud in real time.

[1454] A "generative AI model" is an artificial intelligence model that analyzes voice and text data and detects patterns based on its features.

[1455] The "voice registration means" is a function that allows the user to record the voices of family members and relatives and transmit the data to the server.

[1456] The "call analysis means" is a function that sends the voice data of the call received by the terminal to a server and analyzes it using a generative AI model.

[1457] The "email analysis means" is a function that sends data of emails received by the terminal to a server and analyzes them using a generative AI model.

[1458] The "warning notification means" is a function that issues a warning to the user if a received call or email is determined to be suspicious based on the analysis results.

[1459] The "emotion engine" is a system for determining a user's emotional state from their facial expressions and tone of voice.

[1460] "Emotion recognition means" is a function that enables the terminal to detect the user's emotions in real time and transmit that data to the server.

[1461] "Update methods" are functions that periodically train the generative AI model to respond to new fraudulent methods.

[1462] "Recommended measures" is a function that suggests appropriate measures to the user when they receive a suspicious call or email.

[1463] The present invention provides a smartphone application for fraud prevention targeted at the elderly and those with limited information access. Specific embodiments of the application are described below.

[1464] System Configuration

[1465] This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[1466] Voice registration function

[1467] When a user installs and launches the app on their smartphone, the system first displays a screen for recording and registering the voices of family members and relatives. The user records the voices of their family members and relatives, and the device sends the recording data to the server. The server then analyzes this voice data using a generative AI model, extracting and saving features. This makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[1468] Call and email analysis function

[1469] When the device receives a call, the voice data is sent to the server in real time. The server then uses a generative AI model to analyze the voice data and compare it with the voices of registered family members and relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[1470] Emotion engine function

[1471] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[1472] Warning notification function

[1473] If the server determines that the content of the call or email is suspicious and that the user's emotional state is unstable, the server sends the result of the analysis to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning is adjusted by the emotion engine depending on the user's emotional state.

[1474] Countermeasure recommendation function

[1475] At the same time as issuing the warning notification, the device will also suggest specific countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below," and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The emotion engine will also display response procedures based on the user's emotional state. For example, if the user is panicking, a message urging them to respond calmly will be displayed.

[1476] Model update function

[1477] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then sends a notification to the user. When the user updates the app, the latest generative AI model is installed, keeping the entire system up to date.

[1478] Specific examples

[1479] Specific examples of calls

[1480] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. When the server sends the results of its assessment to the device, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1481] Specific examples of emails

[1482] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[1483] Examples of prompt statements

[1484] "Describe a smartphone app that helps seniors avoid fraud."

[1485] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1486] Step 1:

[1487] The user installs the app on their smartphone and launches it. When the app is launched for the first time, a screen prompting the user to register their voice is displayed. This screen displays instructions such as, "Please record the voices of your family and relatives."

[1488] Step 2:

[1489] The user records the voice of a family member or relative. The user records a voice such as "Hello, I'm Grandpa." This provides voice data (input).

[1490] Step 3:

[1491] The device sends the recorded voice data to the server. The data is sent using a secure protocol (e.g. HTTPS). This action passes the recorded voice data (input) to the server.

[1492] Step 4:

[1493] The server analyzes the received voice data using a generative AI model to extract voice features (data processing and data calculation). For example, it calculates features such as voice tone and speaking rate. The analysis results (output) are used in the next step.

[1494] Step 5:

[1495] The server stores the extracted voice features in a database, which is used for later analysis of calls and emails (data accumulation).

[1496] Step 6:

[1497] When a device receives a call, it sends audio data to the server in real time. At the start of a call, the device captures the audio stream and sends it to the server (input).

[1498] Step 7:

[1499] The server analyzes the received voice data using a generative AI model and compares it with the recorded voices of family members and relatives (data calculation). It checks whether there is a match and passes the result (output) to the next step.

[1500] Step 8:

[1501] The terminal sends the contents of the received email to the server. The email data (input) is transferred to the server.

[1502] Step 9:

[1503] The server analyzes the email data using a generative AI model to determine whether there is any suspicion of fraud (data calculation). If there is a possibility of fraud, the analysis result (output) is passed to the next step.

[1504] Step 10:

[1505] The device recognizes the user's emotions in real time and sends the data to the server. Emotional data (input) is collected through facial recognition and voice tone analysis.

[1506] Step 11:

[1507] The server analyzes the received emotion data and determines the user's emotional state (data calculation). For example, emotions such as anxiety or impatience are detected. The analysis results (output) are used in the next step.

[1508] Step 12:

[1509] The server generates a warning notification based on the results of the call and email analysis and the user's emotional state. If a suspicious call or email is detected and the emotional state is recognized as unstable, a warning is generated (data generation).

[1510] Step 13:

[1511] The server sends a warning notification to the device, which includes the analysis results and a message according to the emotional state (output).

[1512] Step 14:

[1513] The device displays a warning notification to the user. A warning such as "This call / email may be suspicious. Please check it" is displayed. The user considers how to respond based on the displayed warning (output).

[1514] Step 15:

[1515] The server proposes appropriate measures to the user, providing options such as "check with family" or "contact the nearest public institution" (data generation).

[1516] Step 16:

[1517] The server customizes countermeasure procedures based on the user's emotional state. It uses an emotion engine to add messages encouraging users to remain calm (data generation).

[1518] Step 17:

[1519] The server periodically updates the generated AI model. The model learns about new fraud methods and sends updated information to the device (data processing and data calculation).

[1520] Step 18:

[1521] The device notifies the user of update information. When the user updates the app, the latest generative AI model is installed, allowing the entire system to respond to the latest fraudulent techniques (data accumulation).

[1522] (Application example 2)

[1523] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1524] Currently, the elderly and those with limited information are vulnerable to fraud, and it is difficult to prevent such damage. Virtual stores also face the same risk of fraud, making real-time fraud prevention particularly necessary. Another issue is the lack of systems that provide warnings and countermeasures that take into account the user's mental state.

[1525] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for analyzing and monitoring the user's emotional state, and a means for adjusting the content of the notification if the user shows anxiety or impatience. This reduces the risk of the user being scammed in a virtual store and makes it possible to provide appropriate warnings and countermeasures that take into account the user's mental state.

[1526] "Elderly" refers to individuals who are aging, generally aged 65 or older, and who, for social and physical reasons, are more likely to be targeted by fraud.

[1527] "Informationally vulnerable" refers to individuals who have little knowledge or experience regarding the Internet or digital devices and therefore have a low resistance to fraud.

[1528] A "fraud prevention system" is a system that includes technical means and methods to prevent users from becoming victims of fraud.

[1529] "Voice recording and registration means" refers to technology or devices that store the voice of a person designated by the user as digital data and make it available within the system.

[1530] "Means of analyzing call and email data using generative AI models" refers to technologies and methods that use artificial intelligence technology to analyze received voice and text data and understand its context and characteristics.

[1531] "Means for displaying a warning notice" refers to a device or technology for visually or audibly conveying a warning message to a user based on the analysis results.

[1532] "Means for analyzing and monitoring emotional state" refers to technology that analyzes a user's voice, facial expressions, etc. in real time to estimate their emotions and psychological state.

[1533] "Means for adjusting notification content" refers to techniques or methods for changing the intensity of the notification content or the content of the message depending on the emotional state of the user.

[1534] A "virtual store" refers to a shopping environment or platform built on the Internet, rather than a physical store.

[1535] This system is a fraud prevention system aimed at the elderly and those with limited information, and operates primarily on mobile devices such as smartphones and tablets. The system consists of the following main components: voice registration, call and email analysis, emotion engine, warning notification, countermeasure recommendation, and model update. This system can also be applied to fraud prevention in virtual stores.

[1536] Voice registration function

[1537] When a user installs and launches the app, they are first provided with a means to record and register the voices of their family and relatives. The user records the voices of each family member and relative, and the device sends the recordings to the server. The server then analyzes the voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[1538] Call and email analysis function

[1539] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[1540] Emotion engine function

[1541] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[1542] Warning notification function

[1543] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[1544] Countermeasure recommendation function

[1545] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[1546] Model update function

[1547] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[1548] Examples and prompts

[1549] Example

[1550] Example of a call: When a user receives a call from someone claiming to be their "grandchild," the device analyzes the call in real time and sends the voice data to the server. The server uses a generative AI model to analyze the voice data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, and the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1551] Example of email: When a user receives a fraudulent email purporting to be from a different sender, the device sends the received email to a server, which analyzes the email content using a generative AI model. If it determines that the email is suspected to be fraudulent, the server sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message will be displayed saying, "Please remain calm and respond accordingly." The user will then see the warning and delete the email, avoiding opening any suspicious links or attachments.

[1552] Prompt Sentence Examples

[1553] "Analyze the given audio data for known voice features and determine if it matches any registered voices. If a match is not found or the user appears anxious, send a cautionary notification to the user's device."

[1554] "Analyze the given message data to detect any fraudulent content using the AI ​​model. If a fraud is detected, notify the user with a warning message."

[1555] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[1556] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1557] Step 1:

[1558] (Audio recording and registration)

[1559] When a user installs and launches the app, the server provides the user with a screen for recording and registering the voices of their family and relatives.

[1560] Input: User-recorded voice data of family and relatives

[1561] Processing: The device sends the recorded voice data to the server, which uses a generative AI model to extract voice features.

[1562] Output: Family and relatives' characteristics data stored on the server

[1563] How it works: The server analyzes the feature data and stores it in a database as a registered voice profile.

[1564] Step 2:

[1565] (Real-time analysis of call data)

[1566] When the terminal receives a call, the call data is sent to the server in real time.

[1567] Input: Audio data from the received call

[1568] Processing: The server uses the generative AI model to analyze the voice data and match it with the registered voice profile.

[1569] Output: Analysis results (whether or not there is suspicion of fraud)

[1570] Operation: The server performs real-time matching and sends the results to the device.

[1571] Step 3:

[1572] (Email data analysis)

[1573] The terminal sends the contents of the received email to the server.

[1574] Input: Received email data

[1575] Processing: The server uses the generative AI model to analyze the text of the email and determine whether it is likely to be fraudulent.

[1576] Output: Analysis results (whether or not there is suspicion of fraud)

[1577] How it works: Based on the analysis results, the server determines whether the email is likely to be fraudulent and sends the result to the device.

[1578] Step 4:

[1579] (Monitoring emotional state)

[1580] The terminal monitors the user's emotional state.

[1581] Input: User's voice tone and facial expression data

[1582] Processing: The server uses the emotion engine to analyze the user's emotional state.

[1583] Output: Emotional state judgment result

[1584] Operation: The server analyzes the user's emotional state and sends the analysis results to the device.

[1585] Step 5:

[1586] (Display warning notification)

[1587] The server displays a warning notification to the user based on the analysis of calls and emails and the user's emotional state.

[1588] Input: Analysis results of calls and emails, emotional state determination results

[1589] Processing: The server generates the warning content and sends it to the terminal.

[1590] Output: A warning notice that is displayed to the user

[1591] What it does: The device displays a warning notification to the user saying "This call / email may be suspicious."

[1592] Step 6:

[1593] (Recommended measures)

[1594] The device recommends countermeasures to the user.

[1595] Input: Warning notification content

[1596] Processing: The server generates appropriate countermeasures and sends them to the terminal.

[1597] Output: Recommended measures

[1598] What it does: The device prompts the user with options such as "Check with a family member" or "Contact the nearest public agency."

[1599] Step 7:

[1600] (Regular model updates)

[1601] The server periodically updates the generative AI model.

[1602] Input: Latest scam information

[1603] Processing: The server trains the generated AI model on new fraud techniques.

[1604] Output: Updated generative AI model

[1605] How it works: The server sends update information to the device and notifies the user. When the user updates the app, the latest generative AI model is installed.

[1606] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1607] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1608] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1609] [Fourth embodiment]

[1610] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1611] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1612] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1613] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1614] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1615] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1616] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1617] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1618] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1619] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1620] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1621] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1622] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1623] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, and 5) model update function.

[1624] Voice registration function

[1625] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[1626] Call and email analysis function

[1627] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[1628] Warning notification function

[1629] If the server determines that a received call or email is suspicious, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[1630] Countermeasure recommendation function

[1631] At the same time as receiving the warning, the device will also suggest countermeasures to the user. The message will say, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[1632] Model update function

[1633] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[1634] Specific examples

[1635] Specific examples of calls

[1636] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1637] Specific examples of emails

[1638] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[1639] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[1640] The processing flow will be explained below.

[1641] Voice registration function processing steps

[1642] Step 1:

[1643] The user installs and launches the app.

[1644] Step 2:

[1645] The device prompts the user to agree to the terms of use.

[1646] Step 3:

[1647] The user agrees to the terms of use.

[1648] Step 4:

[1649] The device prompts the user to record the voices of family members and relatives.

[1650] Step 5:

[1651] The user records the voices of family members or relatives.

[1652] Step 6:

[1653] The device sends the recorded audio data to the server.

[1654] Step 7:

[1655] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[1656] Processing steps of the call analysis function

[1657] Step 1:

[1658] The device receives the call.

[1659] Step 2:

[1660] The voice data received by the terminal is transmitted to the server in real time.

[1661] Step 3:

[1662] The server analyzes the transmitted voice data using a generative AI model.

[1663] Step 4:

[1664] The server compares the voice characteristics with the voice data of registered family members and relatives.

[1665] Step 5:

[1666] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[1667] Step 6:

[1668] The server transmits the determination result to the terminal.

[1669] Step 7:

[1670] The device displays a warning to the user saying, "This call may be suspicious. Please verify."

[1671] Step 8:

[1672] The user sees the warning notification and chooses whether to end the call or continue.

[1673] Processing steps of email analysis function

[1674] Step 1:

[1675] Your device receives a new email.

[1676] Step 2:

[1677] The terminal sends the contents of the received email to the server.

[1678] Step 3:

[1679] The server analyzes the email content using a generative AI model.

[1680] Step 4:

[1681] The server checks the sender information and content to determine if there is any suspicion of fraud.

[1682] Step 5:

[1683] The server transmits the determination result to the terminal.

[1684] Step 6:

[1685] The device will display a warning to the user saying, "This email may be fraudulent. Please delete it."

[1686] Step 7:

[1687] The user reviews the warning notification and chooses whether to delete the email.

[1688] Steps for recommending measures

[1689] Step 1:

[1690] The device will notify the user of the warning and provide countermeasures at the same time.

[1691] Step 2:

[1692] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[1693] Step 3:

[1694] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[1695] Step 4:

[1696] The device will display appropriate contact information and instructions based on the user's selection.

[1697] Model Update Function Processing Steps

[1698] Step 1:

[1699] The server periodically updates the generative AI model.

[1700] Step 2:

[1701] The server learns about new fraud techniques and updates the model.

[1702] Step 3:

[1703] The server sends the update information to the device.

[1704] Step 4:

[1705] The terminal receives the update information and notifies the user.

[1706] Step 5:

[1707] The user updates the app and installs the latest generative AI model.

[1708] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud.

[1709] Example 1

[1710] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1711] The elderly and those with limited information are easy targets for fraud, and the damage they cause is serious. To prevent these types of fraud, a system is needed that can properly analyze the content of received calls and emails, quickly warn users if fraud is suspected, and suggest countermeasures. Furthermore, because fraud methods are evolving daily, the accuracy of the system must be kept up to date.

[1712] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1713] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for presenting countermeasures to the user simultaneously with the warning notification, and a means for regularly updating the generative AI model. This makes it possible to quickly and appropriately determine the risk of fraud and present effective countermeasures to the user. In addition, by constantly updating the generative AI model to keep up with the latest fraudulent techniques, the effectiveness of the system can be maintained.

[1714] "Users" refers to the elderly and information-poor people who use the system.

[1715] "Means for recording and registering voice" refers to the functionality that allows users to record the voices of their family members or relatives and store them in the system's database.

[1716] "Means for analyzing received call and email data using a generative AI model" refers to a function that allows the system to analyze received call and email data using a generative AI model and evaluate its content.

[1717] "Means for displaying a warning notification" refers to a function for displaying a warning message to the user when a suspicious call or email is identified based on the analysis results.

[1718] "Means for suggesting countermeasures" refers to a function for presenting specific countermeasures to the user at the same time as issuing a warning notification.

[1719] "Means for regularly updating the generative AI model" refers to the system's ability to regularly update the generative AI model to respond to new fraudulent methods.

[1720] "Means for notifying users that an updated generative AI model is available" refers to a function for notifying users that an updated generative AI model is available.

[1721] "Means to recommend contacting family members or public institutions" refers to a function that recommends that users contact family members or public institutions when a suspicious call or email is identified.

[1722] The system of this invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following elements: voice registration function, call and email analysis function, warning notification function, countermeasure recommendation function, and model update function.

[1723] Voice registration function

[1724] First, the user installs the system's application on their smartphone and launches the app. Next, they acquire voice data by recording the voices of their family and relatives. The device then sends the recorded voice data to the server. The server then analyzes the received voice data using a generative AI model, extracts voice characteristics, and stores them in a database. This voice registration function makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[1725] Call and email analysis function

[1726] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with pre-registered voices. Similarly, when the device receives an email, the content of that email is also sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud based on the analysis results.

[1727] Warning notification function

[1728] If the server determines that the received call or email is suspicious based on the analysis results, it sends the result to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[1729] Countermeasure recommendation function

[1730] At the same time as the warning notification, the device will also suggest specific countermeasures to the user. For example, a message will be displayed saying, "This call / email may be suspicious. Please take action from the options below." The user will be able to choose from options such as "Check with a family member" or "Contact the nearest public institution." This will further reduce the risk of fraud.

[1731] Model update function

[1732] The server periodically updates the generative AI model and learns about new fraud techniques. When the latest generative AI model becomes available, the server sends this information to the device. The device then sends an update notification to the user, displaying the message "The latest generative AI model is available. Please update your app." When the user updates the app, the latest generative AI model is installed.

[1733] Specific examples

[1734] Specific examples of calls

[1735] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, which then displays a warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1736] Specific examples of emails

[1737] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the results of its assessment to the device, which then displays a warning to the user saying, "This email may be fraudulent. Please delete it." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[1738] Prompt Sentence Examples

[1739] 1. "Determine if this audio belongs to a specific family member"

[1740] 2. "Evaluate the likelihood that this email is fraudulent."

[1741] 3. "Analyze whether the caller matches the recorded voice."

[1742] This will enable the system to be implemented to prevent fraud and protect the elderly and those with limited information access.

[1743] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1744] Step 1:

[1745] Installing and launching the app

[1746] A user installs and launches a fraud prevention app on their smartphone. The app is installed from the app store, and upon launch, an initial setup screen appears.

[1747] Input: Smartphone operation.

[1748] Output: The app is installed and launched.

[1749] Step 2:

[1750] Voice recording and transmission

[1751] Users can use the app's "audio registration" function to record the voices of their family and relatives, and also enter their names when recording.

[1752] The device compresses the recorded audio data and sends it to the server using a security protocol such as SSL / TLS.

[1753] Input: Audio data recorded by the user.

[1754] Output: The audio data is sent to the server and stored.

[1755] Step 3:

[1756] Audio analysis and storage

[1757] The server analyzes the received voice data using a generative AI model to extract voice characteristics, specifically, multiple parameters such as voice frequency characteristics, pitch, and tempo.

[1758] The server registers the extracted feature data and voice data in a database.

[1759] Input: Audio data sent from the device.

[1760] Output: The extracted speech features are stored in a database.

[1761] Step 4:

[1762] Receiving and sending calls

[1763] When the device receives a call, it automatically starts the process of recording the audio data in real time and sending it to the server.

[1764] The device encrypts the voice data using SSL / TLS and sends it to the server in real time.

[1765] Input: Audio data from the received call.

[1766] Output: Audio data is sent to the server in real time.

[1767] Step 5:

[1768] Voice analysis and judgment

[1769] The server instantly analyzes the transmitted voice data using a generative AI model, which involves comparing the voiceprint with registered voice data.

[1770] The server uses the analysis results to determine whether there is a possibility of fraud.

[1771] Input: Audio data from the received call.

[1772] Output: Determination result regarding likelihood of fraud.

[1773] Step 6:

[1774] Receiving and sending emails

[1775] When the terminal receives an email, the content data is automatically sent to the server.

[1776] The device converts the text content of the email into a format that is easy to parse and then executes the sending process.

[1777] Input: Content data of received email.

[1778] Output: The email content is sent to the server.

[1779] Step 7:

[1780] Email analysis and judgment

[1781] The server analyzes the content of the received email using a generative AI model, specifically assessing the likelihood of fraud based on the content's grammar, keywords, sender address, etc.

[1782] The server uses the analysis results to determine whether the email is likely to be fraudulent.

[1783] Input: Email content data sent from the terminal.

[1784] Output: Determination result regarding likelihood of fraud.

[1785] Step 8:

[1786] Warning notice

[1787] The server sends the analysis results to the terminal.

[1788] Based on the results of the judgment, the device displays a warning notification to the user saying, "This call / email may be suspicious. Please check."

[1789] Input: Verification result from the server.

[1790] Output: A warning notice to the user.

[1791] Step 9:

[1792] Countermeasures

[1793] Along with the warning notification, the device will also present the user with suggestions for countermeasures, displaying the message "This call / email may be suspicious. Please take action from the options below."

[1794] The user selects countermeasures such as "check with family" or "contact public institutions."

[1795] Input: Verification result and countermeasures from the server.

[1796] Output: Proposal of countermeasures to the user.

[1797] Step 10:

[1798] Model Update

[1799] The server periodically updates the generative AI model to learn about new fraud schemes, and this updating happens in the background.

[1800] The server sends information about the new model to the device.

[1801] The device sends a notification to the user saying, "A newer generative AI model is available. Please update your app."

[1802] Input: Data on new scam schemes.

[1803] Output: Latest generative AI model update and user notification.

[1804] (Application example 1)

[1805] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1806] It is important to help the elderly and those with limited information to avoid falling victim to fraud. However, frauds perpetrated via telephone and email are becoming increasingly sophisticated, making it difficult for the elderly and those with limited information to accurately assess these threats on their own. Furthermore, if the devices they use are inconvenient in terms of operability and the way information is presented, a fundamental solution will not be reached. Therefore, there is a need for a system that provides visual information, identifies fraud in real time, and suggests appropriate countermeasures.

[1807] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1808] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This enables elderly people and those with limited information to receive warnings about suspected fraudulent calls and emails in real time and take appropriate measures promptly.

[1809] "Users" refers to the elderly and information-poor people who use the system.

[1810] "Voice of family and relatives" refers to voice data of the user's family and relatives registered in the system.

[1811] "Means for recording and registering" refers to a mechanism that allows users to record the voices of their family members and relatives and register that voice data in the system.

[1812] "Received call and email data" refers to data including the contents of call audio and emails received on the user's smart device.

[1813] "Generative AI model" refers to a machine learning model that analyzes collected voice and email data to determine suspected fraud.

[1814] "Means of analysis" refers to the mechanism for analyzing received call and email data using a generative AI model.

[1815] "Means for displaying a warning notice" refers to a function that notifies the user of suspected fraud based on the analysis results.

[1816] "Means for displaying a notification on the smart glasses" refers to a mechanism for visually displaying an alert notification to a user using the smart glasses.

[1817] "Means for recommending contacting public institutions" refers to a function that recommends to the user to contact family members or public institutions as necessary.

[1818] "Means for regularly updating the generative AI model" refers to the ability to regularly update the generative AI model to respond to the latest fraud techniques.

[1819] This invention is a fraud prevention system using smart glasses for the elderly and those with limited information. The system includes a means for users to record and register the voices of their family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is deemed suspicious based on the analysis results, and a means for displaying the notification on the smart glasses. This allows the elderly and those with limited information to receive warnings in real time about calls or emails that may be fraudulent and take appropriate measures promptly.

[1820] System configuration

[1821] Voice registration function

[1822] When a user first uses the system, they record the voice of their family or relatives using a microphone connected to the smart glasses. The recorded voice data is sent to a server, which analyzes it using a generative AI model to extract and store features that enable speaker recognition during later call analysis.

[1823] Call and email analysis function

[1824] When a user's smart glasses receive a call, the voice data is sent to the server in real time. The server uses a generative AI model to analyze the voice data and compare it with the voice data of pre-registered family members and relatives. Similarly, the content of emails received by the user is sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[1825] Warning notification function

[1826] If the server determines that a received call or email is suspicious, the result of the determination is sent to the smart glasses. The smart glasses then display a warning to the user saying, "This call / email may be suspicious. Please check." This makes it easier for the user to make a calm decision.

[1827] Countermeasure recommendation function

[1828] At the same time as sending the warning, the smart glasses also suggest countermeasures to the user. The message "This call / email may be suspicious. Please take action from the options below" is displayed, and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." This further reduces the risk of fraud.

[1829] Model update function

[1830] The server periodically updates the generative AI model to learn about new fraud techniques. The server sends the updated information to the smart glasses, which then notify the user. When the user updates the app, the latest generative AI model is installed.

[1831] Hardware and software used

[1832] Smart glasses: Requires a microphone for voice input and a display for notifications. Examples include Google Glass and Microsoft HoloLens.

[1833] Server: A high-performance server is required, and analysis processing is performed on the server.

[1834] Python: The main logic of the program uses Python.

[1835] speech_recognition library: Use this library for speech recognition.

[1836] Generative AI model: Uses deep learning frameworks (e.g., TensorFlow, PyTorch) for voice analysis and judgment.

[1837] Specific examples

[1838] When a user receives a call from someone claiming to be their "son," the smart glasses' microphone picks up the call and sends it to the server in real time. The server uses a generative AI model to compare the voice data with the registered "son's" voice, and if it determines there is a mismatch, the smart glasses' display displays a warning saying, "This call may be suspicious. Please check." This allows the user to end the call and check with their actual family member.

[1839] Prompt Sentence Examples

[1840] Design a smart glasses application that analyzes the call in real time when an elderly person receives a fraudulent phone call, and displays a warning message if the voice does not match that of a registered family member. Specifically, please explain the functions of the smart glasses, including voice recognition, real-time analysis, and warning display functions.

[1841] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1842] Step 1:

[1843] User voice recording and registration

[1844] Users use a microphone connected to the smart glasses to record the voices of their family and relatives. The recorded voice data is sent from the device to a server. The server then analyzes the received voice data using a generative AI model, extracts features, and stores them. This provides the reference data needed for subsequent call analysis.

[1845] Input: User-recorded voice data of family and relatives

[1846] Data processing: speech analysis and feature extraction using generative AI models

[1847] Output: Analyzed audio data

[1848] Step 2:

[1849] Real-time analysis of call data

[1850] When the device receives a call, the microphone in the smart glasses picks up the call and the device transmits the audio data in real time to the server. The server then uses a generative AI model to analyze the received audio data and compare it with the voices of registered family members and relatives. Based on the comparison results, it determines whether the voices match.

[1851] Input: Audio data from the received call

[1852] Data processing: Voice analysis and matching with registered voice using a generative AI model

[1853] Output: Matching result (match / mismatch)

[1854] Step 3:

[1855] Email data analysis

[1856] When a device receives an email, it sends the email content to a server. The server uses a generative AI model to analyze the content of the received email and determine whether it is suspected of being fraudulent. Based on the analysis results, the likelihood of fraud is assessed.

[1857] Input: Text data of received email

[1858] Data processing: Text analysis with generative AI models

[1859] Output: Analysis result (suspected of fraud / not suspected)

[1860] Step 4:

[1861] Displaying warning notifications

[1862] Based on the analysis of calls and emails, the server generates a warning notification if fraud is suspected and sends it to the device, which then displays a warning on the smart glasses display saying, "This call / email may be suspicious. Please check."

[1863] Input: Call and email matching or analysis results

[1864] Data processing: Generate warning messages

[1865] Output: Displaying a warning notification

[1866] Step 5:

[1867] Providing recommendations for countermeasures

[1868] The smart glasses display will show a warning message along with suggested actions to take. The message will read, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Ask a family member" or "Contact the nearest public agency." This will allow the user to take appropriate action.

[1869] Input: Warning Notification

[1870] Data manipulation: Providing countermeasure options

[1871] Output: User's selected action

[1872] Step 6:

[1873] Regular updates to generative AI models

[1874] The server periodically updates the generative AI model, learning about new fraud techniques and building the latest model. The server then sends the update information to the device, which then displays a notification on the smart glasses, prompting the user to update the app. When the user updates, the latest generative AI model is installed.

[1875] Input: New dataset, trained generative AI model

[1876] Data processing: Retraining and updating generative AI models

[1877] Output: Install the latest generative AI model

[1878] Through these processing steps, the system helps the elderly and the information-poor to protect themselves from the risk of fraud, and the visual warning notification via the smart glasses allows users to instantly understand the situation and take appropriate measures.

[1879] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1880] The system of the present invention is implemented as a smartphone application for fraud prevention aimed at the elderly and those with limited information. This system mainly consists of the following components: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[1881] Voice registration function

[1882] Once a user installs and launches the app, the system first provides a means to record and register the voices of family members and relatives. The user records each family member's voice, and the device sends the recording to the server. The server analyzes this voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[1883] Call and email analysis function

[1884] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[1885] Emotion engine function

[1886] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[1887] Warning notification function

[1888] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[1889] Countermeasure recommendation function

[1890] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[1891] Model update function

[1892] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[1893] Specific examples

[1894] Specific examples of calls

[1895] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of its judgment to the device, and at the same time, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[1896] Specific examples of emails

[1897] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[1898] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[1899] The processing flow will be explained below.

[1900] Voice registration function processing steps

[1901] Step 1:

[1902] The user installs and launches the app.

[1903] Step 2:

[1904] The device prompts the user to agree to the terms of use.

[1905] Step 3:

[1906] The user agrees to the terms of use.

[1907] Step 4:

[1908] The device prompts the user to record the voices of family members and relatives.

[1909] Step 5:

[1910] The user records the voices of family members or relatives.

[1911] Step 6:

[1912] The device sends the recorded audio data to the server.

[1913] Step 7:

[1914] The server analyzes the voice data using a generative AI model, extracts features, and stores them.

[1915] Processing steps of the call analysis function

[1916] Step 1:

[1917] The device receives the call.

[1918] Step 2:

[1919] The terminal transmits the voice data of the received call to the server in real time.

[1920] Step 3:

[1921] The server analyzes the transmitted voice data using a generative AI model.

[1922] Step 4:

[1923] The server compares the voice characteristics with the voice data of registered family members and relatives.

[1924] Step 5:

[1925] The server analyzes the content of the call and determines whether there is any suspicion of fraud.

[1926] Step 6:

[1927] The server sends the discrimination result and the user's emotional state to the terminal.

[1928] Step 7:

[1929] The device will display a warning to the user saying, "This call may be suspicious. Please check." The emotion engine will adjust the strength of the warning and the message.

[1930] Step 8:

[1931] The user sees the warning notification and chooses whether to end the call or continue.

[1932] Processing steps of email analysis function

[1933] Step 1:

[1934] Your device receives a new email.

[1935] Step 2:

[1936] The terminal sends the contents of the received email to the server.

[1937] Step 3:

[1938] The server analyzes the email content using a generative AI model.

[1939] Step 4:

[1940] The server checks the sender information and content to determine if there is any suspicion of fraud.

[1941] Step 5:

[1942] The server sends the discrimination result and the user's emotional state to the terminal.

[1943] Step 6:

[1944] The device displays a warning to the user saying, "This email may be fraudulent. Please delete it." The emotion engine adjusts the strength of the warning and the message.

[1945] Step 7:

[1946] The user reviews the warning notification and chooses whether to delete the email.

[1947] Emotion Engine Function Processing Steps

[1948] Step 1:

[1949] The device activates the camera and microphone to analyze the user's facial expressions and voice.

[1950] Step 2:

[1951] The device analyzes the user's facial expressions and tone of voice in real time to determine their emotional state.

[1952] Step 3:

[1953] The device transmits the emotion data to the server.

[1954] Step 4:

[1955] The server analyzes the emotional data and correlates it with calls and emails that appear to be fraudulent.

[1956] Steps for recommending measures

[1957] Step 1:

[1958] The device will notify the user of the warning and provide countermeasures at the same time.

[1959] Step 2:

[1960] The device will display a message saying, "This call / email may be suspicious. Please take action from the options below."

[1961] Step 3:

[1962] The user selects one of the options, such as "check with family" or "contact the nearest public institution."

[1963] Step 4:

[1964] The device will display appropriate contact information and procedures based on the user's selection, and an emotion engine will also display messages encouraging users to remain calm based on their emotional state.

[1965] Model Update Function Processing Steps

[1966] Step 1:

[1967] The server periodically updates the generative AI model.

[1968] Step 2:

[1969] The server learns about new fraud techniques and updates the model.

[1970] Step 3:

[1971] The server sends the update information to the device.

[1972] Step 4:

[1973] The terminal receives the update information and notifies the user.

[1974] Step 5:

[1975] The user updates the app and installs the latest generative AI model.

[1976] Through the above processing steps, the system of the present invention functions to protect the elderly and those with limited information from fraud. In particular, the emotion engine function enables appropriate responses according to the user's emotional state, thereby enhancing the effectiveness of fraud prevention.

[1977] Example 2

[1978] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1979] The elderly and those with limited information are highly vulnerable to fraud, particularly fraud via telephone and email. This often results in financial loss and mental stress. Current fraud prevention systems are not sufficient to protect against this problem, and more effective measures are needed.

[1980] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1981] In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, and a means for displaying a warning to the user if the received call or email is determined to be suspicious based on the analysis results, thereby enabling elderly people and those with limited information to be protected from fraud in real time.

[1982] A "generative AI model" is an artificial intelligence model that analyzes voice and text data and detects patterns based on its features.

[1983] The "voice registration means" is a function that allows the user to record the voices of family members and relatives and transmit the data to the server.

[1984] The "call analysis means" is a function that sends the voice data of the call received by the terminal to a server and analyzes it using a generative AI model.

[1985] The "email analysis means" is a function that sends data of emails received by the terminal to a server and analyzes them using a generative AI model.

[1986] The "warning notification means" is a function that issues a warning to the user if a received call or email is determined to be suspicious based on the analysis results.

[1987] The "emotion engine" is a system for determining a user's emotional state from their facial expressions and tone of voice.

[1988] "Emotion recognition means" is a function that enables the terminal to detect the user's emotions in real time and transmit that data to the server.

[1989] "Update methods" are functions that periodically train the generative AI model to respond to new fraudulent methods.

[1990] "Recommended measures" is a function that suggests appropriate measures to the user when they receive a suspicious call or email.

[1991] The present invention provides a smartphone application for fraud prevention targeted at the elderly and those with limited information access. Specific embodiments of the application are described below.

[1992] System Configuration

[1993] This system mainly consists of the following elements: 1) voice registration function, 2) call and email analysis function, 3) warning notification function, 4) countermeasure recommendation function, 5) model update function, and 6) emotion engine function.

[1994] Voice registration function

[1995] When a user installs and launches the app on their smartphone, the system first displays a screen for recording and registering the voices of family members and relatives. The user records the voices of their family members and relatives, and the device sends the recording data to the server. The server then analyzes this voice data using a generative AI model, extracting and saving features. This makes it possible to recognize the speaker when analyzing subsequent calls or emails.

[1996] Call and email analysis function

[1997] When the device receives a call, the voice data is sent to the server in real time. The server then uses a generative AI model to analyze the voice data and compare it with the voices of registered family members and relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed by the generative AI model. The server then determines whether there is any suspicion of fraud.

[1998] Emotion engine function

[1999] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[2000] Warning notification function

[2001] If the server determines that the content of the call or email is suspicious and that the user's emotional state is unstable, the server sends the result of the analysis to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning is adjusted by the emotion engine depending on the user's emotional state.

[2002] Countermeasure recommendation function

[2003] At the same time as issuing the warning notification, the device will also suggest specific countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below," and the user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The emotion engine will also display response procedures based on the user's emotional state. For example, if the user is panicking, a message urging them to respond calmly will be displayed.

[2004] Model update function

[2005] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then sends a notification to the user. When the user updates the app, the latest generative AI model is installed, keeping the entire system up to date.

[2006] Specific examples

[2007] Specific examples of calls

[2008] Let's say a user receives a call from someone claiming to be their "grandchild." The device analyzes the call in real time and sends the audio data to the server. The server uses a generative AI model to analyze the audio data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. When the server sends the results of its assessment to the device, the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[2009] Specific examples of emails

[2010] Suppose a user receives a fraudulent email from a purported sender. The device sends the received email to a server, which analyzes the email content using a generative AI model. If the server determines that the email is suspected to be fraudulent, it sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message is displayed saying, "Please remain calm and respond accordingly." The user sees the warning and deletes the email, avoiding opening any suspicious links or attachments.

[2011] Examples of prompt statements

[2012] "Describe a smartphone app that helps seniors avoid fraud."

[2013] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2014] Step 1:

[2015] The user installs the app on their smartphone and launches it. When the app is launched for the first time, a screen prompting the user to register their voice is displayed. This screen displays instructions such as, "Please record the voices of your family and relatives."

[2016] Step 2:

[2017] The user records the voice of a family member or relative. The user records a voice such as "Hello, I'm Grandpa." This provides voice data (input).

[2018] Step 3:

[2019] The device sends the recorded voice data to the server. The data is sent using a secure protocol (e.g. HTTPS). This action passes the recorded voice data (input) to the server.

[2020] Step 4:

[2021] The server analyzes the received voice data using a generative AI model to extract voice features (data processing and data calculation). For example, it calculates features such as voice tone and speaking rate. The analysis results (output) are used in the next step.

[2022] Step 5:

[2023] The server stores the extracted voice features in a database, which is used for later analysis of calls and emails (data accumulation).

[2024] Step 6:

[2025] When a device receives a call, it sends audio data to the server in real time. At the start of a call, the device captures the audio stream and sends it to the server (input).

[2026] Step 7:

[2027] The server analyzes the received voice data using a generative AI model and compares it with the recorded voices of family members and relatives (data calculation). It checks whether there is a match and passes the result (output) to the next step.

[2028] Step 8:

[2029] The terminal sends the contents of the received email to the server. The email data (input) is transferred to the server.

[2030] Step 9:

[2031] The server analyzes the email data using a generative AI model to determine whether there is any suspicion of fraud (data calculation). If there is a possibility of fraud, the analysis result (output) is passed to the next step.

[2032] Step 10:

[2033] The device recognizes the user's emotions in real time and sends the data to the server. Emotional data (input) is collected through facial recognition and voice tone analysis.

[2034] Step 11:

[2035] The server analyzes the received emotion data and determines the user's emotional state (data calculation). For example, emotions such as anxiety or impatience are detected. The analysis results (output) are used in the next step.

[2036] Step 12:

[2037] The server generates a warning notification based on the results of the call and email analysis and the user's emotional state. If a suspicious call or email is detected and the emotional state is recognized as unstable, a warning is generated (data generation).

[2038] Step 13:

[2039] The server sends a warning notification to the device, which includes the analysis results and a message according to the emotional state (output).

[2040] Step 14:

[2041] The device displays a warning notification to the user. A warning such as "This call / email may be suspicious. Please check it" is displayed. The user considers how to respond based on the displayed warning (output).

[2042] Step 15:

[2043] The server proposes appropriate measures to the user, providing options such as "check with family" or "contact the nearest public institution" (data generation).

[2044] Step 16:

[2045] The server customizes countermeasure procedures based on the user's emotional state. It uses an emotion engine to add messages encouraging users to remain calm (data generation).

[2046] Step 17:

[2047] The server periodically updates the generated AI model. The model learns about new fraud methods and sends updated information to the device (data processing and data calculation).

[2048] Step 18:

[2049] The device notifies the user of update information. When the user updates the app, the latest generative AI model is installed, allowing the entire system to respond to the latest fraudulent techniques (data accumulation).

[2050] (Application example 2)

[2051] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2052] Currently, the elderly and those with limited information are vulnerable to fraud, and it is difficult to prevent such damage. Virtual stores also face the same risk of fraud, making real-time fraud prevention particularly necessary. Another issue is the lack of systems that provide warnings and countermeasures that take into account the user's mental state.

[2053] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for the user to record and register the voices of family members and relatives, a means for analyzing received call and email data using a generative AI model, a means for displaying a warning notification to the user if the received call or email is determined to be suspicious based on the analysis results, a means for analyzing and monitoring the user's emotional state, and a means for adjusting the content of the notification if the user shows anxiety or impatience. This reduces the risk of the user being scammed in a virtual store and makes it possible to provide appropriate warnings and countermeasures that take into account the user's mental state.

[2054] "Elderly" refers to individuals who are aging, generally aged 65 or older, and who, for social and physical reasons, are more likely to be targeted by fraud.

[2055] "Informationally vulnerable" refers to individuals who have little knowledge or experience regarding the Internet or digital devices and therefore have a low resistance to fraud.

[2056] A "fraud prevention system" is a system that includes technical means and methods to prevent users from becoming victims of fraud.

[2057] "Voice recording and registration means" refers to technology or devices that store the voice of a person designated by the user as digital data and make it available within the system.

[2058] "Means of analyzing call and email data using generative AI models" refers to technologies and methods that use artificial intelligence technology to analyze received voice and text data and understand its context and characteristics.

[2059] "Means for displaying a warning notice" refers to a device or technology for visually or audibly conveying a warning message to a user based on the analysis results.

[2060] "Means for analyzing and monitoring emotional state" refers to technology that analyzes a user's voice, facial expressions, etc. in real time to estimate their emotions and psychological state.

[2061] "Means for adjusting notification content" refers to techniques or methods for changing the intensity of the notification content or the content of the message depending on the emotional state of the user.

[2062] A "virtual store" refers to a shopping environment or platform built on the Internet, rather than a physical store.

[2063] This system is a fraud prevention system aimed at the elderly and those with limited information, and operates primarily on mobile devices such as smartphones and tablets. The system consists of the following main components: voice registration, call and email analysis, emotion engine, warning notification, countermeasure recommendation, and model update. This system can also be applied to fraud prevention in virtual stores.

[2064] Voice registration function

[2065] When a user installs and launches the app, they are first provided with a means to record and register the voices of their family and relatives. The user records the voices of each family member and relative, and the device sends the recordings to the server. The server then analyzes the voice data using a generative AI model, extracts features, and stores them. This voice registration function enables speaker recognition during subsequent calls and email analysis.

[2066] Call and email analysis function

[2067] When the device receives a call, the voice data is sent to the server in real time. The server analyzes the received voice data using a generative AI model and compares it with recorded voices of family members or relatives. Similarly, the contents of emails received by the device are sent to the server and analyzed using the generative AI model. The server then determines whether there is any suspicion of fraud.

[2068] Emotion engine function

[2069] The device recognizes the user's emotions in real time and sends the data to the server. The emotion engine uses technologies such as facial recognition and voice tone analysis to determine the user's current emotional state. For example, if the user shows an anxious facial expression or an anxious voice tone, that information is sent to the server.

[2070] Warning notification function

[2071] If the server determines that a received call or email is suspicious and the user's emotions are unstable, it sends the result of the determination to the device. The device then displays a warning to the user saying, "This call / email may be suspicious. Please check." The intensity and type of this warning notification are adjusted by the emotion engine according to the user's emotional state.

[2072] Countermeasure recommendation function

[2073] At the same time as issuing the warning notification, the device will suggest countermeasures to the user. The device will display a message saying, "This call / email may be suspicious. Please take action from the options below." The user can choose from options such as "Check with a family member" or "Contact the nearest public institution." The device will also display response procedures based on the user's emotional state using an emotion engine. For example, if the user is feeling anxious, a message will be displayed urging them to remain calm.

[2074] Model update function

[2075] The server periodically updates the generative AI model and learns about new fraud techniques. The server sends the update information to the device, which then notifies the user. When the user updates the app, the latest generative AI model is installed.

[2076] Examples and prompts

[2077] Example

[2078] Example of a call: When a user receives a call from someone claiming to be their "grandchild," the device analyzes the call in real time and sends the voice data to the server. The server uses a generative AI model to analyze the voice data and determines that it does not match the voice of the registered "grandchild" or that the content is suspicious. The server sends the results of the determination to the device, and the emotion engine also recognizes the user's state of anxiety. The device displays a strong warning to the user saying, "This call may be suspicious. Please check." The user sees the warning, ends the call, and contacts the actual family member to confirm the call.

[2079] Example of email: When a user receives a fraudulent email purporting to be from a different sender, the device sends the received email to a server, which analyzes the email content using a generative AI model. If it determines that the email is suspected to be fraudulent, the server sends the result of the judgment and the user's emotional data to the device. The device then displays a warning to the user saying, "This email may be fraudulent. Please delete it." If the emotion engine detects that the user is impatient, an additional message will be displayed saying, "Please remain calm and respond accordingly." The user will then see the warning and delete the email, avoiding opening any suspicious links or attachments.

[2080] Prompt Sentence Examples

[2081] "Analyze the given audio data for known voice features and determine if it matches any registered voices. If a match is not found or the user appears anxious, send a cautionary notification to the user's device."

[2082] "Analyze the given message data to detect any fraudulent content using the AI ​​model. If a fraud is detected, notify the user with a warning message."

[2083] This will prevent fraud damage before it occurs, and the system can be implemented to effectively protect the elderly and those with limited information from fraud in particular.

[2084] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2085] Step 1:

[2086] (Audio recording and registration)

[2087] When a user installs and launches the app, the server provides the user with a screen for recording and registering the voices of their family and relatives.

[2088] Input: User-recorded voice data of family and relatives

[2089] Processing: The device sends the recorded voice data to the server, which uses a generative AI model to extract voice features.

[2090] Output: Family and relatives' characteristics data stored on the server

[2091] How it works: The server analyzes the feature data and stores it in a database as a registered voice profile.

[2092] Step 2:

[2093] (Real-time analysis of call data)

[2094] When the terminal receives a call, the call data is sent to the server in real time.

[2095] Input: Audio data from the received call

[2096] Processing: The server uses the generative AI model to analyze the voice data and match it with the registered voice profile.

[2097] Output: Analysis results (whether or not there is suspicion of fraud)

[2098] Operation: The server performs real-time matching and sends the results to the device.

[2099] Step 3:

[2100] (Email data analysis)

[2101] The terminal sends the contents of the received email to the server.

[2102] Input: Received email data

[2103] Processing: The server uses the generative AI model to analyze the text of the email and determine whether it is likely to be fraudulent.

[2104] Output: Analysis results (whether or not there is suspicion of fraud)

[2105] How it works: Based on the analysis results, the server determines whether the email is likely to be fraudulent and sends the result to the device.

[2106] Step 4:

[2107] (Monitoring emotional state)

[2108] The terminal monitors the user's emotional state.

[2109] Input: User's voice tone and facial expression data

[2110] Processing: The server uses the emotion engine to analyze the user's emotional state.

[2111] Output: Emotional state judgment result

[2112] Operation: The server analyzes the user's emotional state and sends the analysis results to the device.

[2113] Step 5:

[2114] (Display warning notification)

[2115] The server displays a warning notification to the user based on the analysis of calls and emails and the user's emotional state.

[2116] Input: Analysis results of calls and emails, emotional state determination results

[2117] Processing: The server generates the warning content and sends it to the terminal.

[2118] Output: A warning notice that is displayed to the user

[2119] What it does: The device displays a warning notification to the user saying "This call / email may be suspicious."

[2120] Step 6:

[2121] (Recommended measures)

[2122] The device recommends countermeasures to the user.

[2123] Input: Warning notification content

[2124] Processing: The server generates appropriate countermeasures and sends them to the terminal.

[2125] Output: Recommended measures

[2126] What it does: The device prompts the user with options such as "Check with a family member" or "Contact the nearest public agency."

[2127] Step 7:

[2128] (Regular model updates)

[2129] The server periodically updates the generative AI model.

[2130] Input: Latest scam information

[2131] Processing: The server trains the generated AI model on new fraud techniques.

[2132] Output: Updated generative AI model

[2133] How it works: The server sends update information to the device and notifies the user. When the user updates the app, the latest generative AI model is installed.

[2134] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2135] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2136] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2137] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2138] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotio...

Claims

1. A fraud prevention system aimed at the elderly and those with limited information, A means for a user to record and register the voices of family members and relatives; A means of analyzing received call and email data using generative AI models, A means for displaying a warning notice to the user if the received call or email is deemed suspicious based on the analysis results; A system including:

2. The system includes a means for users to record and register the voices of their family and relatives, a means for analyzing received call and email data using a generative AI model, and a means for displaying a warning notice to the user if the received call or email is deemed suspicious based on the analysis results.

10. The system of claim 1, further comprising means for recommending to the user to contact family members or public institutions.

3. The system includes a means for users to record and register the voices of their family and relatives, a means for analyzing received call and email data using a generative AI model, and a means for displaying a warning notice to the user if the received call or email is deemed suspicious based on the analysis results.

10. The system of claim 1, further comprising means for periodically updating the generative AI model.

Images