System

Abstract

A system is provided.SOLUTION: This system includes a generation AI means for analyzing contents having the possibility of a scam, a means for preparing an alert and notifying a user in the case of judging that the possibility of the scam is high, and a means for updating a generation AI in order to cope with a new scam method.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] A wide variety of special frauds that occur via email, social media, and telephone have become a social problem. In particular, people with low IT literacy and those who tend to panic when contacted suddenly are often victims of fraud. In order to prevent such fraud, a system is needed that reduces contact with fraud and encourages calm judgment. The purpose of this invention is to solve these problems and reduce the number of fraud victims. [Means for solving the problem]

[0005] The present invention is a system that includes a generation AI means for analyzing content that may be fraudulent, a means for creating an alert and notifying the user if it determines that there is a high possibility of fraud, and a means for updating the generation AI to respond to new fraudulent methods. The system also includes a means for the generation AI to respond to calls from withheld or unknown phone numbers with an automated voice, record and analyze the content of the call, and notify the user if it determines that there is a high possibility of fraud. The system also includes a means for the generation AI to analyze the content of emails or social media messages and, if it determines that there is a possibility of fraud, create an alert for the message and notify the user. This reduces users' exposure to fraud and provides an environment that makes it easier for them to make calm decisions.

[0006] A "generative AI means" is an artificial intelligence means that analyzes potentially fraudulent content and makes a judgment based on the results.

[0007] The "means for creating an alert and notifying the user" refers to the means for creating a warning message and notifying the user when the generating AI means determines that there is a high possibility of fraud.

[0008] "Means for updating the generative AI to respond to new fraudulent methods" refers to means for regularly updating the training data and models in order to make the generative AI means compatible with the latest fraudulent methods.

[0009] "Means for responding with an automated voice to calls from anonymous or unknown phone numbers" refers to a means for the AI ​​generating means to automatically respond with a voice to the caller when a call is received from anonymous or unfamiliar phone numbers.

[0010] "Means for recording and analyzing call content" refers to a means for recording the call content after an automated voice response and for the generated AI means to analyze the recorded content.

[0011] "Means for creating and notifying an alert when it is determined that there is a possibility of fraud" refers to means for creating a warning message and notifying the user when, as a result of analyzing the contents of a call or message, it is determined that there is a high possibility of fraud.

[0012] "Means for analyzing the content of emails or social media messages" means generative AI means for analyzing the content of received emails or social media messages to detect possible fraud. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone, and notifies users with appropriate alerts. Specific embodiments of this system are described in detail below.

[0035] Email and SNS fraud detection function

[0036] System Configuration

[0037] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[0038] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[0039] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[0040] Program processing overview

[0041] 1. The server receives an email or SNS message.

[0042] 2. The generated AI means on the server analyzes the content.

[0043] 3. Detect potentially fraudulent claims and unreliable URLs.

[0044] 4. Generate a warning message and notify the user by means of creating an alert and notifying the user.

[0045] Specific examples

[0046] For example, if the server receives an email with the content "Please transfer the money to my bank account immediately," the AI ​​generator will detect the phrase, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[0047] Automated telephone answering function

[0048] System Configuration

[0049] Device: Accept calls from blocked or unknown numbers.

[0050] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[0051] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[0052] Program processing overview

[0053] 1. Your device receives a call from a blocked or unknown number.

[0054] 2. The server initiates an automated response using generated AI means.

[0055] 3. The server records the call, and the generating AI means analyzes the recording.

[0056] 4. A warning message is generated by the means for creating an alert and notifying the user, and the user is notified along with the recorded content.

[0057] Specific examples

[0058] For example, when a call comes in from an unidentified number to a device, the server uses AI generation to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is then analyzed, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and is notified to the user along with the recorded data. This allows the user to respond calmly.

[0059] Responding to new fraud methods

[0060] System Configuration

[0061] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[0062] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[0063] Program processing overview

[0064] 1. The server collects information about new fraud methods.

[0065] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[0066] 3. The generative AI method analyzes the updated model for potential fraud.

[0067] Specific examples

[0068] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the AI ​​generation means as learning data. This allows the AI ​​generation means to respond to the latest fraud techniques and provide accurate alerts to users.

[0069] In this way, it is possible to provide a system that reduces the risk of fraud and protects users from fraud.

[0070] The processing flow will be explained below.

[0071] Email and SNS fraud detection function

[0072] Processing Steps

[0073] Step 1:

[0074] The server receives an email or SNS message and stores the message content and metadata (sender, subject, body) in a database.

[0075] Step 2:

[0076] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[0077] Step 3:

[0078] The extracted information is compared with existing databases to detect potentially fraudulent phrases and unreliable URLs, such as keywords like "transfer" or "urgent," or suspicious domains.

[0079] Step 4:

[0080] Scoring the likelihood of fraud: The generative AI method determines the likelihood of fraud based on the extracted information and assigns a score.

[0081] Step 5:

[0082] If the method for creating an alert and notifying the user determines that the email is likely to be fraudulent, a warning message will be generated. Specifically, an alert message such as "This email may be fraudulent" will be generated.

[0083] Step 6:

[0084] The notification means notifies the user of the alert. Possible notification methods include push notifications and emails.

[0085] Automated telephone answering function

[0086] Processing Steps

[0087] Step 1:

[0088] The device receives a call from a blocked or unknown phone number and sends the received phone number information to the server.

[0089] Step 2:

[0090] The server starts an automated voice response using the generation AI means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[0091] Step 3:

[0092] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[0093] Step 4:

[0094] The generative AI method analyzes the content of the call, checking for the presence of specific keywords and phrases, and scoring the likelihood of fraud. For example, keywords such as "transfer money" and "give me your personal information" indicate a potential fraud.

[0095] Step 5:

[0096] If the means for creating an alert and notifying the user determines that there is a high possibility of fraud, a warning message will be generated. An alert message such as "Possible fraud" will be generated along with the recorded content.

[0097] Step 6:

[0098] The notification means notifies the user of the alert and the recorded content, allowing the user to respond calmly.

[0099] Responding to new fraud methods

[0100] Processing Steps

[0101] Step 1:

[0102] The server collects information about new fraud methods, possibly from police reports or security blogs.

[0103] Step 2:

[0104] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training dataset and retraining the generative AI model.

[0105] Step 3:

[0106] The server deploys the updated generative AI method, updating the system-wide AI model to respond to new fraudulent techniques.

[0107] Step 4:

[0108] The generative AI method uses the updated model to analyze potential fraud, providing analysis capabilities that are in line with new fraud methods, making it possible to respond to the latest fraud techniques.

[0109] In this way, the present invention aims to enable users to avoid the risk of fraud in advance and provide a safe communication environment.

[0110] Example 1

[0111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] Fraudulent activities are on the rise in communications and phone calls over the Internet, increasing the risk of many users becoming victims. However, because it is difficult to sufficiently reduce these risks using conventional methods, there is a need for technology that can detect potential fraud with high accuracy and notify users promptly.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0114] In this invention, the server includes means for receiving emails and messages from social networking services, means for analyzing the content with a generation AI means, means for detecting potentially fraudulent wording and unreliable URLs, means for creating an alert and notifying the user, means for receiving calls from withheld or unknown phone numbers and responding with an automated voice with the generation AI means, recording and analyzing the content of the call, and means for collecting information on new fraud methods and updating the generation AI means. This makes it possible to detect possible fraud with high accuracy and notify the user promptly.

[0115] A "server" is a central processing unit that receives, analyzes, stores, and notifies users of emails and messages from social networking services.

[0116] "Generative AI means" refers to an artificial intelligence model and its processing means for analyzing received content and determining the possibility of fraud.

[0117] "Notification means" refers to a device or method that creates an alert message and sends a warning to the user by email, SMS, push notification, or other means.

[0118] The "message receiving means" is a function that receives messages from emails and social networking services and transmits them to the server.

[0119] "Analysis means" refers to the process of analyzing received messages and call content using the generation AI means.

[0120] "Detection means" is a function that extracts potentially fraudulent wording and unreliable URLs from the analyzed content.

[0121] An "alert generator" is a function that generates a warning message to the user when a possible fraud is detected.

[0122] "Call answering means" is a function in which the server uses AI generation means to respond with an automated voice to calls from anonymous or unknown phone numbers.

[0123] The "call recording means" is a function that records the contents of a call in real time and saves the recorded data.

[0124] "Information gathering means" refers to the function of gathering information about new fraud methods from the Internet and specialized institutions.

[0125] The "update method" is a function that reflects collected information on new fraud methods and retrains the generation AI method.

[0126] The present invention is a system that utilizes generative AI to detect possible fraud occurring through email, messages on social networking services, and telephone calls, and notifies users with appropriate alerts. Specific embodiments for implementing the present invention are described in detail below.

[0127] Email and SNS fraud detection function

[0128] System Configuration

[0129] This system is configured as follows:

[0130] Server: Receives emails and messages from social networking services and analyzes their contents using generative AI.

[0131] Generative AI means: Analyzes the content of received messages to detect potential fraud, using generative AI models such as GPT-3.

[0132] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[0133] operation

[0134] The server receives messages from email or social networking services (e.g., Gmail, Facebook). After receiving the messages, it uses a generative AI method (e.g., GPT-3) to analyze the message content. The generative AI analyzes the message's context and keywords based on the prompt text and assesses the likelihood of fraud.

[0135] For example, if an email with the content "Please transfer the money to my bank account urgently" arrives at the server, the AI ​​generator will detect the wording, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[0136] Prompt Sentence Examples

[0137] Analyze whether this email is a scam: "Please transfer the money to my bank account immediately."

[0138] Automated telephone answering function

[0139] System Configuration

[0140] Device: Accept calls from blocked or unknown numbers.

[0141] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[0142] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[0143] operation

[0144] When the device receives a call from a blocked or unknown phone number, the server uses a generation AI method to respond with an automated voice. For example, it may respond with, "This call is being recorded. Please explain your purpose." The call is then recorded and analyzed by the generation AI method. If the call contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and the user is notified along with the recorded data. This allows the user to respond calmly.

[0145] Prompt Sentence Examples

[0146] Analyze whether this call is a scam: "This call is being recorded. Please explain your purpose."

[0147] Responding to new fraud methods

[0148] System Configuration

[0149] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[0150] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[0151] operation

[0152] The server periodically collects information on the latest fraud techniques. Based on information from the internet and specialized institutions, the generation AI means is retrained and updated. For example, if a new "QR code fraud" is discovered, that information is collected and reflected in the generation AI means as learning data. This allows the generation AI means to respond to the latest fraud techniques and provide accurate alerts to users.

[0153] Prompt Sentence Examples

[0154] Analyze information about new fraud methods and update your AI models to respond: "QR code fraud has been discovered."

[0155] In this way, it is possible to detect possible fraud with high accuracy and notify the user promptly, thereby providing a system that can reduce the risk of fraud and ensure the safety of users.

[0156] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0157] Email and SNS fraud detection function

[0158] Step 1:

[0159] The server receives messages from email or social networking services. The server retrieves message data from external messaging services using a specified API or protocol (e.g., IMAP, Graph API). The input to this process is the message data received from email or social networking services, and the output is the storage of the received messages.

[0160] Step 2:

[0161] The server analyzes the content using generative AI means. The server inputs the received message into a generative AI model (e.g., GPT-3). It generates a prompt (e.g., "Please analyze whether this email is fraudulent: 'Please transfer the money to my bank account quickly.'") and passes it to the generative AI model. The input to this process is the content of the received message, and the output is the generative AI model's assessment of the likelihood of fraud.

[0162] Step 3:

[0163] The server detects potentially fraudulent text and unreliable URLs. Based on the analysis results from the generative AI model, the server detects specific keywords and phrases (e.g., "transfer" or "bank account"). It also checks the reliability of URLs and whether they are blacklisted. The input to this process is the evaluation result of the generative AI model, and the output is a list of text and URLs that are deemed to be potentially fraudulent.

[0164] Step 4:

[0165] The server creates an alert and notifies the user via a notification mechanism. If the server determines that there is a high possibility of fraud, it generates a warning message. It sends the alert to the user via a notification mechanism (email, SMS, push notification, etc.). The input to this process is the data that has been determined to be potentially fraudulent, and the output is a warning message that is sent to the user.

[0166] Automated telephone answering function

[0167] Step 1:

[0168] The terminal receives an incoming call from a blocked or unknown phone number. When the terminal detects the incoming call, it sends the information to the server. The input of this process is the incoming call notification, and the output is the incoming call information sent to the server.

[0169] Step 2:

[0170] The server uses the generation AI means to initiate an automatic response. The server uses the generation AI means to respond with an automated voice saying, "This call is being recorded. Please explain your purpose." The input of this process is the incoming call information, and the output is an automated voice message.

[0171] Step 3:

[0172] The server records the call content, and the generative AI means analyzes the recording. The server records the call content in real time and inputs the recording data into the generative AI model. The generative AI model analyzes specific keywords and phrases and evaluates the likelihood of fraud. The input to this process is the recorded call content, and the output is the evaluation result obtained from the generative AI model.

[0173] Step 4:

[0174] The server creates an alert and notifies the user via a notification method. If it determines that there is a high possibility of fraud, the server creates a warning message and notifies the user along with the recorded data. The input to this process is the evaluation result of the generative AI model, and the output is the warning message and recorded data sent to the user.

[0175] Responding to new fraud methods

[0176] Step 1:

[0177] The server collects information about new fraud methods. The server periodically collects information about the latest fraud methods from the Internet and specialized organizations. The input of this process is information collected from outside, and the output is information about fraud methods stored in an internal database.

[0178] Step 2:

[0179] The server updates the generative AI means. Based on the collected information, the server updates the learning dataset of the generative AI means and retrains it. The input of this process is the collected information on fraudulent techniques, and the output is an updated generative AI model.

[0180] Step 3:

[0181] The server performs the analysis using the updated generative AI model. The server then re-analyzes the email and call content using the updated AI model. The input to this process is the data to be analyzed based on the new fraud technique, and the output is the latest evaluation result.

[0182] These specific processing steps enable the system to detect potential fraud with high accuracy and notify the user promptly.

[0183] (Application example 1)

[0184] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0185] There is a problem that it is difficult for users to prevent fraudulent communications from emails, SNS messages, and calls from anonymous or unknown phone numbers. Current technology lacks the means to efficiently detect these fraudulent communications and quickly warn users, which means that users are unable to take appropriate precautions.

[0186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0187] In this invention, the server includes a generating AI means for analyzing content that may be fraudulent, a means for creating an alert and notifying the user if it is determined that there is a high possibility of fraud, a means for updating the generating AI to respond to new fraudulent methods, a means for analyzing emails and SMS messages in real time, a means for automatically answering, recording, and analyzing telephone calls and sending an immediate alert if there is a possibility of fraud, a means for analyzing the content of emails and telephone calls that may be fraudulent based on a generating AI model, and a means for sending notifications in real time. This makes it possible to detect possible fraud occurring through emails, SNS messages, and telephone calls and to send warnings to users quickly and accurately.

[0188] A "server" is a computer system that uses generative AI means to analyze emails, social media messages, and phone calls, detect potential fraud, and respond to new fraud methods.

[0189] "Generative AI means" refers to a function that uses a generative AI model to analyze the content of emails, social media messages, and phone calls to detect possible fraud.

[0190] "Means for creating alerts and notifying users" refers to a function that immediately generates and sends a warning message to users when it is determined that there is a high possibility of fraud.

[0191] "Means for updating" refers to the ability to update the learning data and algorithms of the generative AI model to respond to new fraudulent methods.

[0192] "Means for analyzing emails and SMS messages in real time" refers to a function that instantly analyzes received emails and SMS messages using AI-generated means to determine whether they are fraudulent.

[0193] "Means for automatically answering, recording and analyzing calls, and sending immediate alerts in the event of a possible fraud" refers to a function that automatically answers calls from anonymous or unknown phone numbers, records and analyzes the content of the call, and sends a warning to the user if it is determined that there is a high possibility of fraud.

[0194] "Generative AI Model" means a machine learning algorithm utilized in a generative AI method, and is a trained model used to detect potential fraud.

[0195] A "prompt sentence" is text data input to a generative AI model, and is the sentence that is analyzed to determine the possibility of fraud.

[0196] This invention is a system that uses generative AI to detect potential fraudulent activity occurring through email, social media messages, and phone calls, and quickly and accurately notifies users of the alert. Specific embodiments of the invention are described in detail below.

[0197] System Configuration

[0198] The system consists of the following major components:

[0199] 1. Server: Analyzes emails, social media messages, and phone calls using generative AI methods to detect potential fraud. Also, updates the generative AI model to adapt to new fraud methods.

[0200] 2. Generative AI methods: These are machine learning models that analyze the content of emails, social media messages, and phone calls to determine the likelihood of fraud. Generative AI methods are implemented using the OpenAI API, among other things.

[0201] 3. Notification method: The method by which the user will be alerted, which may include SMS, email, or phone notification.

[0202] 4. Automated Telephone Answering System: Uses the Twilio API to answer calls from anonymous or unknown phone numbers.

[0203] Email and SNS message analysis

[0204] The server analyzes received emails and SNS messages using the AI ​​generation means. For example, if the email content contains a phrase such as "Please transfer the money to my bank account immediately," the AI ​​generation means will determine that it is "possibly fraudulent," and an alert will be immediately created to notify the user. This will allow the user to be on guard. An example of a prompt sentence is as follows: "Do you think the following message is potentially fraudulent?" (followed by the specific email content).

[0205] Automated call answering and analysis

[0206] When the server receives a call from an unidentified or unknown phone number, it uses the Twilio API to initiate an automatic response. For example, a message saying "This call is being recorded, please state your purpose" is played, and then the call is recorded. A generating AI method analyzes the recording, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating "Possible fraud" and notifying the user. An example of a prompt sentence is as follows: "Do you think the following call content is likely to be fraudulent?" (followed by the specific recording content).

[0207] Responding to new fraud methods

[0208] The server collects information about new fraud methods and periodically updates the generative AI model. For example, if a new "QR code fraud" is discovered, that information is reflected in the generative AI model to improve its response capabilities. This allows users to be alerted to the latest fraud methods.

[0209] Hardware and software used

[0210] Specifically, the server uses a high-performance computer system, the AI ​​generation method uses OpenAI's API, the telephone answering system incorporates Twilio's API, and notification methods use common SMS gateways and email servers.

[0211] As a result, this system can efficiently detect potential fraudulent activity via email, telephone, or social media, and quickly send warnings to users, significantly reducing the risk of becoming a victim of fraud.

[0212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0213] Step 1:

[0214] The server receives emails and SMS messages.

[0215] Input: The content of emails and SMS messages that are sent to users.

[0216] Processing: The server stores the received emails and SMS messages and prepares them for analysis.

[0217] Output: The message data to be parsed.

[0218] Step 2:

[0219] The server analyzes the message content using generated AI methods.

[0220] Input: The message data obtained in step 1.

[0221] Processing: A prompt is input into the generative AI model to evaluate the likelihood of fraud. The specific message content is input into the generative AI model along with the prompt, "Is the following message likely to be fraudulent?"

[0222] Output: Assessment results for likelihood of fraud.

[0223] Step 3:

[0224] Determine if the server is likely to be fraudulent.

[0225] Input: Evaluation results obtained in step 2.

[0226] Processing: If the evaluation results in a high probability of fraud, the server creates an alert.

[0227] Output: Fraud warning message.

[0228] Step 4:

[0229] The server notifies the user with a warning message.

[0230] Input: The fraud warning message created in step 3.

[0231] Processing: Use an SMS gateway or mail server to send a warning message to the user's device.

[0232] Output: A warning message that will be displayed on the user's terminal.

[0233] Step 5:

[0234] The server receives a call from a blocked or unknown phone number.

[0235] Input: Incoming call signal from a blocked or unknown number.

[0236] Processing: Detects an incoming call signal and initiates an automatic response.

[0237] Output: Command to start the auto-reply.

[0238] Step 6:

[0239] The server records the call and analyzes it using generative AI.

[0240] Input: The call content obtained in step 5.

[0241] Processing: The call is recorded using the Twilio API and the recording is fed into the generative AI model along with a prompt: "Do you think the following call is likely to be fraudulent?"

[0242] Output: Assessment results for likelihood of fraud.

[0243] Step 7:

[0244] The server issues a warning to the user if there is a high possibility of fraud.

[0245] Input: Evaluation results obtained in step 6.

[0246] Processing: If a fraudulent activity is deemed likely, a warning message is sent to the user via an SMS gateway or calling system.

[0247] Output: The warning message and audio recording that will be displayed on the user's device.

[0248] Step 8:

[0249] The server collects information about new fraud methods and updates the generative AI model.

[0250] Input: Information on new fraud methods.

[0251] Processing: Retraining the generative AI model based on collected information to improve fraud detection accuracy.

[0252] Output: An updated generative AI model.

[0253] As a result, the system can detect potential fraud in real time and provide appropriate warnings to users.

[0254] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0255] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone calls, and notifies the user with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state. The following describes specific embodiments of the present invention.

[0256] Email and SNS fraud detection function

[0257] System Configuration

[0258] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[0259] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[0260] Emotion engine: Analyzes the user's emotional state and optimizes notification content based on that information.

[0261] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[0262] Program processing overview

[0263] 1. The server receives an email or SNS message.

[0264] 2. Generative AI methods on the server analyze the content and detect possible fraud.

[0265] 3. The emotion engine analyzes the user's emotional state in real time.

[0266] 4. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[0267] 5. The notification means notifies the user of the warning message.

[0268] Specific examples

[0269] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the generation AI means will detect the message and create an alert saying "This may be a scam." The emotion engine will analyze the user's emotional state, and if the user is nervous, it will add a message to the notification such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[0270] Automated telephone answering function

[0271] System Configuration

[0272] Device: Accept calls from blocked or unknown numbers.

[0273] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[0274] Emotion Engine: Analyzes the user's emotional state and optimizes notification content.

[0275] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[0276] Program processing overview

[0277] 1. Your device receives a call from a blocked or unknown number.

[0278] 2. The server initiates an automated response using generated AI means.

[0279] 3. The server records the call, and the generating AI means analyzes the recording.

[0280] 4. The emotion engine analyzes the user's emotional state in real time.

[0281] 5. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[0282] 6. The notification means notifies the user of the warning message and the recorded content.

[0283] Specific examples

[0284] For example, when a device receives a call from an unidentified number, the server uses a generative AI method to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion engine analyzes the user's emotional state and detects that the user is in a tense state. In this case, the server notifies the user by adding a message saying, "Please remain calm." This allows the user to respond calmly.

[0285] Responding to new fraud methods

[0286] System Configuration

[0287] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[0288] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[0289] Emotion engine: Analyzes the user's emotional state in real time and optimizes notification content based on that information.

[0290] Program processing overview

[0291] 1. The server collects information about new fraud methods.

[0292] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[0293] 3. The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud techniques.

[0294] 4. The generative AI method uses the updated model to analyze potential fraud.

[0295] Specific examples

[0296] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it as learning data in the generation AI. Furthermore, the emotion engine analyzes the user's emotional state, allowing it to provide appropriate responses in real time. For example, if a user is nervous about a new fraudulent technique, it will provide a message urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[0297] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[0298] The processing flow will be explained below.

[0299] Email and SNS fraud detection function

[0300] Processing Steps

[0301] Step 1:

[0302] The server receives an email or SNS message and stores the received content and metadata (sender, subject, body) in a database.

[0303] Step 2:

[0304] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[0305] Step 3:

[0306] The generative AI method compares the extracted information with existing databases to detect potentially fraudulent phrases and unreliable URLs, particularly those containing keywords that indicate potential fraud, such as "transfer" or "urgent," as well as suspicious domains.

[0307] Step 4:

[0308] Generative AI methods score the likelihood of fraud, quantifying fraud risk based on extracted information and flagging high risk cases.

[0309] Step 5:

[0310] The emotion engine analyzes the user's emotional state in real time, for example by analyzing the user's facial expressions and voice to detect tension or anxiety.

[0311] Step 6:

[0312] If a fraudulent activity is deemed likely, the emotion engine will create a warning message based on the user's emotional state. For users in a tense state, a message such as "Stay calm and don't react immediately" will be added.

[0313] Step 7:

[0314] The notification method notifies the user of the created alert, sending a warning message via push notification or email to alert the user.

[0315] Automated telephone answering function

[0316] Processing Steps

[0317] Step 1:

[0318] The device receives a call from a blocked or unknown phone number and sends the received call information to the server.

[0319] Step 2:

[0320] The server starts an automated voice response using the AI ​​generation means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[0321] Step 3:

[0322] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[0323] Step 4:

[0324] The generative AI method analyzes the content of the call, checking for the presence of certain keywords and phrases and assessing the likelihood of fraud. For example, phrases such as "transfer money" and "give me your personal information" are recognized as indicators of fraud.

[0325] Step 5:

[0326] The emotion engine analyzes the user's emotional state in real time, analyzing the user's tone of voice and choice of words during a call to detect tension or anxiety.

[0327] Step 6:

[0328] If a fraudulent activity is deemed likely, the emotion engine will create a warning message for users in a state of anxiety, for example adding a message saying "Please stay calm."

[0329] Step 7:

[0330] A notification mechanism will notify the user of the generated alert and recording, and in particularly high-risk cases, notifications will also be sent to the user's next of kin or trusted third parties.

[0331] Responding to new fraud methods

[0332] Processing Steps

[0333] Step 1:

[0334] The server collects information about new fraud methods, deriving data from sources such as police reports and security blogs.

[0335] Step 2:

[0336] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training data and retraining the generative AI model.

[0337] Step 3:

[0338] Deploy new models with updated generative AI methods to update the system to address the latest fraud techniques.

[0339] Step 4:

[0340] The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address new fraud techniques, helping users stay calm even when faced with new fraud methods.

[0341] Step 5:

[0342] The generative AI method uses the updated model to analyze new communications from users, providing analysis capabilities tailored to new fraud techniques and generating appropriate alerts for users.

[0343] Example 2

[0344] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0345] In recent years, fraudulent activities using email, social media, and telephone have been increasing, and their methods have become more diverse and sophisticated. Therefore, users need to be able to quickly and reliably identify fraud risks and take appropriate measures. However, conventional fraud detection systems focus on identifying potential fraud and do not provide flexible responses that take into account the user's emotional state. Furthermore, they face the problem of being difficult to update quickly to respond to new fraud methods. Therefore, a method is needed that enables real-time fraud detection and flexible responses while ensuring user safety.

[0346] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0347] In this invention, the server includes means for receiving and analyzing the contents of electronic messages, means for analyzing the contents of electronic messages and detecting the possibility of fraud, means for analyzing the emotional state of the user and optimizing the notification content, means for notifying the user of a warning message when it is determined that there is a high possibility of fraud, and means for updating the generation AI to respond to new fraud methods. This makes it possible to quickly detect the risk of fraud in messages and calls received by the user and provide an appropriate response according to the user's emotional state.

[0348] "Electronic Message" refers to text messages sent or received over the Internet, such as emails or social media messages.

[0349] "Generative AI methods" refer to algorithms or models that use machine learning or deep learning to analyze text or voice data to determine the likelihood of fraud.

[0350] "Emotion analysis means" refers to technology that analyzes the user's emotional state in real time based on facial expressions and voice data, and optimizes the content of notifications.

[0351] A "warning message" refers to a message that is sent to the user to warn them when it is determined that there is a high possibility of fraud.

[0352] "New fraud methods" refers to newly discovered means and techniques for committing fraud, in addition to traditional fraud methods.

[0353] "Means for receiving and analyzing the content of electronic messages" refers to methods and technologies for retrieving electronic messages from mail servers or social media platforms and analyzing their text content.

[0354] "Generative AI methods for detecting potential fraud" refers to methods that use machine learning and deep learning models to analyze electronic messages and phone calls to identify signs and risks of fraud.

[0355] "Means for analyzing the user's emotional state and optimizing the content of notifications" refers to technology that analyzes the user's psychological state and generates an appropriate response message based on the results.

[0356] "Means for notifying users when it is determined that there is a high possibility of fraud" refers to communication technology for conveying a warning to users based on the analysis results of the generating AI means.

[0357] "Means to update generative AI" refers to technology that retrains generative AI models based on new data and technical information, enabling them to respond to the latest fraud techniques.

[0358] This invention is a system that uses generative AI to detect possible fraud via email, social media, and phone calls, and notifies users with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state.

[0359] Email and SNS fraud detection function

[0360] System Configuration

[0361] Server: Receives emails and social media messages and uses the Gmail API and Twitter API to analyze their contents using generative AI.

[0362] Generative AI method: Using OpenAI's GPT-3 and other technologies, the content of received emails and social media messages is analyzed to detect possible fraud.

[0363] Emotion analysis method: Using Microsoft's Azure Emotion API and other tools, the user's emotional state is analyzed in real time and the content of notifications is optimized based on that information.

[0364] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user via smartphone push notifications or email notifications.

[0365] Program processing overview

[0366] The server analyzes received emails and SNS messages. The generation AI means analyzes the content and detects the possibility of fraud. The emotion analysis means analyzes the user's emotional state, and if it determines that there is a high possibility of fraud, it optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user with a warning message.

[0367] Specific examples

[0368] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the AI ​​generation means will detect the message and create an alert saying "This may be a scam." The emotion analysis means will analyze the user's emotional state, and if the user is nervous, it will add a message such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[0369] Prompt Sentence Examples

[0370] When you receive an email saying "Please transfer money to your bank account urgently," rate it as likely to be a scam. Also, if the user is nervous, add a message saying "Please stay calm and don't respond immediately."

[0371] Automated telephone answering function

[0372] System Configuration

[0373] Device: Use a VoIP service connected to your smartphone or landline to receive calls from blocked or unknown numbers.

[0374] Server: Generates automated voice responses using AI and uses voice recognition services such as Google Dialogflow to record and analyze call content.

[0375] Sentiment analysis: Analyze the user's emotional state in real time using the Amazon Polly API, etc.

[0376] Notification method: If a fraudulent activity is deemed likely, a warning message and recording will be sent to the user via push notification or email notification.

[0377] Program processing overview

[0378] The device receives a call from a blocked or unknown phone number. The server initiates an automated voice response and records the call. The recording is analyzed by the generative AI means to assess the possibility of fraud. At the same time, the emotion analysis means analyzes the user's emotional state in real time and optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user of a warning message and the recording content.

[0379] Specific examples

[0380] For example, when a call comes in from an unidentified number to a terminal, the server automatically responds with "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion analysis means analyzes the user's emotional state and detects that the user is nervous. In this case, the system adds a message to the user saying, "Please remain calm." This allows the user to respond calmly.

[0381] Prompt Sentence Examples

[0382] If you receive a call from an unidentified number, check whether the call contains keywords such as "transfer" or "personal information." If the user is nervous, add a message to the call saying, "Please stay calm."

[0383] Responding to new fraud methods

[0384] System Configuration

[0385] Server: Uses web scraping techniques, news sites, and forum data to gather information on the latest fraudulent techniques and update the generative AI methods.

[0386] Generative AI methods: Retraining machine learning models to constantly update learning data and models based on new fraud techniques.

[0387] Sentiment analysis method: We use the sentiment analysis API to analyze the user's emotional state in real time and optimize the notification content.

[0388] Program processing overview

[0389] The server collects information on new fraud methods and updates the AI ​​generation method. The emotion analysis method analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud methods. The AI ​​generation method uses the updated model to analyze the possibility of fraud.

[0390] Specific examples

[0391] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the generation AI means as learning data. Furthermore, the emotion analysis means analyzes the user's emotional state, allowing for appropriate responses to be provided in real time. For example, if a user is nervous about a new fraudulent technique, a message is displayed urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[0392] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[0393] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0394] Email and SNS fraud detection function

[0395] Processing Steps

[0396] Step 1:

[0397] The server receives electronic messages. Specifically, it periodically checks emails and SNS messages using the Gmail API and Twitter API to retrieve new messages.

[0398] Input: New email or social media message

[0399] Output: Received electronic message

[0400] Step 2:

[0401] The server analyzes the content of the received electronic message using a generative AI method. The generative AI method (e.g., OpenAI GPT-3) is used to analyze the message text and assess its likelihood of fraud.

[0402] Input: Received electronic message

[0403] Data processing / computation: Detecting fraud patterns through natural language processing

[0404] Output: Fraud probability assessment result

[0405] Step 3:

[0406] The server analyzes the user's emotional state using emotion analysis tools, such as Microsoft's Azure Emotion API, which analyzes the user's facial expressions and voice in real time to estimate their emotional state.

[0407] Input: User's facial expression and voice data

[0408] Data processing / computation: Classifying emotional states using emotion recognition models

[0409] Output: User's emotional state

[0410] Step 4:

[0411] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[0412] Input: Fraud probability assessment, user emotional state

[0413] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[0414] Output: personalized warning message

[0415] Step 5:

[0416] The server notifies the user of the warning message. Appropriate warnings are sent to the user via smartphone push notifications or email.

[0417] Input: personalized warning message

[0418] Output: Message notified to the user

[0419] Automated telephone answering function

[0420] Processing Steps

[0421] Step 1:

[0422] Your device receives calls from blocked or unknown numbers. Use a VoIP service connected to your smartphone or landline to monitor whether there are any incoming calls.

[0423] Input: Calls from blocked or unknown numbers

[0424] Output: Incoming call notification

[0425] Step 2:

[0426] The server initiates an automated voice response using a generative AI method, using a speech recognition service such as Google Dialogflow to play a message such as "This call is being recorded, please state your purpose."

[0427] Input: Incoming call notification

[0428] Output: Playback of automated voice message

[0429] Step 3:

[0430] The server records the call, and the AI ​​generator analyzes the recording. The recorded voice data is converted into text using an NLP service and analyzed.

[0431] Input: Recorded call

[0432] Data processing / computation: speech-to-text conversion and text analysis

[0433] Output: Fraud probability assessment result

[0434] Step 4:

[0435] The server analyzes the user's emotional state using emotion analysis tools. It uses the Amazon Polly API to analyze the voice data during the call and evaluates the user's emotional state in real time.

[0436] Input: User's voice data

[0437] Data processing / calculation: Analyzing emotional states from voice data

[0438] Output: User's emotional state

[0439] Step 5:

[0440] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[0441] Input: Fraud probability assessment, user emotional state

[0442] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[0443] Output: personalized warning message

[0444] Step 6:

[0445] The server notifies the user of the warning message and recording contents, and sends appropriate warnings to the user via push notification or email.

[0446] Input: personalized warning message, recording

[0447] Output: Message and recording notified to the user

[0448] Responding to new fraud methods

[0449] Processing Steps

[0450] Step 1:

[0451] The server collects information about new fraud methods, using web scraping technology to retrieve the latest information from security news sites and forums.

[0452] Input: Information about a new fraud scheme

[0453] Output: Collected fraud data

[0454] Step 2:

[0455] The server updates the generative AI method and retrains the generative AI model to adapt to new fraud methods based on the collected information.

[0456] Input: Data on new fraud methods

[0457] Data processing / computation: Retraining AI models

[0458] Output: Updated generative AI model

[0459] Step 3:

[0460] The server uses emotion analysis to analyze the user's emotional state in real time, optimizing alerts to address the latest fraud techniques.

[0461] Input: User emotional state data

[0462] Data processing / calculation: Optimizing notification messages based on emotion data

[0463] Output: Optimized warning message

[0464] Step 4:

[0465] The server uses the updated generative AI model to analyze the possibility of fraud. The latest model analyzes received messages and call content to respond to new fraud methods.

[0466] Input: Received messages and calls

[0467] Data manipulation / calculation: Analysis based on new fraud methods

[0468] Output: Fraud probability assessment result

[0469] These processing steps allow users to proactively avoid the risk of fraud and receive the most appropriate response based on their emotional state.

[0470] (Application example 2)

[0471] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0472] In recent years, fraudulent activities have evolved on a daily basis, requiring immediate and reliable responses to prevent actual damage. However, no systems exist that take into account the user's emotional state, and fraud detection and responses are often inappropriate. In particular, in physical stores, it is difficult for staff to detect fraud while interacting with customers, and it is difficult to make calm decisions when the user is in a tense state. To solve these issues, a system is needed that utilizes wearable devices such as smart glasses to detect possible fraud in real time and provide optimal responses based on the user's emotions.

[0473] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means for analyzing possible fraudulent content, a means for creating an alert and notifying the user when it is determined that fraud is highly likely, a means for updating the generation AI to respond to new fraudulent methods, a means for analyzing the user's emotional state using an emotion engine and optimizing the notification content, and a means for displaying the alert on the smart glasses. This enables fraudulent acts in physical stores to be detected in real time and an optimal response based on emotions. Specifically, when staff are wearing smart glasses, the smart glasses analyze customer behavior and conversation content and display an effective warning message when possible fraud is detected. In addition, instructions are provided according to the staff's emotional state, allowing for a calm and prompt response.

[0474] "Generative AI" is a system that uses artificial intelligence techniques to analyze data and detect specific patterns and anomalies.

[0475] An "emotion engine" is a technology that analyzes a user's emotional state from their voice and images, and is a system that grasps emotions such as stress and tension in real time.

[0476] "Notification means" refers to a system or device that conveys warnings or instructions to users based on information detected by the generative AI or emotion engine.

[0477] "Update methods" are functions that allow systems and models to be updated with the latest data and methods, keeping them up to date at all times.

[0478] "Smart glasses" are wearable devices equipped with a display and communication functions, allowing users to visually receive and integrate information.

[0479] A "server" is a computer system that receives, processes, and transmits data over a network, and is used to manage and operate multiple functions in an integrated manner.

[0480] "Fraud detection" is the process of determining whether certain actions or words are potentially fraudulent, and is performed using generative AI.

[0481] The present invention is a system for detecting possible fraudulent activity in a brick-and-mortar store and providing an optimal response based on the emotional state of the user. Specific embodiments for carrying out the present invention will be described below.

[0482] System Configuration

[0483] 1. Hardware Configuration

[0484] Smart glasses: Equipped with a display, camera, and microphone, worn by staff.

[0485] Server: A computer with high-performance data processing capabilities, primarily responsible for data analysis and running generative AI models.

[0486] Communication method: Uses 5G networks that enable high-speed data communication.

[0487] 2. Software Configuration

[0488] Generative AI models, such as GPT-4 and BERT, can be used to detect potential fraud from audio and video data.

[0489] Emotion Engine: Analyzes staff emotional states in real time using Affectiva and Emotion AI.

[0490] Notification system: Software that notifies staff in real time of warnings and appropriate responses.

[0491] Program processing

[0492] The server receives the video and audio data sent from the smart glasses and analyzes it using a generative AI model. Specifically, it determines whether the customer's behavior or comments indicate the possibility of fraud. If the analyzed data indicates the possibility of fraud, the emotion engine analyzes the staff member's current emotional state. For example, if a staff member is nervous, the notification system will display a message on the smart glasses' display such as, "Please stay calm and treat the customer kindly."

[0493] Specific usage scenarios

[0494] Consider a scenario where a customer visits a brick-and-mortar store and wishes to make a large cash transaction. The smart glasses record the conversation with the customer, and the data is sent to a server. The server's generative AI model analyzes the customer's comments to determine whether they are likely to be fraudulent. For example, if keywords such as "large cash transaction" or "request for personal information" are detected, the emotion engine detects tension in the staff member's heart rate and facial expression. The notification system then displays a message on the smart glasses, such as "This is a possible fraud, so please ask the customer to present identification."

[0495] Prompt Sentence Examples

[0496] Analyze all video and audio recordings containing "large cash transactions" and "requests for personal information" to determine the possibility of fraud.

[0497] This invention enables fast and accurate fraud detection and response in brick-and-mortar stores, ensuring the safety of both customers and staff.

[0498] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0499] Step 1:

[0500] The smart glasses' cameras and microphones capture and record the customer's movements and conversations in real time. The input is the customer's video and audio data. The output is digital data that is sent to the server.

[0501] Step 2:

[0502] The server receives the video and audio data sent from the smart glasses. The input is the video and audio data from the smart glasses. The output is that this data is stored in the server.

[0503] Step 3:

[0504] The generative AI model in the server analyzes the received video and audio data. Specifically, it analyzes the content of the conversation and customer behavior patterns to detect the possibility of fraud. The input is the video and audio data stored in the server. The output is an analysis result indicating the possibility of fraud.

[0505] Step 4:

[0506] The server's emotion engine analyzes the emotional state of staff in real time. This is done by receiving and analyzing biometric information such as the staff's heart rate and facial expressions as input. The input is the staff's biometric information. The output is the analysis result regarding the staff's emotional state.

[0507] Step 5:

[0508] The server combines the analysis results of the generative AI model and the emotion engine and sends them to the notification system. The inputs are the analysis results indicating possible fraud and the analysis results regarding the emotional state of the staff. The output is an optimized warning message.

[0509] Step 6:

[0510] The notification system displays the alert message from the server on the display of the smart glasses. The input is the alert message sent from the server. The output is the specific alert message displayed on the smart glasses.

[0511] Step 7:

[0512] The staff member, who is the user, checks the warning message displayed on the smart glasses and takes appropriate action. The input is the warning message displayed on the smart glasses. The output is the specific action that the staff member takes for the customer.

[0513] These steps enable the system to detect potential fraud in real time in physical stores and provide optimal responses based on the emotional state of staff. For example, effective fraud detection can be achieved by analyzing all video and audio data containing "large cash transactions" and "requests for personal information" into a generative AI model and inputting prompt sentences to determine the likelihood of fraud.

[0514] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0515] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0516] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0517] [Second embodiment]

[0518] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0519] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0520] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0521] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0522] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0523] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0524] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0525] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0526] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0527] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0528] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0529] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0530] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone, and notifies users with appropriate alerts. Specific embodiments of this system are described in detail below.

[0531] Email and SNS fraud detection function

[0532] System Configuration

[0533] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[0534] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[0535] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[0536] Program processing overview

[0537] 1. The server receives an email or SNS message.

[0538] 2. The generated AI means on the server analyzes the content.

[0539] 3. Detect potentially fraudulent claims and unreliable URLs.

[0540] 4. Generate a warning message and notify the user by means of creating an alert and notifying the user.

[0541] Specific examples

[0542] For example, if the server receives an email with the content "Please transfer the money to my bank account immediately," the AI ​​generator will detect the phrase, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[0543] Automated telephone answering function

[0544] System Configuration

[0545] Device: Accept calls from blocked or unknown numbers.

[0546] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[0547] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[0548] Program processing overview

[0549] 1. Your device receives a call from a blocked or unknown number.

[0550] 2. The server initiates an automated response using generated AI means.

[0551] 3. The server records the call, and the generating AI means analyzes the recording.

[0552] 4. A warning message is generated by the means for creating an alert and notifying the user, and the user is notified along with the recorded content.

[0553] Specific examples

[0554] For example, when a call comes in from an unidentified number to a device, the server uses AI generation to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is then analyzed, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and is notified to the user along with the recorded data. This allows the user to respond calmly.

[0555] Responding to new fraud methods

[0556] System Configuration

[0557] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[0558] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[0559] Program processing overview

[0560] 1. The server collects information about new fraud methods.

[0561] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[0562] 3. The generative AI method analyzes the updated model for potential fraud.

[0563] Specific examples

[0564] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the AI ​​generation means as learning data. This allows the AI ​​generation means to respond to the latest fraud techniques and provide accurate alerts to users.

[0565] In this way, it is possible to provide a system that reduces the risk of fraud and protects users from fraud.

[0566] The processing flow will be explained below.

[0567] Email and SNS fraud detection function

[0568] Processing Steps

[0569] Step 1:

[0570] The server receives an email or SNS message and stores the message content and metadata (sender, subject, body) in a database.

[0571] Step 2:

[0572] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[0573] Step 3:

[0574] The extracted information is compared with existing databases to detect potentially fraudulent phrases and unreliable URLs, such as keywords like "transfer" or "urgent," or suspicious domains.

[0575] Step 4:

[0576] Scoring the likelihood of fraud: The generative AI method determines the likelihood of fraud based on the extracted information and assigns a score.

[0577] Step 5:

[0578] If the method for creating an alert and notifying the user determines that the email is likely to be fraudulent, a warning message will be generated. Specifically, an alert message such as "This email may be fraudulent" will be generated.

[0579] Step 6:

[0580] The notification means notifies the user of the alert. Possible notification methods include push notifications and emails.

[0581] Automated telephone answering function

[0582] Processing Steps

[0583] Step 1:

[0584] The device receives a call from a blocked or unknown phone number and sends the received phone number information to the server.

[0585] Step 2:

[0586] The server starts an automated voice response using the generation AI means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[0587] Step 3:

[0588] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[0589] Step 4:

[0590] The generative AI method analyzes the content of the call, checking for the presence of specific keywords and phrases, and scoring the likelihood of fraud. For example, keywords such as "transfer money" and "give me your personal information" indicate a potential fraud.

[0591] Step 5:

[0592] If the means for creating an alert and notifying the user determines that there is a high possibility of fraud, a warning message will be generated. An alert message such as "Possible fraud" will be generated along with the recorded content.

[0593] Step 6:

[0594] The notification means notifies the user of the alert and the recorded content, allowing the user to respond calmly.

[0595] Responding to new fraud methods

[0596] Processing Steps

[0597] Step 1:

[0598] The server collects information about new fraud methods, possibly from police reports or security blogs.

[0599] Step 2:

[0600] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training dataset and retraining the generative AI model.

[0601] Step 3:

[0602] The server deploys the updated generative AI method, updating the system-wide AI model to respond to new fraudulent techniques.

[0603] Step 4:

[0604] The generative AI method uses the updated model to analyze potential fraud, providing analysis capabilities that are in line with new fraud methods, making it possible to respond to the latest fraud techniques.

[0605] In this way, the present invention aims to enable users to avoid the risk of fraud in advance and provide a safe communication environment.

[0606] Example 1

[0607] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0608] Fraudulent activities are on the rise in communications and phone calls over the Internet, increasing the risk of many users becoming victims. However, because it is difficult to sufficiently reduce these risks using conventional methods, there is a need for technology that can detect potential fraud with high accuracy and notify users promptly.

[0609] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0610] In this invention, the server includes means for receiving emails and messages from social networking services, means for analyzing the content with a generation AI means, means for detecting potentially fraudulent wording and unreliable URLs, means for creating an alert and notifying the user, means for receiving calls from withheld or unknown phone numbers and responding with an automated voice with the generation AI means, recording and analyzing the content of the call, and means for collecting information on new fraud methods and updating the generation AI means. This makes it possible to detect possible fraud with high accuracy and notify the user promptly.

[0611] A "server" is a central processing unit that receives, analyzes, stores, and notifies users of emails and messages from social networking services.

[0612] "Generative AI means" refers to an artificial intelligence model and its processing means for analyzing received content and determining the possibility of fraud.

[0613] "Notification means" refers to a device or method that creates an alert message and sends a warning to the user by email, SMS, push notification, or other means.

[0614] The "message receiving means" is a function that receives messages from emails and social networking services and transmits them to the server.

[0615] "Analysis means" refers to the process of analyzing received messages and call content using the generation AI means.

[0616] "Detection means" is a function that extracts potentially fraudulent wording and unreliable URLs from the analyzed content.

[0617] An "alert generator" is a function that generates a warning message to the user when a possible fraud is detected.

[0618] "Call answering means" is a function in which the server uses AI generation means to respond with an automated voice to calls from anonymous or unknown phone numbers.

[0619] The "call recording means" is a function that records the contents of a call in real time and saves the recorded data.

[0620] "Information gathering means" refers to the function of gathering information about new fraud methods from the Internet and specialized institutions.

[0621] The "update method" is a function that reflects collected information on new fraud methods and retrains the generation AI method.

[0622] The present invention is a system that utilizes generative AI to detect possible fraud occurring through email, messages on social networking services, and telephone calls, and notifies users with appropriate alerts. Specific embodiments for implementing the present invention are described in detail below.

[0623] Email and SNS fraud detection function

[0624] System Configuration

[0625] This system is configured as follows:

[0626] Server: Receives emails and messages from social networking services and analyzes their contents using generative AI.

[0627] Generative AI means: Analyzes the content of received messages to detect potential fraud, using generative AI models such as GPT-3.

[0628] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[0629] operation

[0630] The server receives messages from email or social networking services (e.g., Gmail, Facebook). After receiving the messages, it uses a generative AI method (e.g., GPT-3) to analyze the message content. The generative AI analyzes the message's context and keywords based on the prompt text and assesses the likelihood of fraud.

[0631] For example, if an email with the content "Please transfer the money to my bank account urgently" arrives at the server, the AI ​​generator will detect the wording, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[0632] Prompt Sentence Examples

[0633] Analyze whether this email is a scam: "Please transfer the money to my bank account immediately."

[0634] Automated telephone answering function

[0635] System Configuration

[0636] Device: Accept calls from blocked or unknown numbers.

[0637] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[0638] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[0639] operation

[0640] When the device receives a call from a blocked or unknown phone number, the server uses a generation AI method to respond with an automated voice. For example, it may respond with, "This call is being recorded. Please explain your purpose." The call is then recorded and analyzed by the generation AI method. If the call contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and the user is notified along with the recorded data. This allows the user to respond calmly.

[0641] Prompt Sentence Examples

[0642] Analyze whether this call is a scam: "This call is being recorded. Please explain your purpose."

[0643] Responding to new fraud methods

[0644] System Configuration

[0645] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[0646] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[0647] operation

[0648] The server periodically collects information on the latest fraud techniques. Based on information from the internet and specialized institutions, the generation AI means is retrained and updated. For example, if a new "QR code fraud" is discovered, that information is collected and reflected in the generation AI means as learning data. This allows the generation AI means to respond to the latest fraud techniques and provide accurate alerts to users.

[0649] Prompt Sentence Examples

[0650] Analyze information about new fraud methods and update your AI models to respond: "QR code fraud has been discovered."

[0651] In this way, it is possible to detect possible fraud with high accuracy and notify the user promptly, thereby providing a system that can reduce the risk of fraud and ensure the safety of users.

[0652] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0653] Email and SNS fraud detection function

[0654] Step 1:

[0655] The server receives messages from email or social networking services. The server retrieves message data from external messaging services using a specified API or protocol (e.g., IMAP, Graph API). The input to this process is the message data received from email or social networking services, and the output is the storage of the received messages.

[0656] Step 2:

[0657] The server analyzes the content using generative AI means. The server inputs the received message into a generative AI model (e.g., GPT-3). It generates a prompt (e.g., "Please analyze whether this email is fraudulent: 'Please transfer the money to my bank account quickly.'") and passes it to the generative AI model. The input to this process is the content of the received message, and the output is the generative AI model's assessment of the likelihood of fraud.

[0658] Step 3:

[0659] The server detects potentially fraudulent text and unreliable URLs. Based on the analysis results from the generative AI model, the server detects specific keywords and phrases (e.g., "transfer" or "bank account"). It also checks the reliability of URLs and whether they are blacklisted. The input to this process is the evaluation result of the generative AI model, and the output is a list of text and URLs that are deemed to be potentially fraudulent.

[0660] Step 4:

[0661] The server creates an alert and notifies the user via a notification mechanism. If the server determines that there is a high possibility of fraud, it generates a warning message. It sends the alert to the user via a notification mechanism (email, SMS, push notification, etc.). The input to this process is the data that has been determined to be potentially fraudulent, and the output is a warning message that is sent to the user.

[0662] Automated telephone answering function

[0663] Step 1:

[0664] The terminal receives an incoming call from a blocked or unknown phone number. When the terminal detects the incoming call, it sends the information to the server. The input of this process is the incoming call notification, and the output is the incoming call information sent to the server.

[0665] Step 2:

[0666] The server uses the generation AI means to initiate an automatic response. The server uses the generation AI means to respond with an automated voice saying, "This call is being recorded. Please explain your purpose." The input of this process is the incoming call information, and the output is an automated voice message.

[0667] Step 3:

[0668] The server records the call content, and the generative AI means analyzes the recording. The server records the call content in real time and inputs the recording data into the generative AI model. The generative AI model analyzes specific keywords and phrases and evaluates the likelihood of fraud. The input to this process is the recorded call content, and the output is the evaluation result obtained from the generative AI model.

[0669] Step 4:

[0670] The server creates an alert and notifies the user via a notification method. If it determines that there is a high possibility of fraud, the server creates a warning message and notifies the user along with the recorded data. The input to this process is the evaluation result of the generative AI model, and the output is the warning message and recorded data sent to the user.

[0671] Responding to new fraud methods

[0672] Step 1:

[0673] The server collects information about new fraud methods. The server periodically collects information about the latest fraud methods from the Internet and specialized organizations. The input of this process is information collected from outside, and the output is information about fraud methods stored in an internal database.

[0674] Step 2:

[0675] The server updates the generative AI means. Based on the collected information, the server updates the learning dataset of the generative AI means and retrains it. The input of this process is the collected information on fraudulent techniques, and the output is an updated generative AI model.

[0676] Step 3:

[0677] The server performs the analysis using the updated generative AI model. The server then re-analyzes the email and call content using the updated AI model. The input to this process is the data to be analyzed based on the new fraud technique, and the output is the latest evaluation result.

[0678] These specific processing steps enable the system to detect potential fraud with high accuracy and notify the user promptly.

[0679] (Application example 1)

[0680] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0681] There is a problem that it is difficult for users to prevent fraudulent communications from emails, SNS messages, and calls from anonymous or unknown phone numbers. Current technology lacks the means to efficiently detect these fraudulent communications and quickly warn users, which means that users are unable to take appropriate precautions.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0683] In this invention, the server includes a generating AI means for analyzing content that may be fraudulent, a means for creating an alert and notifying the user if it is determined that there is a high possibility of fraud, a means for updating the generating AI to respond to new fraudulent methods, a means for analyzing emails and SMS messages in real time, a means for automatically answering, recording, and analyzing telephone calls and sending an immediate alert if there is a possibility of fraud, a means for analyzing the content of emails and telephone calls that may be fraudulent based on a generating AI model, and a means for sending notifications in real time. This makes it possible to detect possible fraud occurring through emails, SNS messages, and telephone calls and to send warnings to users quickly and accurately.

[0684] A "server" is a computer system that uses generative AI means to analyze emails, social media messages, and phone calls, detect potential fraud, and respond to new fraud methods.

[0685] "Generative AI means" refers to a function that uses a generative AI model to analyze the content of emails, social media messages, and phone calls to detect possible fraud.

[0686] "Means for creating alerts and notifying users" refers to a function that immediately generates and sends a warning message to users when it is determined that there is a high possibility of fraud.

[0687] "Means for updating" refers to the ability to update the learning data and algorithms of the generative AI model to respond to new fraudulent methods.

[0688] "Means for analyzing emails and SMS messages in real time" refers to a function that instantly analyzes received emails and SMS messages using AI-generated means to determine whether they are fraudulent.

[0689] "Means for automatically answering, recording and analyzing calls, and sending immediate alerts in the event of a possible fraud" refers to a function that automatically answers calls from anonymous or unknown phone numbers, records and analyzes the content of the call, and sends a warning to the user if it is determined that there is a high possibility of fraud.

[0690] "Generative AI Model" means a machine learning algorithm utilized in a generative AI method, and is a trained model used to detect potential fraud.

[0691] A "prompt sentence" is text data input to a generative AI model, and is the sentence that is analyzed to determine the possibility of fraud.

[0692] This invention is a system that uses generative AI to detect potential fraudulent activity occurring through email, social media messages, and phone calls, and quickly and accurately notifies users of the alert. Specific embodiments of the invention are described in detail below.

[0693] System Configuration

[0694] The system consists of the following major components:

[0695] 1. Server: Analyzes emails, social media messages, and phone calls using generative AI methods to detect potential fraud. Also, updates the generative AI model to adapt to new fraud methods.

[0696] 2. Generative AI methods: These are machine learning models that analyze the content of emails, social media messages, and phone calls to determine the likelihood of fraud. Generative AI methods are implemented using the OpenAI API, among other things.

[0697] 3. Notification method: The method by which the user will be alerted, which may include SMS, email, or phone notification.

[0698] 4. Automated Telephone Answering System: Uses the Twilio API to answer calls from anonymous or unknown phone numbers.

[0699] Email and SNS message analysis

[0700] The server analyzes received emails and SNS messages using the AI ​​generation means. For example, if the email content contains a phrase such as "Please transfer the money to my bank account immediately," the AI ​​generation means will determine that it is "possibly fraudulent," and an alert will be immediately created to notify the user. This will allow the user to be on guard. An example of a prompt sentence is as follows: "Do you think the following message is potentially fraudulent?" (followed by the specific email content).

[0701] Automated call answering and analysis

[0702] When the server receives a call from an unidentified or unknown phone number, it uses the Twilio API to initiate an automatic response. For example, a message saying "This call is being recorded, please state your purpose" is played, and then the call is recorded. A generating AI method analyzes the recording, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating "Possible fraud" and notifying the user. An example of a prompt sentence is as follows: "Do you think the following call content is likely to be fraudulent?" (followed by the specific recording content).

[0703] Responding to new fraud methods

[0704] The server collects information about new fraud methods and periodically updates the generative AI model. For example, if a new "QR code fraud" is discovered, that information is reflected in the generative AI model to improve its response capabilities. This allows users to be alerted to the latest fraud methods.

[0705] Hardware and software used

[0706] Specifically, the server uses a high-performance computer system, the AI ​​generation method uses OpenAI's API, the telephone answering system incorporates Twilio's API, and notification methods use common SMS gateways and email servers.

[0707] As a result, this system can efficiently detect potential fraudulent activity via email, telephone, or social media, and quickly send warnings to users, significantly reducing the risk of becoming a victim of fraud.

[0708] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0709] Step 1:

[0710] The server receives emails and SMS messages.

[0711] Input: The content of emails and SMS messages that are sent to users.

[0712] Processing: The server stores the received emails and SMS messages and prepares them for analysis.

[0713] Output: The message data to be parsed.

[0714] Step 2:

[0715] The server analyzes the message content using generated AI methods.

[0716] Input: The message data obtained in step 1.

[0717] Processing: A prompt is input into the generative AI model to evaluate the likelihood of fraud. The specific message content is input into the generative AI model along with the prompt, "Is the following message likely to be fraudulent?"

[0718] Output: Assessment results for likelihood of fraud.

[0719] Step 3:

[0720] Determine if the server is likely to be fraudulent.

[0721] Input: Evaluation results obtained in step 2.

[0722] Processing: If the evaluation results in a high probability of fraud, the server creates an alert.

[0723] Output: Fraud warning message.

[0724] Step 4:

[0725] The server notifies the user with a warning message.

[0726] Input: The fraud warning message created in step 3.

[0727] Processing: Use an SMS gateway or mail server to send a warning message to the user's device.

[0728] Output: A warning message that will be displayed on the user's terminal.

[0729] Step 5:

[0730] The server receives a call from a blocked or unknown phone number.

[0731] Input: Incoming call signal from a blocked or unknown number.

[0732] Processing: Detects an incoming call signal and initiates an automatic response.

[0733] Output: Command to start the auto-reply.

[0734] Step 6:

[0735] The server records the call and analyzes it using generative AI.

[0736] Input: The call content obtained in step 5.

[0737] Processing: The call is recorded using the Twilio API and the recording is fed into the generative AI model along with a prompt: "Do you think the following call is likely to be fraudulent?"

[0738] Output: Assessment results for likelihood of fraud.

[0739] Step 7:

[0740] The server issues a warning to the user if there is a high possibility of fraud.

[0741] Input: Evaluation results obtained in step 6.

[0742] Processing: If a fraudulent activity is deemed likely, a warning message is sent to the user via an SMS gateway or calling system.

[0743] Output: The warning message and audio recording that will be displayed on the user's device.

[0744] Step 8:

[0745] The server collects information about new fraud methods and updates the generative AI model.

[0746] Input: Information on new fraud methods.

[0747] Processing: Retraining the generative AI model based on collected information to improve fraud detection accuracy.

[0748] Output: An updated generative AI model.

[0749] As a result, the system can detect potential fraud in real time and provide appropriate warnings to users.

[0750] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0751] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone calls, and notifies the user with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state. The following describes specific embodiments of the present invention.

[0752] Email and SNS fraud detection function

[0753] System Configuration

[0754] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[0755] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[0756] Emotion engine: Analyzes the user's emotional state and optimizes notification content based on that information.

[0757] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[0758] Program processing overview

[0759] 1. The server receives an email or SNS message.

[0760] 2. Generative AI methods on the server analyze the content and detect possible fraud.

[0761] 3. The emotion engine analyzes the user's emotional state in real time.

[0762] 4. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[0763] 5. The notification means notifies the user of the warning message.

[0764] Specific examples

[0765] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the generation AI means will detect the message and create an alert saying "This may be a scam." The emotion engine will analyze the user's emotional state, and if the user is nervous, it will add a message to the notification such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[0766] Automated telephone answering function

[0767] System Configuration

[0768] Device: Accept calls from blocked or unknown numbers.

[0769] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[0770] Emotion Engine: Analyzes the user's emotional state and optimizes notification content.

[0771] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[0772] Program processing overview

[0773] 1. Your device receives a call from a blocked or unknown number.

[0774] 2. The server initiates an automated response using generated AI means.

[0775] 3. The server records the call, and the generating AI means analyzes the recording.

[0776] 4. The emotion engine analyzes the user's emotional state in real time.

[0777] 5. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[0778] 6. The notification means notifies the user of the warning message and the recorded content.

[0779] Specific examples

[0780] For example, when a device receives a call from an unidentified number, the server uses a generative AI method to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion engine analyzes the user's emotional state and detects that the user is in a tense state. In this case, the server notifies the user by adding a message saying, "Please remain calm." This allows the user to respond calmly.

[0781] Responding to new fraud methods

[0782] System Configuration

[0783] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[0784] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[0785] Emotion engine: Analyzes the user's emotional state in real time and optimizes notification content based on that information.

[0786] Program processing overview

[0787] 1. The server collects information about new fraud methods.

[0788] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[0789] 3. The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud techniques.

[0790] 4. The generative AI method uses the updated model to analyze potential fraud.

[0791] Specific examples

[0792] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it as learning data in the generation AI. Furthermore, the emotion engine analyzes the user's emotional state, allowing it to provide appropriate responses in real time. For example, if a user is nervous about a new fraudulent technique, it will provide a message urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[0793] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[0794] The processing flow will be explained below.

[0795] Email and SNS fraud detection function

[0796] Processing Steps

[0797] Step 1:

[0798] The server receives an email or SNS message and stores the received content and metadata (sender, subject, body) in a database.

[0799] Step 2:

[0800] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[0801] Step 3:

[0802] The generative AI method compares the extracted information with existing databases to detect potentially fraudulent phrases and unreliable URLs, particularly those containing keywords that indicate potential fraud, such as "transfer" or "urgent," as well as suspicious domains.

[0803] Step 4:

[0804] Generative AI methods score the likelihood of fraud, quantifying fraud risk based on extracted information and flagging high risk cases.

[0805] Step 5:

[0806] The emotion engine analyzes the user's emotional state in real time, for example by analyzing the user's facial expressions and voice to detect tension or anxiety.

[0807] Step 6:

[0808] If a fraudulent activity is deemed likely, the emotion engine will create a warning message based on the user's emotional state. For users in a tense state, a message such as "Stay calm and don't react immediately" will be added.

[0809] Step 7:

[0810] The notification method notifies the user of the created alert, sending a warning message via push notification or email to alert the user.

[0811] Automated telephone answering function

[0812] Processing Steps

[0813] Step 1:

[0814] The device receives a call from a blocked or unknown phone number and sends the received call information to the server.

[0815] Step 2:

[0816] The server starts an automated voice response using the AI ​​generation means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[0817] Step 3:

[0818] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[0819] Step 4:

[0820] The generative AI method analyzes the content of the call, checking for the presence of certain keywords and phrases and assessing the likelihood of fraud. For example, phrases such as "transfer money" and "give me your personal information" are recognized as indicators of fraud.

[0821] Step 5:

[0822] The emotion engine analyzes the user's emotional state in real time, analyzing the user's tone of voice and choice of words during a call to detect tension or anxiety.

[0823] Step 6:

[0824] If a fraudulent activity is deemed likely, the emotion engine will create a warning message for users in a state of anxiety, for example adding a message saying "Please stay calm."

[0825] Step 7:

[0826] A notification mechanism will notify the user of the generated alert and recording, and in particularly high-risk cases, notifications will also be sent to the user's next of kin or trusted third parties.

[0827] Responding to new fraud methods

[0828] Processing Steps

[0829] Step 1:

[0830] The server collects information about new fraud methods, deriving data from sources such as police reports and security blogs.

[0831] Step 2:

[0832] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training data and retraining the generative AI model.

[0833] Step 3:

[0834] Deploy new models with updated generative AI methods to update the system to address the latest fraud techniques.

[0835] Step 4:

[0836] The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address new fraud techniques, helping users stay calm even when faced with new fraud methods.

[0837] Step 5:

[0838] The generative AI method uses the updated model to analyze new communications from users, providing analysis capabilities tailored to new fraud techniques and generating appropriate alerts for users.

[0839] Example 2

[0840] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0841] In recent years, fraudulent activities using email, social media, and telephone have been increasing, and their methods have become more diverse and sophisticated. Therefore, users need to be able to quickly and reliably identify fraud risks and take appropriate measures. However, conventional fraud detection systems focus on identifying potential fraud and do not provide flexible responses that take into account the user's emotional state. Furthermore, they face the problem of being difficult to update quickly to respond to new fraud methods. Therefore, a method is needed that enables real-time fraud detection and flexible responses while ensuring user safety.

[0842] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0843] In this invention, the server includes means for receiving and analyzing the contents of electronic messages, means for analyzing the contents of electronic messages and detecting the possibility of fraud, means for analyzing the emotional state of the user and optimizing the notification content, means for notifying the user of a warning message when it is determined that there is a high possibility of fraud, and means for updating the generation AI to respond to new fraud methods. This makes it possible to quickly detect the risk of fraud in messages and calls received by the user and provide an appropriate response according to the user's emotional state.

[0844] "Electronic Message" refers to text messages sent or received over the Internet, such as emails or social media messages.

[0845] "Generative AI methods" refer to algorithms or models that use machine learning or deep learning to analyze text or voice data to determine the likelihood of fraud.

[0846] "Emotion analysis means" refers to technology that analyzes the user's emotional state in real time based on facial expressions and voice data, and optimizes the content of notifications.

[0847] A "warning message" refers to a message that is sent to the user to warn them when it is determined that there is a high possibility of fraud.

[0848] "New fraud methods" refers to newly discovered means and techniques for committing fraud, in addition to traditional fraud methods.

[0849] "Means for receiving and analyzing the content of electronic messages" refers to methods and technologies for retrieving electronic messages from mail servers or social media platforms and analyzing their text content.

[0850] "Generative AI methods for detecting potential fraud" refers to methods that use machine learning and deep learning models to analyze electronic messages and phone calls to identify signs and risks of fraud.

[0851] "Means for analyzing the user's emotional state and optimizing the content of notifications" refers to technology that analyzes the user's psychological state and generates an appropriate response message based on the results.

[0852] "Means for notifying users when it is determined that there is a high possibility of fraud" refers to communication technology for conveying a warning to users based on the analysis results of the generating AI means.

[0853] "Means to update generative AI" refers to technology that retrains generative AI models based on new data and technical information, enabling them to respond to the latest fraud techniques.

[0854] This invention is a system that uses generative AI to detect possible fraud via email, social media, and phone calls, and notifies users with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state.

[0855] Email and SNS fraud detection function

[0856] System Configuration

[0857] Server: Receives emails and social media messages and uses the Gmail API and Twitter API to analyze their contents using generative AI.

[0858] Generative AI method: Using OpenAI's GPT-3 and other technologies, the content of received emails and social media messages is analyzed to detect possible fraud.

[0859] Emotion analysis method: Using Microsoft's Azure Emotion API and other tools, the user's emotional state is analyzed in real time and the content of notifications is optimized based on that information.

[0860] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user via smartphone push notifications or email notifications.

[0861] Program processing overview

[0862] The server analyzes received emails and SNS messages. The generation AI means analyzes the content and detects the possibility of fraud. The emotion analysis means analyzes the user's emotional state, and if it determines that there is a high possibility of fraud, it optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user with a warning message.

[0863] Specific examples

[0864] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the AI ​​generation means will detect the message and create an alert saying "This may be a scam." The emotion analysis means will analyze the user's emotional state, and if the user is nervous, it will add a message such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[0865] Prompt Sentence Examples

[0866] When you receive an email saying "Please transfer money to your bank account urgently," rate it as likely to be a scam. Also, if the user is nervous, add a message saying "Please stay calm and don't respond immediately."

[0867] Automated telephone answering function

[0868] System Configuration

[0869] Device: Use a VoIP service connected to your smartphone or landline to receive calls from blocked or unknown numbers.

[0870] Server: Generates automated voice responses using AI and uses voice recognition services such as Google Dialogflow to record and analyze call content.

[0871] Sentiment analysis: Analyze the user's emotional state in real time using the Amazon Polly API, etc.

[0872] Notification method: If a fraudulent activity is deemed likely, a warning message and recording will be sent to the user via push notification or email notification.

[0873] Program processing overview

[0874] The device receives a call from a blocked or unknown phone number. The server initiates an automated voice response and records the call. The recording is analyzed by the generative AI means to assess the possibility of fraud. At the same time, the emotion analysis means analyzes the user's emotional state in real time and optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user of a warning message and the recording content.

[0875] Specific examples

[0876] For example, when a call comes in from an unidentified number to a terminal, the server automatically responds with "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion analysis means analyzes the user's emotional state and detects that the user is nervous. In this case, the system adds a message to the user saying, "Please remain calm." This allows the user to respond calmly.

[0877] Prompt Sentence Examples

[0878] If you receive a call from an unidentified number, check whether the call contains keywords such as "transfer" or "personal information." If the user is nervous, add a message to the call saying, "Please stay calm."

[0879] Responding to new fraud methods

[0880] System Configuration

[0881] Server: Uses web scraping techniques, news sites, and forum data to gather information on the latest fraudulent techniques and update the generative AI methods.

[0882] Generative AI methods: Retraining machine learning models to constantly update learning data and models based on new fraud techniques.

[0883] Sentiment analysis method: We use the sentiment analysis API to analyze the user's emotional state in real time and optimize the notification content.

[0884] Program processing overview

[0885] The server collects information on new fraud methods and updates the AI ​​generation method. The emotion analysis method analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud methods. The AI ​​generation method uses the updated model to analyze the possibility of fraud.

[0886] Specific examples

[0887] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the generation AI means as learning data. Furthermore, the emotion analysis means analyzes the user's emotional state, allowing for appropriate responses to be provided in real time. For example, if a user is nervous about a new fraudulent technique, a message is displayed urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[0888] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[0889] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0890] Email and SNS fraud detection function

[0891] Processing Steps

[0892] Step 1:

[0893] The server receives electronic messages. Specifically, it periodically checks emails and SNS messages using the Gmail API and Twitter API to retrieve new messages.

[0894] Input: New email or social media message

[0895] Output: Received electronic message

[0896] Step 2:

[0897] The server analyzes the content of the received electronic message using a generative AI method. The generative AI method (e.g., OpenAI GPT-3) is used to analyze the message text and assess its likelihood of fraud.

[0898] Input: Received electronic message

[0899] Data processing / computation: Detecting fraud patterns through natural language processing

[0900] Output: Fraud probability assessment result

[0901] Step 3:

[0902] The server analyzes the user's emotional state using emotion analysis tools, such as Microsoft's Azure Emotion API, which analyzes the user's facial expressions and voice in real time to estimate their emotional state.

[0903] Input: User's facial expression and voice data

[0904] Data processing / computation: Classifying emotional states using emotion recognition models

[0905] Output: User's emotional state

[0906] Step 4:

[0907] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[0908] Input: Fraud probability assessment, user emotional state

[0909] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[0910] Output: personalized warning message

[0911] Step 5:

[0912] The server notifies the user of the warning message. Appropriate warnings are sent to the user via smartphone push notifications or email.

[0913] Input: personalized warning message

[0914] Output: Message notified to the user

[0915] Automated telephone answering function

[0916] Processing Steps

[0917] Step 1:

[0918] Your device receives calls from blocked or unknown numbers. Use a VoIP service connected to your smartphone or landline to monitor whether there are any incoming calls.

[0919] Input: Calls from blocked or unknown numbers

[0920] Output: Incoming call notification

[0921] Step 2:

[0922] The server initiates an automated voice response using a generative AI method, using a speech recognition service such as Google Dialogflow to play a message such as "This call is being recorded, please state your purpose."

[0923] Input: Incoming call notification

[0924] Output: Playback of automated voice message

[0925] Step 3:

[0926] The server records the call, and the AI ​​generator analyzes the recording. The recorded voice data is converted into text using an NLP service and analyzed.

[0927] Input: Recorded call

[0928] Data processing / computation: speech-to-text conversion and text analysis

[0929] Output: Fraud probability assessment result

[0930] Step 4:

[0931] The server analyzes the user's emotional state using emotion analysis tools. It uses the Amazon Polly API to analyze the voice data during the call and evaluates the user's emotional state in real time.

[0932] Input: User's voice data

[0933] Data processing / calculation: Analyzing emotional states from voice data

[0934] Output: User's emotional state

[0935] Step 5:

[0936] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[0937] Input: Fraud probability assessment, user emotional state

[0938] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[0939] Output: personalized warning message

[0940] Step 6:

[0941] The server notifies the user of the warning message and recording contents, and sends appropriate warnings to the user via push notification or email.

[0942] Input: personalized warning message, recording

[0943] Output: Message and recording notified to the user

[0944] Responding to new fraud methods

[0945] Processing Steps

[0946] Step 1:

[0947] The server collects information about new fraud methods, using web scraping technology to retrieve the latest information from security news sites and forums.

[0948] Input: Information about a new fraud scheme

[0949] Output: Collected fraud data

[0950] Step 2:

[0951] The server updates the generative AI method and retrains the generative AI model to adapt to new fraud methods based on the collected information.

[0952] Input: Data on new fraud methods

[0953] Data processing / computation: Retraining AI models

[0954] Output: Updated generative AI model

[0955] Step 3:

[0956] The server uses emotion analysis to analyze the user's emotional state in real time, optimizing alerts to address the latest fraud techniques.

[0957] Input: User emotional state data

[0958] Data processing / calculation: Optimizing notification messages based on emotion data

[0959] Output: Optimized warning message

[0960] Step 4:

[0961] The server uses the updated generative AI model to analyze the possibility of fraud. The latest model analyzes received messages and call content to respond to new fraud methods.

[0962] Input: Received messages and calls

[0963] Data manipulation / calculation: Analysis based on new fraud methods

[0964] Output: Fraud probability assessment result

[0965] These processing steps allow users to proactively avoid the risk of fraud and receive the most appropriate response based on their emotional state.

[0966] (Application example 2)

[0967] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0968] In recent years, fraudulent activities have evolved on a daily basis, requiring immediate and reliable responses to prevent actual damage. However, no systems exist that take into account the user's emotional state, and fraud detection and responses are often inappropriate. In particular, in physical stores, it is difficult for staff to detect fraud while interacting with customers, and it is difficult to make calm decisions when the user is in a tense state. To solve these issues, a system is needed that utilizes wearable devices such as smart glasses to detect possible fraud in real time and provide optimal responses based on the user's emotions.

[0969] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means for analyzing possible fraudulent content, a means for creating an alert and notifying the user when it is determined that fraud is highly likely, a means for updating the generation AI to respond to new fraudulent methods, a means for analyzing the user's emotional state using an emotion engine and optimizing the notification content, and a means for displaying the alert on the smart glasses. This enables fraudulent acts in physical stores to be detected in real time and an optimal response based on emotions. Specifically, when staff are wearing smart glasses, the smart glasses analyze customer behavior and conversation content and display an effective warning message when possible fraud is detected. In addition, instructions are provided according to the staff's emotional state, allowing for a calm and prompt response.

[0970] "Generative AI" is a system that uses artificial intelligence techniques to analyze data and detect specific patterns and anomalies.

[0971] An "emotion engine" is a technology that analyzes a user's emotional state from their voice and images, and is a system that grasps emotions such as stress and tension in real time.

[0972] "Notification means" refers to a system or device that conveys warnings or instructions to users based on information detected by the generative AI or emotion engine.

[0973] "Update methods" are functions that allow systems and models to be updated with the latest data and methods, keeping them up to date at all times.

[0974] "Smart glasses" are wearable devices equipped with a display and communication functions, allowing users to visually receive and integrate information.

[0975] A "server" is a computer system that receives, processes, and transmits data over a network, and is used to manage and operate multiple functions in an integrated manner.

[0976] "Fraud detection" is the process of determining whether certain actions or words are potentially fraudulent, and is performed using generative AI.

[0977] The present invention is a system for detecting possible fraudulent activity in a brick-and-mortar store and providing an optimal response based on the emotional state of the user. Specific embodiments for carrying out the present invention will be described below.

[0978] System Configuration

[0979] 1. Hardware Configuration

[0980] Smart glasses: Equipped with a display, camera, and microphone, worn by staff.

[0981] Server: A computer with high-performance data processing capabilities, primarily responsible for data analysis and running generative AI models.

[0982] Communication method: Uses 5G networks that enable high-speed data communication.

[0983] 2. Software Configuration

[0984] Generative AI models, such as GPT-4 and BERT, can be used to detect potential fraud from audio and video data.

[0985] Emotion Engine: Analyzes staff emotional states in real time using Affectiva and Emotion AI.

[0986] Notification system: Software that notifies staff in real time of warnings and appropriate responses.

[0987] Program processing

[0988] The server receives the video and audio data sent from the smart glasses and analyzes it using a generative AI model. Specifically, it determines whether the customer's behavior or comments indicate the possibility of fraud. If the analyzed data indicates the possibility of fraud, the emotion engine analyzes the staff member's current emotional state. For example, if a staff member is nervous, the notification system will display a message on the smart glasses' display such as, "Please stay calm and treat the customer kindly."

[0989] Specific usage scenarios

[0990] Consider a scenario where a customer visits a brick-and-mortar store and wishes to make a large cash transaction. The smart glasses record the conversation with the customer, and the data is sent to a server. The server's generative AI model analyzes the customer's comments to determine whether they are likely to be fraudulent. For example, if keywords such as "large cash transaction" or "request for personal information" are detected, the emotion engine detects tension in the staff member's heart rate and facial expression. The notification system then displays a message on the smart glasses, such as "This is a possible fraud, so please ask the customer to present identification."

[0991] Prompt Sentence Examples

[0992] Analyze all video and audio recordings containing "large cash transactions" and "requests for personal information" to determine the possibility of fraud.

[0993] This invention enables fast and accurate fraud detection and response in brick-and-mortar stores, ensuring the safety of both customers and staff.

[0994] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0995] Step 1:

[0996] The smart glasses' cameras and microphones capture and record the customer's movements and conversations in real time. The input is the customer's video and audio data. The output is digital data that is sent to the server.

[0997] Step 2:

[0998] The server receives the video and audio data sent from the smart glasses. The input is the video and audio data from the smart glasses. The output is that this data is stored in the server.

[0999] Step 3:

[1000] The generative AI model in the server analyzes the received video and audio data. Specifically, it analyzes the content of the conversation and customer behavior patterns to detect the possibility of fraud. The input is the video and audio data stored in the server. The output is an analysis result indicating the possibility of fraud.

[1001] Step 4:

[1002] The server's emotion engine analyzes the emotional state of staff in real time. This is done by receiving and analyzing biometric information such as the staff's heart rate and facial expressions as input. The input is the staff's biometric information. The output is the analysis result regarding the staff's emotional state.

[1003] Step 5:

[1004] The server combines the analysis results of the generative AI model and the emotion engine and sends them to the notification system. The inputs are the analysis results indicating possible fraud and the analysis results regarding the emotional state of the staff. The output is an optimized warning message.

[1005] Step 6:

[1006] The notification system displays the alert message from the server on the display of the smart glasses. The input is the alert message sent from the server. The output is the specific alert message displayed on the smart glasses.

[1007] Step 7:

[1008] The staff member, who is the user, checks the warning message displayed on the smart glasses and takes appropriate action. The input is the warning message displayed on the smart glasses. The output is the specific action that the staff member takes for the customer.

[1009] These steps enable the system to detect potential fraud in real time in physical stores and provide optimal responses based on the emotional state of staff. For example, effective fraud detection can be achieved by analyzing all video and audio data containing "large cash transactions" and "requests for personal information" into a generative AI model and inputting prompt sentences to determine the likelihood of fraud.

[1010] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1011] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1012] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1013] [Third embodiment]

[1014] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1015] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1016] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1017] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1018] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1019] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1020] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1021] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1022] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1023] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1024] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1025] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1026] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone, and notifies users with appropriate alerts. Specific embodiments of this system are described in detail below.

[1027] Email and SNS fraud detection function

[1028] System Configuration

[1029] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[1030] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[1031] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[1032] Program processing overview

[1033] 1. The server receives an email or SNS message.

[1034] 2. The generated AI means on the server analyzes the content.

[1035] 3. Detect potentially fraudulent claims and unreliable URLs.

[1036] 4. Generate a warning message and notify the user by means of creating an alert and notifying the user.

[1037] Specific examples

[1038] For example, if the server receives an email with the content "Please transfer the money to my bank account immediately," the AI ​​generator will detect the phrase, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[1039] Automated telephone answering function

[1040] System Configuration

[1041] Device: Accept calls from blocked or unknown numbers.

[1042] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[1043] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[1044] Program processing overview

[1045] 1. Your device receives a call from a blocked or unknown number.

[1046] 2. The server initiates an automated response using generated AI means.

[1047] 3. The server records the call, and the generating AI means analyzes the recording.

[1048] 4. A warning message is generated by the means for creating an alert and notifying the user, and the user is notified along with the recorded content.

[1049] Specific examples

[1050] For example, when a call comes in from an unidentified number to a device, the server uses AI generation to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is then analyzed, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and is notified to the user along with the recorded data. This allows the user to respond calmly.

[1051] Responding to new fraud methods

[1052] System Configuration

[1053] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[1054] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[1055] Program processing overview

[1056] 1. The server collects information about new fraud methods.

[1057] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[1058] 3. The generative AI method analyzes the updated model for potential fraud.

[1059] Specific examples

[1060] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the AI ​​generation means as learning data. This allows the AI ​​generation means to respond to the latest fraud techniques and provide accurate alerts to users.

[1061] In this way, it is possible to provide a system that reduces the risk of fraud and protects users from fraud.

[1062] The processing flow will be explained below.

[1063] Email and SNS fraud detection function

[1064] Processing Steps

[1065] Step 1:

[1066] The server receives an email or SNS message and stores the message content and metadata (sender, subject, body) in a database.

[1067] Step 2:

[1068] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[1069] Step 3:

[1070] The extracted information is compared with existing databases to detect potentially fraudulent phrases and unreliable URLs, such as keywords like "transfer" or "urgent," or suspicious domains.

[1071] Step 4:

[1072] Scoring the likelihood of fraud: The generative AI method determines the likelihood of fraud based on the extracted information and assigns a score.

[1073] Step 5:

[1074] If the method for creating an alert and notifying the user determines that the email is likely to be fraudulent, a warning message will be generated. Specifically, an alert message such as "This email may be fraudulent" will be generated.

[1075] Step 6:

[1076] The notification means notifies the user of the alert. Possible notification methods include push notifications and emails.

[1077] Automated telephone answering function

[1078] Processing Steps

[1079] Step 1:

[1080] The device receives a call from a blocked or unknown phone number and sends the received phone number information to the server.

[1081] Step 2:

[1082] The server starts an automated voice response using the generation AI means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[1083] Step 3:

[1084] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[1085] Step 4:

[1086] The generative AI method analyzes the content of the call, checking for the presence of specific keywords and phrases, and scoring the likelihood of fraud. For example, keywords such as "transfer money" and "give me your personal information" indicate a potential fraud.

[1087] Step 5:

[1088] If the means for creating an alert and notifying the user determines that there is a high possibility of fraud, a warning message will be generated. An alert message such as "Possible fraud" will be generated along with the recorded content.

[1089] Step 6:

[1090] The notification means notifies the user of the alert and the recorded content, allowing the user to respond calmly.

[1091] Responding to new fraud methods

[1092] Processing Steps

[1093] Step 1:

[1094] The server collects information about new fraud methods, possibly from police reports or security blogs.

[1095] Step 2:

[1096] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training dataset and retraining the generative AI model.

[1097] Step 3:

[1098] The server deploys the updated generative AI method, updating the system-wide AI model to respond to new fraudulent techniques.

[1099] Step 4:

[1100] The generative AI method uses the updated model to analyze potential fraud, providing analysis capabilities that are in line with new fraud methods, making it possible to respond to the latest fraud techniques.

[1101] In this way, the present invention aims to enable users to avoid the risk of fraud in advance and provide a safe communication environment.

[1102] Example 1

[1103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1104] Fraudulent activities are on the rise in communications and phone calls over the Internet, increasing the risk of many users becoming victims. However, because it is difficult to sufficiently reduce these risks using conventional methods, there is a need for technology that can detect potential fraud with high accuracy and notify users promptly.

[1105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1106] In this invention, the server includes means for receiving emails and messages from social networking services, means for analyzing the content with a generation AI means, means for detecting potentially fraudulent wording and unreliable URLs, means for creating an alert and notifying the user, means for receiving calls from withheld or unknown phone numbers and responding with an automated voice with the generation AI means, recording and analyzing the content of the call, and means for collecting information on new fraud methods and updating the generation AI means. This makes it possible to detect possible fraud with high accuracy and notify the user promptly.

[1107] A "server" is a central processing unit that receives, analyzes, stores, and notifies users of emails and messages from social networking services.

[1108] "Generative AI means" refers to an artificial intelligence model and its processing means for analyzing received content and determining the possibility of fraud.

[1109] "Notification means" refers to a device or method that creates an alert message and sends a warning to the user by email, SMS, push notification, or other means.

[1110] The "message receiving means" is a function that receives messages from emails and social networking services and transmits them to the server.

[1111] "Analysis means" refers to the process of analyzing received messages and call content using the generation AI means.

[1112] "Detection means" is a function that extracts potentially fraudulent wording and unreliable URLs from the analyzed content.

[1113] An "alert generator" is a function that generates a warning message to the user when a possible fraud is detected.

[1114] "Call answering means" is a function in which the server uses AI generation means to respond with an automated voice to calls from anonymous or unknown phone numbers.

[1115] The "call recording means" is a function that records the contents of a call in real time and saves the recorded data.

[1116] "Information gathering means" refers to the function of gathering information about new fraud methods from the Internet and specialized institutions.

[1117] The "update method" is a function that reflects collected information on new fraud methods and retrains the generation AI method.

[1118] The present invention is a system that utilizes generative AI to detect possible fraud occurring through email, messages on social networking services, and telephone calls, and notifies users with appropriate alerts. Specific embodiments for implementing the present invention are described in detail below.

[1119] Email and SNS fraud detection function

[1120] System Configuration

[1121] This system is configured as follows:

[1122] Server: Receives emails and messages from social networking services and analyzes their contents using generative AI.

[1123] Generative AI means: Analyzes the content of received messages to detect potential fraud, using generative AI models such as GPT-3.

[1124] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[1125] operation

[1126] The server receives messages from email or social networking services (e.g., Gmail, Facebook). After receiving the messages, it uses a generative AI method (e.g., GPT-3) to analyze the message content. The generative AI analyzes the message's context and keywords based on the prompt text and assesses the likelihood of fraud.

[1127] For example, if an email with the content "Please transfer the money to my bank account urgently" arrives at the server, the AI ​​generator will detect the wording, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[1128] Prompt Sentence Examples

[1129] Analyze whether this email is a scam: "Please transfer the money to my bank account immediately."

[1130] Automated telephone answering function

[1131] System Configuration

[1132] Device: Accept calls from blocked or unknown numbers.

[1133] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[1134] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[1135] operation

[1136] When the device receives a call from a blocked or unknown phone number, the server uses a generation AI method to respond with an automated voice. For example, it may respond with, "This call is being recorded. Please explain your purpose." The call is then recorded and analyzed by the generation AI method. If the call contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and the user is notified along with the recorded data. This allows the user to respond calmly.

[1137] Prompt Sentence Examples

[1138] Analyze whether this call is a scam: "This call is being recorded. Please explain your purpose."

[1139] Responding to new fraud methods

[1140] System Configuration

[1141] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[1142] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[1143] operation

[1144] The server periodically collects information on the latest fraud techniques. Based on information from the internet and specialized institutions, the generation AI means is retrained and updated. For example, if a new "QR code fraud" is discovered, that information is collected and reflected in the generation AI means as learning data. This allows the generation AI means to respond to the latest fraud techniques and provide accurate alerts to users.

[1145] Prompt Sentence Examples

[1146] Analyze information about new fraud methods and update your AI models to respond: "QR code fraud has been discovered."

[1147] In this way, it is possible to detect possible fraud with high accuracy and notify the user promptly, thereby providing a system that can reduce the risk of fraud and ensure the safety of users.

[1148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1149] Email and SNS fraud detection function

[1150] Step 1:

[1151] The server receives messages from email or social networking services. The server retrieves message data from external messaging services using a specified API or protocol (e.g., IMAP, Graph API). The input to this process is the message data received from email or social networking services, and the output is the storage of the received messages.

[1152] Step 2:

[1153] The server analyzes the content using generative AI means. The server inputs the received message into a generative AI model (e.g., GPT-3). It generates a prompt (e.g., "Please analyze whether this email is fraudulent: 'Please transfer the money to my bank account quickly.'") and passes it to the generative AI model. The input to this process is the content of the received message, and the output is the generative AI model's assessment of the likelihood of fraud.

[1154] Step 3:

[1155] The server detects potentially fraudulent text and unreliable URLs. Based on the analysis results from the generative AI model, the server detects specific keywords and phrases (e.g., "transfer" or "bank account"). It also checks the reliability of URLs and whether they are blacklisted. The input to this process is the evaluation result of the generative AI model, and the output is a list of text and URLs that are deemed to be potentially fraudulent.

[1156] Step 4:

[1157] The server creates an alert and notifies the user via a notification mechanism. If the server determines that there is a high possibility of fraud, it generates a warning message. It sends the alert to the user via a notification mechanism (email, SMS, push notification, etc.). The input to this process is the data that has been determined to be potentially fraudulent, and the output is a warning message that is sent to the user.

[1158] Automated telephone answering function

[1159] Step 1:

[1160] The terminal receives an incoming call from a blocked or unknown phone number. When the terminal detects the incoming call, it sends the information to the server. The input of this process is the incoming call notification, and the output is the incoming call information sent to the server.

[1161] Step 2:

[1162] The server uses the generation AI means to initiate an automatic response. The server uses the generation AI means to respond with an automated voice saying, "This call is being recorded. Please explain your purpose." The input of this process is the incoming call information, and the output is an automated voice message.

[1163] Step 3:

[1164] The server records the call content, and the generative AI means analyzes the recording. The server records the call content in real time and inputs the recording data into the generative AI model. The generative AI model analyzes specific keywords and phrases and evaluates the likelihood of fraud. The input to this process is the recorded call content, and the output is the evaluation result obtained from the generative AI model.

[1165] Step 4:

[1166] The server creates an alert and notifies the user via a notification method. If it determines that there is a high possibility of fraud, the server creates a warning message and notifies the user along with the recorded data. The input to this process is the evaluation result of the generative AI model, and the output is the warning message and recorded data sent to the user.

[1167] Responding to new fraud methods

[1168] Step 1:

[1169] The server collects information about new fraud methods. The server periodically collects information about the latest fraud methods from the Internet and specialized organizations. The input of this process is information collected from outside, and the output is information about fraud methods stored in an internal database.

[1170] Step 2:

[1171] The server updates the generative AI means. Based on the collected information, the server updates the learning dataset of the generative AI means and retrains it. The input of this process is the collected information on fraudulent techniques, and the output is an updated generative AI model.

[1172] Step 3:

[1173] The server performs the analysis using the updated generative AI model. The server then re-analyzes the email and call content using the updated AI model. The input to this process is the data to be analyzed based on the new fraud technique, and the output is the latest evaluation result.

[1174] These specific processing steps enable the system to detect potential fraud with high accuracy and notify the user promptly.

[1175] (Application example 1)

[1176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1177] There is a problem that it is difficult for users to prevent fraudulent communications from emails, SNS messages, and calls from anonymous or unknown phone numbers. Current technology lacks the means to efficiently detect these fraudulent communications and quickly warn users, which means that users are unable to take appropriate precautions.

[1178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1179] In this invention, the server includes a generating AI means for analyzing content that may be fraudulent, a means for creating an alert and notifying the user if it is determined that there is a high possibility of fraud, a means for updating the generating AI to respond to new fraudulent methods, a means for analyzing emails and SMS messages in real time, a means for automatically answering, recording, and analyzing telephone calls and sending an immediate alert if there is a possibility of fraud, a means for analyzing the content of emails and telephone calls that may be fraudulent based on a generating AI model, and a means for sending notifications in real time. This makes it possible to detect possible fraud occurring through emails, SNS messages, and telephone calls and to send warnings to users quickly and accurately.

[1180] A "server" is a computer system that uses generative AI means to analyze emails, social media messages, and phone calls, detect potential fraud, and respond to new fraud methods.

[1181] "Generative AI means" refers to a function that uses a generative AI model to analyze the content of emails, social media messages, and phone calls to detect possible fraud.

[1182] "Means for creating alerts and notifying users" refers to a function that immediately generates and sends a warning message to users when it is determined that there is a high possibility of fraud.

[1183] "Means for updating" refers to the ability to update the learning data and algorithms of the generative AI model to respond to new fraudulent methods.

[1184] "Means for analyzing emails and SMS messages in real time" refers to a function that instantly analyzes received emails and SMS messages using AI-generated means to determine whether they are fraudulent.

[1185] "Means for automatically answering, recording and analyzing calls, and sending immediate alerts in the event of a possible fraud" refers to a function that automatically answers calls from anonymous or unknown phone numbers, records and analyzes the content of the call, and sends a warning to the user if it is determined that there is a high possibility of fraud.

[1186] "Generative AI Model" means a machine learning algorithm utilized in a generative AI method, and is a trained model used to detect potential fraud.

[1187] A "prompt sentence" is text data input to a generative AI model, and is the sentence that is analyzed to determine the possibility of fraud.

[1188] This invention is a system that uses generative AI to detect potential fraudulent activity occurring through email, social media messages, and phone calls, and quickly and accurately notifies users of the alert. Specific embodiments of the invention are described in detail below.

[1189] System Configuration

[1190] The system consists of the following major components:

[1191] 1. Server: Analyzes emails, social media messages, and phone calls using generative AI methods to detect potential fraud. Also, updates the generative AI model to adapt to new fraud methods.

[1192] 2. Generative AI methods: These are machine learning models that analyze the content of emails, social media messages, and phone calls to determine the likelihood of fraud. Generative AI methods are implemented using the OpenAI API, among other things.

[1193] 3. Notification method: The method by which the user will be alerted, which may include SMS, email, or phone notification.

[1194] 4. Automated Telephone Answering System: Uses the Twilio API to answer calls from anonymous or unknown phone numbers.

[1195] Email and SNS message analysis

[1196] The server analyzes received emails and SNS messages using the AI ​​generation means. For example, if the email content contains a phrase such as "Please transfer the money to my bank account immediately," the AI ​​generation means will determine that it is "possibly fraudulent," and an alert will be immediately created to notify the user. This will allow the user to be on guard. An example of a prompt sentence is as follows: "Do you think the following message is potentially fraudulent?" (followed by the specific email content).

[1197] Automated call answering and analysis

[1198] When the server receives a call from an unidentified or unknown phone number, it uses the Twilio API to initiate an automatic response. For example, a message saying "This call is being recorded, please state your purpose" is played, and then the call is recorded. A generating AI method analyzes the recording, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating "Possible fraud" and notifying the user. An example of a prompt sentence is as follows: "Do you think the following call content is likely to be fraudulent?" (followed by the specific recording content).

[1199] Responding to new fraud methods

[1200] The server collects information about new fraud methods and periodically updates the generative AI model. For example, if a new "QR code fraud" is discovered, that information is reflected in the generative AI model to improve its response capabilities. This allows users to be alerted to the latest fraud methods.

[1201] Hardware and software used

[1202] Specifically, the server uses a high-performance computer system, the AI ​​generation method uses OpenAI's API, the telephone answering system incorporates Twilio's API, and notification methods use common SMS gateways and email servers.

[1203] As a result, this system can efficiently detect potential fraudulent activity via email, telephone, or social media, and quickly send warnings to users, significantly reducing the risk of becoming a victim of fraud.

[1204] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1205] Step 1:

[1206] The server receives emails and SMS messages.

[1207] Input: The content of emails and SMS messages that are sent to users.

[1208] Processing: The server stores the received emails and SMS messages and prepares them for analysis.

[1209] Output: The message data to be parsed.

[1210] Step 2:

[1211] The server analyzes the message content using generated AI methods.

[1212] Input: The message data obtained in step 1.

[1213] Processing: A prompt is input into the generative AI model to evaluate the likelihood of fraud. The specific message content is input into the generative AI model along with the prompt, "Is the following message likely to be fraudulent?"

[1214] Output: Assessment results for likelihood of fraud.

[1215] Step 3:

[1216] Determine if the server is likely to be fraudulent.

[1217] Input: Evaluation results obtained in step 2.

[1218] Processing: If the evaluation results in a high probability of fraud, the server creates an alert.

[1219] Output: Fraud warning message.

[1220] Step 4:

[1221] The server notifies the user with a warning message.

[1222] Input: The fraud warning message created in step 3.

[1223] Processing: Use an SMS gateway or mail server to send a warning message to the user's device.

[1224] Output: A warning message that will be displayed on the user's terminal.

[1225] Step 5:

[1226] The server receives a call from a blocked or unknown phone number.

[1227] Input: Incoming call signal from a blocked or unknown number.

[1228] Processing: Detects an incoming call signal and initiates an automatic response.

[1229] Output: Command to start the auto-reply.

[1230] Step 6:

[1231] The server records the call and analyzes it using generative AI.

[1232] Input: The call content obtained in step 5.

[1233] Processing: The call is recorded using the Twilio API and the recording is fed into the generative AI model along with a prompt: "Do you think the following call is likely to be fraudulent?"

[1234] Output: Assessment results for likelihood of fraud.

[1235] Step 7:

[1236] The server issues a warning to the user if there is a high possibility of fraud.

[1237] Input: Evaluation results obtained in step 6.

[1238] Processing: If a fraudulent activity is deemed likely, a warning message is sent to the user via an SMS gateway or calling system.

[1239] Output: The warning message and audio recording that will be displayed on the user's device.

[1240] Step 8:

[1241] The server collects information about new fraud methods and updates the generative AI model.

[1242] Input: Information on new fraud methods.

[1243] Processing: Retraining the generative AI model based on collected information to improve fraud detection accuracy.

[1244] Output: An updated generative AI model.

[1245] As a result, the system can detect potential fraud in real time and provide appropriate warnings to users.

[1246] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1247] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone calls, and notifies the user with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state. The following describes specific embodiments of the present invention.

[1248] Email and SNS fraud detection function

[1249] System Configuration

[1250] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[1251] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[1252] Emotion engine: Analyzes the user's emotional state and optimizes notification content based on that information.

[1253] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[1254] Program processing overview

[1255] 1. The server receives an email or SNS message.

[1256] 2. Generative AI methods on the server analyze the content and detect possible fraud.

[1257] 3. The emotion engine analyzes the user's emotional state in real time.

[1258] 4. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[1259] 5. The notification means notifies the user of the warning message.

[1260] Specific examples

[1261] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the generation AI means will detect the message and create an alert saying "This may be a scam." The emotion engine will analyze the user's emotional state, and if the user is nervous, it will add a message to the notification such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[1262] Automated telephone answering function

[1263] System Configuration

[1264] Device: Accept calls from blocked or unknown numbers.

[1265] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[1266] Emotion Engine: Analyzes the user's emotional state and optimizes notification content.

[1267] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[1268] Program processing overview

[1269] 1. Your device receives a call from a blocked or unknown number.

[1270] 2. The server initiates an automated response using generated AI means.

[1271] 3. The server records the call, and the generating AI means analyzes the recording.

[1272] 4. The emotion engine analyzes the user's emotional state in real time.

[1273] 5. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[1274] 6. The notification means notifies the user of the warning message and the recorded content.

[1275] Specific examples

[1276] For example, when a device receives a call from an unidentified number, the server uses a generative AI method to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion engine analyzes the user's emotional state and detects that the user is in a tense state. In this case, the server notifies the user by adding a message saying, "Please remain calm." This allows the user to respond calmly.

[1277] Responding to new fraud methods

[1278] System Configuration

[1279] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[1280] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[1281] Emotion engine: Analyzes the user's emotional state in real time and optimizes notification content based on that information.

[1282] Program processing overview

[1283] 1. The server collects information about new fraud methods.

[1284] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[1285] 3. The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud techniques.

[1286] 4. The generative AI method uses the updated model to analyze potential fraud.

[1287] Specific examples

[1288] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it as learning data in the generation AI. Furthermore, the emotion engine analyzes the user's emotional state, allowing it to provide appropriate responses in real time. For example, if a user is nervous about a new fraudulent technique, it will provide a message urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[1289] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[1290] The processing flow will be explained below.

[1291] Email and SNS fraud detection function

[1292] Processing Steps

[1293] Step 1:

[1294] The server receives an email or SNS message and stores the received content and metadata (sender, subject, body) in a database.

[1295] Step 2:

[1296] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[1297] Step 3:

[1298] The generative AI method compares the extracted information with existing databases to detect potentially fraudulent phrases and unreliable URLs, particularly those containing keywords that indicate potential fraud, such as "transfer" or "urgent," as well as suspicious domains.

[1299] Step 4:

[1300] Generative AI methods score the likelihood of fraud, quantifying fraud risk based on extracted information and flagging high risk cases.

[1301] Step 5:

[1302] The emotion engine analyzes the user's emotional state in real time, for example by analyzing the user's facial expressions and voice to detect tension or anxiety.

[1303] Step 6:

[1304] If a fraudulent activity is deemed likely, the emotion engine will create a warning message based on the user's emotional state. For users in a tense state, a message such as "Stay calm and don't react immediately" will be added.

[1305] Step 7:

[1306] The notification method notifies the user of the created alert, sending a warning message via push notification or email to alert the user.

[1307] Automated telephone answering function

[1308] Processing Steps

[1309] Step 1:

[1310] The device receives a call from a blocked or unknown phone number and sends the received call information to the server.

[1311] Step 2:

[1312] The server starts an automated voice response using the AI ​​generation means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[1313] Step 3:

[1314] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[1315] Step 4:

[1316] The generative AI method analyzes the content of the call, checking for the presence of certain keywords and phrases and assessing the likelihood of fraud. For example, phrases such as "transfer money" and "give me your personal information" are recognized as indicators of fraud.

[1317] Step 5:

[1318] The emotion engine analyzes the user's emotional state in real time, analyzing the user's tone of voice and choice of words during a call to detect tension or anxiety.

[1319] Step 6:

[1320] If a fraudulent activity is deemed likely, the emotion engine will create a warning message for users in a state of anxiety, for example adding a message saying "Please stay calm."

[1321] Step 7:

[1322] A notification mechanism will notify the user of the generated alert and recording, and in particularly high-risk cases, notifications will also be sent to the user's next of kin or trusted third parties.

[1323] Responding to new fraud methods

[1324] Processing Steps

[1325] Step 1:

[1326] The server collects information about new fraud methods, deriving data from sources such as police reports and security blogs.

[1327] Step 2:

[1328] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training data and retraining the generative AI model.

[1329] Step 3:

[1330] Deploy new models with updated generative AI methods to update the system to address the latest fraud techniques.

[1331] Step 4:

[1332] The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address new fraud techniques, helping users stay calm even when faced with new fraud methods.

[1333] Step 5:

[1334] The generative AI method uses the updated model to analyze new communications from users, providing analysis capabilities tailored to new fraud techniques and generating appropriate alerts for users.

[1335] Example 2

[1336] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1337] In recent years, fraudulent activities using email, social media, and telephone have been increasing, and their methods have become more diverse and sophisticated. Therefore, users need to be able to quickly and reliably identify fraud risks and take appropriate measures. However, conventional fraud detection systems focus on identifying potential fraud and do not provide flexible responses that take into account the user's emotional state. Furthermore, they face the problem of being difficult to update quickly to respond to new fraud methods. Therefore, a method is needed that enables real-time fraud detection and flexible responses while ensuring user safety.

[1338] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1339] In this invention, the server includes means for receiving and analyzing the contents of electronic messages, means for analyzing the contents of electronic messages and detecting the possibility of fraud, means for analyzing the emotional state of the user and optimizing the notification content, means for notifying the user of a warning message when it is determined that there is a high possibility of fraud, and means for updating the generation AI to respond to new fraud methods. This makes it possible to quickly detect the risk of fraud in messages and calls received by the user and provide an appropriate response according to the user's emotional state.

[1340] "Electronic Message" refers to text messages sent or received over the Internet, such as emails or social media messages.

[1341] "Generative AI methods" refer to algorithms or models that use machine learning or deep learning to analyze text or voice data to determine the likelihood of fraud.

[1342] "Emotion analysis means" refers to technology that analyzes the user's emotional state in real time based on facial expressions and voice data, and optimizes the content of notifications.

[1343] A "warning message" refers to a message that is sent to the user to warn them when it is determined that there is a high possibility of fraud.

[1344] "New fraud methods" refers to newly discovered means and techniques for committing fraud, in addition to traditional fraud methods.

[1345] "Means for receiving and analyzing the content of electronic messages" refers to methods and technologies for retrieving electronic messages from mail servers or social media platforms and analyzing their text content.

[1346] "Generative AI methods for detecting potential fraud" refers to methods that use machine learning and deep learning models to analyze electronic messages and phone calls to identify signs and risks of fraud.

[1347] "Means for analyzing the user's emotional state and optimizing the content of notifications" refers to technology that analyzes the user's psychological state and generates an appropriate response message based on the results.

[1348] "Means for notifying users when it is determined that there is a high possibility of fraud" refers to communication technology for conveying a warning to users based on the analysis results of the generating AI means.

[1349] "Means to update generative AI" refers to technology that retrains generative AI models based on new data and technical information, enabling them to respond to the latest fraud techniques.

[1350] This invention is a system that uses generative AI to detect possible fraud via email, social media, and phone calls, and notifies users with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state.

[1351] Email and SNS fraud detection function

[1352] System Configuration

[1353] Server: Receives emails and social media messages and uses the Gmail API and Twitter API to analyze their contents using generative AI.

[1354] Generative AI method: Using OpenAI's GPT-3 and other technologies, the content of received emails and social media messages is analyzed to detect possible fraud.

[1355] Emotion analysis method: Using Microsoft's Azure Emotion API and other tools, the user's emotional state is analyzed in real time and the content of notifications is optimized based on that information.

[1356] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user via smartphone push notifications or email notifications.

[1357] Program processing overview

[1358] The server analyzes received emails and SNS messages. The generation AI means analyzes the content and detects the possibility of fraud. The emotion analysis means analyzes the user's emotional state, and if it determines that there is a high possibility of fraud, it optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user with a warning message.

[1359] Specific examples

[1360] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the AI ​​generation means will detect the message and create an alert saying "This may be a scam." The emotion analysis means will analyze the user's emotional state, and if the user is nervous, it will add a message such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[1361] Prompt Sentence Examples

[1362] When you receive an email saying "Please transfer money to your bank account urgently," rate it as likely to be a scam. Also, if the user is nervous, add a message saying "Please stay calm and don't respond immediately."

[1363] Automated telephone answering function

[1364] System Configuration

[1365] Device: Use a VoIP service connected to your smartphone or landline to receive calls from blocked or unknown numbers.

[1366] Server: Generates automated voice responses using AI and uses voice recognition services such as Google Dialogflow to record and analyze call content.

[1367] Sentiment analysis: Analyze the user's emotional state in real time using the Amazon Polly API, etc.

[1368] Notification method: If a fraudulent activity is deemed likely, a warning message and recording will be sent to the user via push notification or email notification.

[1369] Program processing overview

[1370] The device receives a call from a blocked or unknown phone number. The server initiates an automated voice response and records the call. The recording is analyzed by the generative AI means to assess the possibility of fraud. At the same time, the emotion analysis means analyzes the user's emotional state in real time and optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user of a warning message and the recording content.

[1371] Specific examples

[1372] For example, when a call comes in from an unidentified number to a terminal, the server automatically responds with "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion analysis means analyzes the user's emotional state and detects that the user is nervous. In this case, the system adds a message to the user saying, "Please remain calm." This allows the user to respond calmly.

[1373] Prompt Sentence Examples

[1374] If you receive a call from an unidentified number, check whether the call contains keywords such as "transfer" or "personal information." If the user is nervous, add a message to the call saying, "Please stay calm."

[1375] Responding to new fraud methods

[1376] System Configuration

[1377] Server: Uses web scraping techniques, news sites, and forum data to gather information on the latest fraudulent techniques and update the generative AI methods.

[1378] Generative AI methods: Retraining machine learning models to constantly update learning data and models based on new fraud techniques.

[1379] Sentiment analysis method: We use the sentiment analysis API to analyze the user's emotional state in real time and optimize the notification content.

[1380] Program processing overview

[1381] The server collects information on new fraud methods and updates the AI ​​generation method. The emotion analysis method analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud methods. The AI ​​generation method uses the updated model to analyze the possibility of fraud.

[1382] Specific examples

[1383] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the generation AI means as learning data. Furthermore, the emotion analysis means analyzes the user's emotional state, allowing for appropriate responses to be provided in real time. For example, if a user is nervous about a new fraudulent technique, a message is displayed urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[1384] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[1385] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1386] Email and SNS fraud detection function

[1387] Processing Steps

[1388] Step 1:

[1389] The server receives electronic messages. Specifically, it periodically checks emails and SNS messages using the Gmail API and Twitter API to retrieve new messages.

[1390] Input: New email or social media message

[1391] Output: Received electronic message

[1392] Step 2:

[1393] The server analyzes the content of the received electronic message using a generative AI method. The generative AI method (e.g., OpenAI GPT-3) is used to analyze the message text and assess its likelihood of fraud.

[1394] Input: Received electronic message

[1395] Data processing / computation: Detecting fraud patterns through natural language processing

[1396] Output: Fraud probability assessment result

[1397] Step 3:

[1398] The server analyzes the user's emotional state using emotion analysis tools, such as Microsoft's Azure Emotion API, which analyzes the user's facial expressions and voice in real time to estimate their emotional state.

[1399] Input: User's facial expression and voice data

[1400] Data processing / computation: Classifying emotional states using emotion recognition models

[1401] Output: User's emotional state

[1402] Step 4:

[1403] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[1404] Input: Fraud probability assessment, user emotional state

[1405] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[1406] Output: personalized warning message

[1407] Step 5:

[1408] The server notifies the user of the warning message. Appropriate warnings are sent to the user via smartphone push notifications or email.

[1409] Input: personalized warning message

[1410] Output: Message notified to the user

[1411] Automated telephone answering function

[1412] Processing Steps

[1413] Step 1:

[1414] Your device receives calls from blocked or unknown numbers. Use a VoIP service connected to your smartphone or landline to monitor whether there are any incoming calls.

[1415] Input: Calls from blocked or unknown numbers

[1416] Output: Incoming call notification

[1417] Step 2:

[1418] The server initiates an automated voice response using a generative AI method, using a speech recognition service such as Google Dialogflow to play a message such as "This call is being recorded, please state your purpose."

[1419] Input: Incoming call notification

[1420] Output: Playback of automated voice message

[1421] Step 3:

[1422] The server records the call, and the AI ​​generator analyzes the recording. The recorded voice data is converted into text using an NLP service and analyzed.

[1423] Input: Recorded call

[1424] Data processing / computation: speech-to-text conversion and text analysis

[1425] Output: Fraud probability assessment result

[1426] Step 4:

[1427] The server analyzes the user's emotional state using emotion analysis tools. It uses the Amazon Polly API to analyze the voice data during the call and evaluates the user's emotional state in real time.

[1428] Input: User's voice data

[1429] Data processing / calculation: Analyzing emotional states from voice data

[1430] Output: User's emotional state

[1431] Step 5:

[1432] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[1433] Input: Fraud probability assessment, user emotional state

[1434] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[1435] Output: personalized warning message

[1436] Step 6:

[1437] The server notifies the user of the warning message and recording contents, and sends appropriate warnings to the user via push notification or email.

[1438] Input: personalized warning message, recording

[1439] Output: Message and recording notified to the user

[1440] Responding to new fraud methods

[1441] Processing Steps

[1442] Step 1:

[1443] The server collects information about new fraud methods, using web scraping technology to retrieve the latest information from security news sites and forums.

[1444] Input: Information about a new fraud scheme

[1445] Output: Collected fraud data

[1446] Step 2:

[1447] The server updates the generative AI method and retrains the generative AI model to adapt to new fraud methods based on the collected information.

[1448] Input: Data on new fraud methods

[1449] Data processing / computation: Retraining AI models

[1450] Output: Updated generative AI model

[1451] Step 3:

[1452] The server uses emotion analysis to analyze the user's emotional state in real time, optimizing alerts to address the latest fraud techniques.

[1453] Input: User emotional state data

[1454] Data processing / calculation: Optimizing notification messages based on emotion data

[1455] Output: Optimized warning message

[1456] Step 4:

[1457] The server uses the updated generative AI model to analyze the possibility of fraud. The latest model analyzes received messages and call content to respond to new fraud methods.

[1458] Input: Received messages and calls

[1459] Data manipulation / calculation: Analysis based on new fraud methods

[1460] Output: Fraud probability assessment result

[1461] These processing steps allow users to proactively avoid the risk of fraud and receive the most appropriate response based on their emotional state.

[1462] (Application example 2)

[1463] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1464] In recent years, fraudulent activities have evolved on a daily basis, requiring immediate and reliable responses to prevent actual damage. However, no systems exist that take into account the user's emotional state, and fraud detection and responses are often inappropriate. In particular, in physical stores, it is difficult for staff to detect fraud while interacting with customers, and it is difficult to make calm decisions when the user is in a tense state. To solve these issues, a system is needed that utilizes wearable devices such as smart glasses to detect possible fraud in real time and provide optimal responses based on the user's emotions.

[1465] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means for analyzing possible fraudulent content, a means for creating an alert and notifying the user when it is determined that fraud is highly likely, a means for updating the generation AI to respond to new fraudulent methods, a means for analyzing the user's emotional state using an emotion engine and optimizing the notification content, and a means for displaying the alert on the smart glasses. This enables fraudulent acts in physical stores to be detected in real time and an optimal response based on emotions. Specifically, when staff are wearing smart glasses, the smart glasses analyze customer behavior and conversation content and display an effective warning message when possible fraud is detected. In addition, instructions are provided according to the staff's emotional state, allowing for a calm and prompt response.

[1466] "Generative AI" is a system that uses artificial intelligence techniques to analyze data and detect specific patterns and anomalies.

[1467] An "emotion engine" is a technology that analyzes a user's emotional state from their voice and images, and is a system that grasps emotions such as stress and tension in real time.

[1468] "Notification means" refers to a system or device that conveys warnings or instructions to users based on information detected by the generative AI or emotion engine.

[1469] "Update methods" are functions that allow systems and models to be updated with the latest data and methods, keeping them up to date at all times.

[1470] "Smart glasses" are wearable devices equipped with a display and communication functions, allowing users to visually receive and integrate information.

[1471] A "server" is a computer system that receives, processes, and transmits data over a network, and is used to manage and operate multiple functions in an integrated manner.

[1472] "Fraud detection" is the process of determining whether certain actions or words are potentially fraudulent, and is performed using generative AI.

[1473] The present invention is a system for detecting possible fraudulent activity in a brick-and-mortar store and providing an optimal response based on the emotional state of the user. Specific embodiments for carrying out the present invention will be described below.

[1474] System Configuration

[1475] 1. Hardware Configuration

[1476] Smart glasses: Equipped with a display, camera, and microphone, worn by staff.

[1477] Server: A computer with high-performance data processing capabilities, primarily responsible for data analysis and running generative AI models.

[1478] Communication method: Uses 5G networks that enable high-speed data communication.

[1479] 2. Software Configuration

[1480] Generative AI models, such as GPT-4 and BERT, can be used to detect potential fraud from audio and video data.

[1481] Emotion Engine: Analyzes staff emotional states in real time using Affectiva and Emotion AI.

[1482] Notification system: Software that notifies staff in real time of warnings and appropriate responses.

[1483] Program processing

[1484] The server receives the video and audio data sent from the smart glasses and analyzes it using a generative AI model. Specifically, it determines whether the customer's behavior or comments indicate the possibility of fraud. If the analyzed data indicates the possibility of fraud, the emotion engine analyzes the staff member's current emotional state. For example, if a staff member is nervous, the notification system will display a message on the smart glasses' display such as, "Please stay calm and treat the customer kindly."

[1485] Specific usage scenarios

[1486] Consider a scenario where a customer visits a brick-and-mortar store and wishes to make a large cash transaction. The smart glasses record the conversation with the customer, and the data is sent to a server. The server's generative AI model analyzes the customer's comments to determine whether they are likely to be fraudulent. For example, if keywords such as "large cash transaction" or "request for personal information" are detected, the emotion engine detects tension in the staff member's heart rate and facial expression. The notification system then displays a message on the smart glasses, such as "This is a possible fraud, so please ask the customer to present identification."

[1487] Prompt Sentence Examples

[1488] Analyze all video and audio recordings containing "large cash transactions" and "requests for personal information" to determine the possibility of fraud.

[1489] This invention enables fast and accurate fraud detection and response in brick-and-mortar stores, ensuring the safety of both customers and staff.

[1490] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1491] Step 1:

[1492] The smart glasses' cameras and microphones capture and record the customer's movements and conversations in real time. The input is the customer's video and audio data. The output is digital data that is sent to the server.

[1493] Step 2:

[1494] The server receives the video and audio data sent from the smart glasses. The input is the video and audio data from the smart glasses. The output is that this data is stored in the server.

[1495] Step 3:

[1496] The generative AI model in the server analyzes the received video and audio data. Specifically, it analyzes the content of the conversation and customer behavior patterns to detect the possibility of fraud. The input is the video and audio data stored in the server. The output is an analysis result indicating the possibility of fraud.

[1497] Step 4:

[1498] The server's emotion engine analyzes the emotional state of staff in real time. This is done by receiving and analyzing biometric information such as the staff's heart rate and facial expressions as input. The input is the staff's biometric information. The output is the analysis result regarding the staff's emotional state.

[1499] Step 5:

[1500] The server combines the analysis results of the generative AI model and the emotion engine and sends them to the notification system. The inputs are the analysis results indicating possible fraud and the analysis results regarding the emotional state of the staff. The output is an optimized warning message.

[1501] Step 6:

[1502] The notification system displays the alert message from the server on the display of the smart glasses. The input is the alert message sent from the server. The output is the specific alert message displayed on the smart glasses.

[1503] Step 7:

[1504] The staff member, who is the user, checks the warning message displayed on the smart glasses and takes appropriate action. The input is the warning message displayed on the smart glasses. The output is the specific action that the staff member takes for the customer.

[1505] These steps enable the system to detect potential fraud in real time in physical stores and provide optimal responses based on the emotional state of staff. For example, effective fraud detection can be achieved by analyzing all video and audio data containing "large cash transactions" and "requests for personal information" into a generative AI model and inputting prompt sentences to determine the likelihood of fraud.

[1506] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1507] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1508] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1509] [Fourth embodiment]

[1510] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1511] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1512] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1513] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1514] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1515] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1516] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1517] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1518] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1519] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1520] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1521] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1522] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1523] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone, and notifies users with appropriate alerts. Specific embodiments of this system are described in detail below.

[1524] Email and SNS fraud detection function

[1525] System Configuration

[1526] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[1527] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[1528] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[1529] Program processing overview

[1530] 1. The server receives an email or SNS message.

[1531] 2. The generated AI means on the server analyzes the content.

[1532] 3. Detect potentially fraudulent claims and unreliable URLs.

[1533] 4. Generate a warning message and notify the user by means of creating an alert and notifying the user.

[1534] Specific examples

[1535] For example, if the server receives an email with the content "Please transfer the money to my bank account immediately," the AI ​​generator will detect the phrase, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[1536] Automated telephone answering function

[1537] System Configuration

[1538] Device: Accept calls from blocked or unknown numbers.

[1539] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[1540] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[1541] Program processing overview

[1542] 1. Your device receives a call from a blocked or unknown number.

[1543] 2. The server initiates an automated response using generated AI means.

[1544] 3. The server records the call, and the generating AI means analyzes the recording.

[1545] 4. A warning message is generated by the means for creating an alert and notifying the user, and the user is notified along with the recorded content.

[1546] Specific examples

[1547] For example, when a call comes in from an unidentified number to a device, the server uses AI generation to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is then analyzed, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and is notified to the user along with the recorded data. This allows the user to respond calmly.

[1548] Responding to new fraud methods

[1549] System Configuration

[1550] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[1551] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[1552] Program processing overview

[1553] 1. The server collects information about new fraud methods.

[1554] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[1555] 3. The generative AI method analyzes the updated model for potential fraud.

[1556] Specific examples

[1557] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the AI ​​generation means as learning data. This allows the AI ​​generation means to respond to the latest fraud techniques and provide accurate alerts to users.

[1558] In this way, it is possible to provide a system that reduces the risk of fraud and protects users from fraud.

[1559] The processing flow will be explained below.

[1560] Email and SNS fraud detection function

[1561] Processing Steps

[1562] Step 1:

[1563] The server receives an email or SNS message and stores the message content and metadata (sender, subject, body) in a database.

[1564] Step 2:

[1565] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[1566] Step 3:

[1567] The extracted information is compared with existing databases to detect potentially fraudulent phrases and unreliable URLs, such as keywords like "transfer" or "urgent," or suspicious domains.

[1568] Step 4:

[1569] Scoring the likelihood of fraud: The generative AI method determines the likelihood of fraud based on the extracted information and assigns a score.

[1570] Step 5:

[1571] If the method for creating an alert and notifying the user determines that the email is likely to be fraudulent, a warning message will be generated. Specifically, an alert message such as "This email may be fraudulent" will be generated.

[1572] Step 6:

[1573] The notification means notifies the user of the alert. Possible notification methods include push notifications and emails.

[1574] Automated telephone answering function

[1575] Processing Steps

[1576] Step 1:

[1577] The device receives a call from a blocked or unknown phone number and sends the received phone number information to the server.

[1578] Step 2:

[1579] The server starts an automated voice response using the generation AI means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[1580] Step 3:

[1581] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[1582] Step 4:

[1583] The generative AI method analyzes the content of the call, checking for the presence of specific keywords and phrases, and scoring the likelihood of fraud. For example, keywords such as "transfer money" and "give me your personal information" indicate a potential fraud.

[1584] Step 5:

[1585] If the means for creating an alert and notifying the user determines that there is a high possibility of fraud, a warning message will be generated. An alert message such as "Possible fraud" will be generated along with the recorded content.

[1586] Step 6:

[1587] The notification means notifies the user of the alert and the recorded content, allowing the user to respond calmly.

[1588] Responding to new fraud methods

[1589] Processing Steps

[1590] Step 1:

[1591] The server collects information about new fraud methods, possibly from police reports or security blogs.

[1592] Step 2:

[1593] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training dataset and retraining the generative AI model.

[1594] Step 3:

[1595] The server deploys the updated generative AI method, updating the system-wide AI model to respond to new fraudulent techniques.

[1596] Step 4:

[1597] The generative AI method uses the updated model to analyze potential fraud, providing analysis capabilities that are in line with new fraud methods, making it possible to respond to the latest fraud techniques.

[1598] In this way, the present invention aims to enable users to avoid the risk of fraud in advance and provide a safe communication environment.

[1599] Example 1

[1600] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1601] Fraudulent activities are on the rise in communications and phone calls over the Internet, increasing the risk of many users becoming victims. However, because it is difficult to sufficiently reduce these risks using conventional methods, there is a need for technology that can detect potential fraud with high accuracy and notify users promptly.

[1602] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1603] In this invention, the server includes means for receiving emails and messages from social networking services, means for analyzing the content with a generation AI means, means for detecting potentially fraudulent wording and unreliable URLs, means for creating an alert and notifying the user, means for receiving calls from withheld or unknown phone numbers and responding with an automated voice with the generation AI means, recording and analyzing the content of the call, and means for collecting information on new fraud methods and updating the generation AI means. This makes it possible to detect possible fraud with high accuracy and notify the user promptly.

[1604] A "server" is a central processing unit that receives, analyzes, stores, and notifies users of emails and messages from social networking services.

[1605] "Generative AI means" refers to an artificial intelligence model and its processing means for analyzing received content and determining the possibility of fraud.

[1606] "Notification means" refers to a device or method that creates an alert message and sends a warning to the user by email, SMS, push notification, or other means.

[1607] The "message receiving means" is a function that receives messages from emails and social networking services and transmits them to the server.

[1608] "Analysis means" refers to the process of analyzing received messages and call content using the generation AI means.

[1609] "Detection means" is a function that extracts potentially fraudulent wording and unreliable URLs from the analyzed content.

[1610] An "alert generator" is a function that generates a warning message to the user when a possible fraud is detected.

[1611] "Call answering means" is a function in which the server uses AI generation means to respond with an automated voice to calls from anonymous or unknown phone numbers.

[1612] The "call recording means" is a function that records the contents of a call in real time and saves the recorded data.

[1613] "Information gathering means" refers to the function of gathering information about new fraud methods from the Internet and specialized institutions.

[1614] The "update method" is a function that reflects collected information on new fraud methods and retrains the generation AI method.

[1615] The present invention is a system that utilizes generative AI to detect possible fraud occurring through email, messages on social networking services, and telephone calls, and notifies users with appropriate alerts. Specific embodiments for implementing the present invention are described in detail below.

[1616] Email and SNS fraud detection function

[1617] System Configuration

[1618] This system is configured as follows:

[1619] Server: Receives emails and messages from social networking services and analyzes their contents using generative AI.

[1620] Generative AI means: Analyzes the content of received messages to detect potential fraud, using generative AI models such as GPT-3.

[1621] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[1622] operation

[1623] The server receives messages from email or social networking services (e.g., Gmail, Facebook). After receiving the messages, it uses a generative AI method (e.g., GPT-3) to analyze the message content. The generative AI analyzes the message's context and keywords based on the prompt text and assesses the likelihood of fraud.

[1624] For example, if an email with the content "Please transfer the money to my bank account urgently" arrives at the server, the AI ​​generator will detect the wording, create an alert saying "This may be a scam," and send it to the user's smartphone. This allows the user to be cautious before opening the email.

[1625] Prompt Sentence Examples

[1626] Analyze whether this email is a scam: "Please transfer the money to my bank account immediately."

[1627] Automated telephone answering function

[1628] System Configuration

[1629] Device: Accept calls from blocked or unknown numbers.

[1630] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[1631] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[1632] operation

[1633] When the device receives a call from a blocked or unknown phone number, the server uses a generation AI method to respond with an automated voice. For example, it may respond with, "This call is being recorded. Please explain your purpose." The call is then recorded and analyzed by the generation AI method. If the call contains keywords such as "transfer" or "personal information," an alert is generated stating, "This may be a scam," and the user is notified along with the recorded data. This allows the user to respond calmly.

[1634] Prompt Sentence Examples

[1635] Analyze whether this call is a scam: "This call is being recorded. Please explain your purpose."

[1636] Responding to new fraud methods

[1637] System Configuration

[1638] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[1639] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[1640] operation

[1641] The server periodically collects information on the latest fraud techniques. Based on information from the internet and specialized institutions, the generation AI means is retrained and updated. For example, if a new "QR code fraud" is discovered, that information is collected and reflected in the generation AI means as learning data. This allows the generation AI means to respond to the latest fraud techniques and provide accurate alerts to users.

[1642] Prompt Sentence Examples

[1643] Analyze information about new fraud methods and update your AI models to respond: "QR code fraud has been discovered."

[1644] In this way, it is possible to detect possible fraud with high accuracy and notify the user promptly, thereby providing a system that can reduce the risk of fraud and ensure the safety of users.

[1645] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1646] Email and SNS fraud detection function

[1647] Step 1:

[1648] The server receives messages from email or social networking services. The server retrieves message data from external messaging services using a specified API or protocol (e.g., IMAP, Graph API). The input to this process is the message data received from email or social networking services, and the output is the storage of the received messages.

[1649] Step 2:

[1650] The server analyzes the content using generative AI means. The server inputs the received message into a generative AI model (e.g., GPT-3). It generates a prompt (e.g., "Please analyze whether this email is fraudulent: 'Please transfer the money to my bank account quickly.'") and passes it to the generative AI model. The input to this process is the content of the received message, and the output is the generative AI model's assessment of the likelihood of fraud.

[1651] Step 3:

[1652] The server detects potentially fraudulent text and unreliable URLs. Based on the analysis results from the generative AI model, the server detects specific keywords and phrases (e.g., "transfer" or "bank account"). It also checks the reliability of URLs and whether they are blacklisted. The input to this process is the evaluation result of the generative AI model, and the output is a list of text and URLs that are deemed to be potentially fraudulent.

[1653] Step 4:

[1654] The server creates an alert and notifies the user via a notification mechanism. If the server determines that there is a high possibility of fraud, it generates a warning message. It sends the alert to the user via a notification mechanism (email, SMS, push notification, etc.). The input to this process is the data that has been determined to be potentially fraudulent, and the output is a warning message that is sent to the user.

[1655] Automated telephone answering function

[1656] Step 1:

[1657] The terminal receives an incoming call from a blocked or unknown phone number. When the terminal detects the incoming call, it sends the information to the server. The input of this process is the incoming call notification, and the output is the incoming call information sent to the server.

[1658] Step 2:

[1659] The server uses the generation AI means to initiate an automatic response. The server uses the generation AI means to respond with an automated voice saying, "This call is being recorded. Please explain your purpose." The input of this process is the incoming call information, and the output is an automated voice message.

[1660] Step 3:

[1661] The server records the call content, and the generative AI means analyzes the recording. The server records the call content in real time and inputs the recording data into the generative AI model. The generative AI model analyzes specific keywords and phrases and evaluates the likelihood of fraud. The input to this process is the recorded call content, and the output is the evaluation result obtained from the generative AI model.

[1662] Step 4:

[1663] The server creates an alert and notifies the user via a notification method. If it determines that there is a high possibility of fraud, the server creates a warning message and notifies the user along with the recorded data. The input to this process is the evaluation result of the generative AI model, and the output is the warning message and recorded data sent to the user.

[1664] Responding to new fraud methods

[1665] Step 1:

[1666] The server collects information about new fraud methods. The server periodically collects information about the latest fraud methods from the Internet and specialized organizations. The input of this process is information collected from outside, and the output is information about fraud methods stored in an internal database.

[1667] Step 2:

[1668] The server updates the generative AI means. Based on the collected information, the server updates the learning dataset of the generative AI means and retrains it. The input of this process is the collected information on fraudulent techniques, and the output is an updated generative AI model.

[1669] Step 3:

[1670] The server performs the analysis using the updated generative AI model. The server then re-analyzes the email and call content using the updated AI model. The input to this process is the data to be analyzed based on the new fraud technique, and the output is the latest evaluation result.

[1671] These specific processing steps enable the system to detect potential fraud with high accuracy and notify the user promptly.

[1672] (Application example 1)

[1673] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1674] There is a problem that it is difficult for users to prevent fraudulent communications from emails, SNS messages, and calls from anonymous or unknown phone numbers. Current technology lacks the means to efficiently detect these fraudulent communications and quickly warn users, which means that users are unable to take appropriate precautions.

[1675] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1676] In this invention, the server includes a generating AI means for analyzing content that may be fraudulent, a means for creating an alert and notifying the user if it is determined that there is a high possibility of fraud, a means for updating the generating AI to respond to new fraudulent methods, a means for analyzing emails and SMS messages in real time, a means for automatically answering, recording, and analyzing telephone calls and sending an immediate alert if there is a possibility of fraud, a means for analyzing the content of emails and telephone calls that may be fraudulent based on a generating AI model, and a means for sending notifications in real time. This makes it possible to detect possible fraud occurring through emails, SNS messages, and telephone calls and to send warnings to users quickly and accurately.

[1677] A "server" is a computer system that uses generative AI means to analyze emails, social media messages, and phone calls, detect potential fraud, and respond to new fraud methods.

[1678] "Generative AI means" refers to a function that uses a generative AI model to analyze the content of emails, social media messages, and phone calls to detect possible fraud.

[1679] "Means for creating alerts and notifying users" refers to a function that immediately generates and sends a warning message to users when it is determined that there is a high possibility of fraud.

[1680] "Means for updating" refers to the ability to update the learning data and algorithms of the generative AI model to respond to new fraudulent methods.

[1681] "Means for analyzing emails and SMS messages in real time" refers to a function that instantly analyzes received emails and SMS messages using AI-generated means to determine whether they are fraudulent.

[1682] "Means for automatically answering, recording and analyzing calls, and sending immediate alerts in the event of a possible fraud" refers to a function that automatically answers calls from anonymous or unknown phone numbers, records and analyzes the content of the call, and sends a warning to the user if it is determined that there is a high possibility of fraud.

[1683] "Generative AI Model" means a machine learning algorithm utilized in a generative AI method, and is a trained model used to detect potential fraud.

[1684] A "prompt sentence" is text data input to a generative AI model, and is the sentence that is analyzed to determine the possibility of fraud.

[1685] This invention is a system that uses generative AI to detect potential fraudulent activity occurring through email, social media messages, and phone calls, and quickly and accurately notifies users of the alert. Specific embodiments of the invention are described in detail below.

[1686] System Configuration

[1687] The system consists of the following major components:

[1688] 1. Server: Analyzes emails, social media messages, and phone calls using generative AI methods to detect potential fraud. Also, updates the generative AI model to adapt to new fraud methods.

[1689] 2. Generative AI methods: These are machine learning models that analyze the content of emails, social media messages, and phone calls to determine the likelihood of fraud. Generative AI methods are implemented using the OpenAI API, among other things.

[1690] 3. Notification method: The method by which the user will be alerted, which may include SMS, email, or phone notification.

[1691] 4. Automated Telephone Answering System: Uses the Twilio API to answer calls from anonymous or unknown phone numbers.

[1692] Email and SNS message analysis

[1693] The server analyzes received emails and SNS messages using the AI ​​generation means. For example, if the email content contains a phrase such as "Please transfer the money to my bank account immediately," the AI ​​generation means will determine that it is "possibly fraudulent," and an alert will be immediately created to notify the user. This will allow the user to be on guard. An example of a prompt sentence is as follows: "Do you think the following message is potentially fraudulent?" (followed by the specific email content).

[1694] Automated call answering and analysis

[1695] When the server receives a call from an unidentified or unknown phone number, it uses the Twilio API to initiate an automatic response. For example, a message saying "This call is being recorded, please state your purpose" is played, and then the call is recorded. A generating AI method analyzes the recording, and if it contains keywords such as "transfer" or "personal information," an alert is generated stating "Possible fraud" and notifying the user. An example of a prompt sentence is as follows: "Do you think the following call content is likely to be fraudulent?" (followed by the specific recording content).

[1696] Responding to new fraud methods

[1697] The server collects information about new fraud methods and periodically updates the generative AI model. For example, if a new "QR code fraud" is discovered, that information is reflected in the generative AI model to improve its response capabilities. This allows users to be alerted to the latest fraud methods.

[1698] Hardware and software used

[1699] Specifically, the server uses a high-performance computer system, the AI ​​generation method uses OpenAI's API, the telephone answering system incorporates Twilio's API, and notification methods use common SMS gateways and email servers.

[1700] As a result, this system can efficiently detect potential fraudulent activity via email, telephone, or social media, and quickly send warnings to users, significantly reducing the risk of becoming a victim of fraud.

[1701] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1702] Step 1:

[1703] The server receives emails and SMS messages.

[1704] Input: The content of emails and SMS messages that are sent to users.

[1705] Processing: The server stores the received emails and SMS messages and prepares them for analysis.

[1706] Output: The message data to be parsed.

[1707] Step 2:

[1708] The server analyzes the message content using generated AI methods.

[1709] Input: The message data obtained in step 1.

[1710] Processing: A prompt is input into the generative AI model to evaluate the likelihood of fraud. The specific message content is input into the generative AI model along with the prompt, "Is the following message likely to be fraudulent?"

[1711] Output: Assessment results for likelihood of fraud.

[1712] Step 3:

[1713] Determine if the server is likely to be fraudulent.

[1714] Input: Evaluation results obtained in step 2.

[1715] Processing: If the evaluation results in a high probability of fraud, the server creates an alert.

[1716] Output: Fraud warning message.

[1717] Step 4:

[1718] The server notifies the user with a warning message.

[1719] Input: The fraud warning message created in step 3.

[1720] Processing: Use an SMS gateway or mail server to send a warning message to the user's device.

[1721] Output: A warning message that will be displayed on the user's terminal.

[1722] Step 5:

[1723] The server receives a call from a blocked or unknown phone number.

[1724] Input: Incoming call signal from a blocked or unknown number.

[1725] Processing: Detects an incoming call signal and initiates an automatic response.

[1726] Output: Command to start the auto-reply.

[1727] Step 6:

[1728] The server records the call and analyzes it using generative AI.

[1729] Input: The call content obtained in step 5.

[1730] Processing: The call is recorded using the Twilio API and the recording is fed into the generative AI model along with a prompt: "Do you think the following call is likely to be fraudulent?"

[1731] Output: Assessment results for likelihood of fraud.

[1732] Step 7:

[1733] The server issues a warning to the user if there is a high possibility of fraud.

[1734] Input: Evaluation results obtained in step 6.

[1735] Processing: If a fraudulent activity is deemed likely, a warning message is sent to the user via an SMS gateway or calling system.

[1736] Output: The warning message and audio recording that will be displayed on the user's device.

[1737] Step 8:

[1738] The server collects information about new fraud methods and updates the generative AI model.

[1739] Input: Information on new fraud methods.

[1740] Processing: Retraining the generative AI model based on collected information to improve fraud detection accuracy.

[1741] Output: An updated generative AI model.

[1742] As a result, the system can detect potential fraud in real time and provide appropriate warnings to users.

[1743] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1744] The present invention is a system that uses generative AI to detect possible fraud occurring through email, social media, and telephone calls, and notifies the user with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state. The following describes specific embodiments of the present invention.

[1745] Email and SNS fraud detection function

[1746] System Configuration

[1747] Server: Receives emails and social media messages and analyzes their contents using generative AI.

[1748] Generative AI method: Analyzes the content of received emails and social media messages to detect possible fraud.

[1749] Emotion engine: Analyzes the user's emotional state and optimizes notification content based on that information.

[1750] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user.

[1751] Program processing overview

[1752] 1. The server receives an email or SNS message.

[1753] 2. Generative AI methods on the server analyze the content and detect possible fraud.

[1754] 3. The emotion engine analyzes the user's emotional state in real time.

[1755] 4. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[1756] 5. The notification means notifies the user of the warning message.

[1757] Specific examples

[1758] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the generation AI means will detect the message and create an alert saying "This may be a scam." The emotion engine will analyze the user's emotional state, and if the user is nervous, it will add a message to the notification such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[1759] Automated telephone answering function

[1760] System Configuration

[1761] Device: Accept calls from blocked or unknown numbers.

[1762] Server: Generates automated voice responses using AI, and records and analyzes the content of calls.

[1763] Emotion Engine: Analyzes the user's emotional state and optimizes notification content.

[1764] Notification method: If fraud is deemed likely, the user will be notified with a warning message and recording.

[1765] Program processing overview

[1766] 1. Your device receives a call from a blocked or unknown number.

[1767] 2. The server initiates an automated response using generated AI means.

[1768] 3. The server records the call, and the generating AI means analyzes the recording.

[1769] 4. The emotion engine analyzes the user's emotional state in real time.

[1770] 5. If fraud is deemed likely, optimize the notification content based on the user's emotional state.

[1771] 6. The notification means notifies the user of the warning message and the recorded content.

[1772] Specific examples

[1773] For example, when a device receives a call from an unidentified number, the server uses a generative AI method to automatically respond with, "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion engine analyzes the user's emotional state and detects that the user is in a tense state. In this case, the server notifies the user by adding a message saying, "Please remain calm." This allows the user to respond calmly.

[1774] Responding to new fraud methods

[1775] System Configuration

[1776] Server: Collects information on the latest fraud techniques and updates the generative AI methods.

[1777] Generative AI methods: Constantly updating training data and models based on new fraud techniques.

[1778] Emotion engine: Analyzes the user's emotional state in real time and optimizes notification content based on that information.

[1779] Program processing overview

[1780] 1. The server collects information about new fraud methods.

[1781] 2. The server updates the AI ​​generation methods to adapt to new fraudulent methods.

[1782] 3. The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud techniques.

[1783] 4. The generative AI method uses the updated model to analyze potential fraud.

[1784] Specific examples

[1785] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it as learning data in the generation AI. Furthermore, the emotion engine analyzes the user's emotional state, allowing it to provide appropriate responses in real time. For example, if a user is nervous about a new fraudulent technique, it will provide a message urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[1786] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[1787] The processing flow will be explained below.

[1788] Email and SNS fraud detection function

[1789] Processing Steps

[1790] Step 1:

[1791] The server receives an email or SNS message and stores the received content and metadata (sender, subject, body) in a database.

[1792] Step 2:

[1793] The server-based AI analyzes the received message and uses natural language processing technology to extract important phrases and links from the text.

[1794] Step 3:

[1795] The generative AI method compares the extracted information with existing databases to detect potentially fraudulent phrases and unreliable URLs, particularly those containing keywords that indicate potential fraud, such as "transfer" or "urgent," as well as suspicious domains.

[1796] Step 4:

[1797] Generative AI methods score the likelihood of fraud, quantifying fraud risk based on extracted information and flagging high risk cases.

[1798] Step 5:

[1799] The emotion engine analyzes the user's emotional state in real time, for example by analyzing the user's facial expressions and voice to detect tension or anxiety.

[1800] Step 6:

[1801] If a fraudulent activity is deemed likely, the emotion engine will create a warning message based on the user's emotional state. For users in a tense state, a message such as "Stay calm and don't react immediately" will be added.

[1802] Step 7:

[1803] The notification method notifies the user of the created alert, sending a warning message via push notification or email to alert the user.

[1804] Automated telephone answering function

[1805] Processing Steps

[1806] Step 1:

[1807] The device receives a call from a blocked or unknown phone number and sends the received call information to the server.

[1808] Step 2:

[1809] The server starts an automated voice response using the AI ​​generation means, which plays a voice message saying, "This call will be automatically recorded. Please tell us your business."

[1810] Step 3:

[1811] The server records the call, and the recorded data is analyzed in real time by a generating AI tool.

[1812] Step 4:

[1813] The generative AI method analyzes the content of the call, checking for the presence of certain keywords and phrases and assessing the likelihood of fraud. For example, phrases such as "transfer money" and "give me your personal information" are recognized as indicators of fraud.

[1814] Step 5:

[1815] The emotion engine analyzes the user's emotional state in real time, analyzing the user's tone of voice and choice of words during a call to detect tension or anxiety.

[1816] Step 6:

[1817] If a fraudulent activity is deemed likely, the emotion engine will create a warning message for users in a state of anxiety, for example adding a message saying "Please stay calm."

[1818] Step 7:

[1819] A notification mechanism will notify the user of the generated alert and recording, and in particularly high-risk cases, notifications will also be sent to the user's next of kin or trusted third parties.

[1820] Responding to new fraud methods

[1821] Processing Steps

[1822] Step 1:

[1823] The server collects information about new fraud methods, deriving data from sources such as police reports and security blogs.

[1824] Step 2:

[1825] The server updates the generative AI methods based on the collected information, adding new fraud techniques to the training data and retraining the generative AI model.

[1826] Step 3:

[1827] Deploy new models with updated generative AI methods to update the system to address the latest fraud techniques.

[1828] Step 4:

[1829] The emotion engine analyzes the user's emotional state in real time and optimizes alerts to address new fraud techniques, helping users stay calm even when faced with new fraud methods.

[1830] Step 5:

[1831] The generative AI method uses the updated model to analyze new communications from users, providing analysis capabilities tailored to new fraud techniques and generating appropriate alerts for users.

[1832] Example 2

[1833] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1834] In recent years, fraudulent activities using email, social media, and telephone have been increasing, and their methods have become more diverse and sophisticated. Therefore, users need to be able to quickly and reliably identify fraud risks and take appropriate measures. However, conventional fraud detection systems focus on identifying potential fraud and do not provide flexible responses that take into account the user's emotional state. Furthermore, they face the problem of being difficult to update quickly to respond to new fraud methods. Therefore, a method is needed that enables real-time fraud detection and flexible responses while ensuring user safety.

[1835] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1836] In this invention, the server includes means for receiving and analyzing the contents of electronic messages, means for analyzing the contents of electronic messages and detecting the possibility of fraud, means for analyzing the emotional state of the user and optimizing the notification content, means for notifying the user of a warning message when it is determined that there is a high possibility of fraud, and means for updating the generation AI to respond to new fraud methods. This makes it possible to quickly detect the risk of fraud in messages and calls received by the user and provide an appropriate response according to the user's emotional state.

[1837] "Electronic Message" refers to text messages sent or received over the Internet, such as emails or social media messages.

[1838] "Generative AI methods" refer to algorithms or models that use machine learning or deep learning to analyze text or voice data to determine the likelihood of fraud.

[1839] "Emotion analysis means" refers to technology that analyzes the user's emotional state in real time based on facial expressions and voice data, and optimizes the content of notifications.

[1840] A "warning message" refers to a message that is sent to the user to warn them when it is determined that there is a high possibility of fraud.

[1841] "New fraud methods" refers to newly discovered means and techniques for committing fraud, in addition to traditional fraud methods.

[1842] "Means for receiving and analyzing the content of electronic messages" refers to methods and technologies for retrieving electronic messages from mail servers or social media platforms and analyzing their text content.

[1843] "Generative AI methods for detecting potential fraud" refers to methods that use machine learning and deep learning models to analyze electronic messages and phone calls to identify signs and risks of fraud.

[1844] "Means for analyzing the user's emotional state and optimizing the content of notifications" refers to technology that analyzes the user's psychological state and generates an appropriate response message based on the results.

[1845] "Means for notifying users when it is determined that there is a high possibility of fraud" refers to communication technology for conveying a warning to users based on the analysis results of the generating AI means.

[1846] "Means to update generative AI" refers to technology that retrains generative AI models based on new data and technical information, enabling them to respond to the latest fraud techniques.

[1847] This invention is a system that uses generative AI to detect possible fraud via email, social media, and phone calls, and notifies users with appropriate alerts. Furthermore, by combining it with an emotion engine that analyzes the user's emotions, it realizes optimal responses based on the user's emotional state.

[1848] Email and SNS fraud detection function

[1849] System Configuration

[1850] Server: Receives emails and social media messages and uses the Gmail API and Twitter API to analyze their contents using generative AI.

[1851] Generative AI method: Using OpenAI's GPT-3 and other technologies, the content of received emails and social media messages is analyzed to detect possible fraud.

[1852] Emotion analysis method: Using Microsoft's Azure Emotion API and other tools, the user's emotional state is analyzed in real time and the content of notifications is optimized based on that information.

[1853] Notification method: If a fraudulent activity is deemed likely, a warning message will be sent to the user via smartphone push notifications or email notifications.

[1854] Program processing overview

[1855] The server analyzes received emails and SNS messages. The generation AI means analyzes the content and detects the possibility of fraud. The emotion analysis means analyzes the user's emotional state, and if it determines that there is a high possibility of fraud, it optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user with a warning message.

[1856] Specific examples

[1857] For example, if the server receives an email with the message "Please transfer the money to my bank account quickly," the AI ​​generation means will detect the message and create an alert saying "This may be a scam." The emotion analysis means will analyze the user's emotional state, and if the user is nervous, it will add a message such as "Please stay calm and do not respond immediately." This will allow the user to take appropriate action.

[1858] Prompt Sentence Examples

[1859] When you receive an email saying "Please transfer money to your bank account urgently," rate it as likely to be a scam. Also, if the user is nervous, add a message saying "Please stay calm and don't respond immediately."

[1860] Automated telephone answering function

[1861] System Configuration

[1862] Device: Use a VoIP service connected to your smartphone or landline to receive calls from blocked or unknown numbers.

[1863] Server: Generates automated voice responses using AI and uses voice recognition services such as Google Dialogflow to record and analyze call content.

[1864] Sentiment analysis: Analyze the user's emotional state in real time using the Amazon Polly API, etc.

[1865] Notification method: If a fraudulent activity is deemed likely, a warning message and recording will be sent to the user via push notification or email notification.

[1866] Program processing overview

[1867] The device receives a call from a blocked or unknown phone number. The server initiates an automated voice response and records the call. The recording is analyzed by the generative AI means to assess the possibility of fraud. At the same time, the emotion analysis means analyzes the user's emotional state in real time and optimizes the notification content according to the user's emotional state. Finally, the notification means notifies the user of a warning message and the recording content.

[1868] Specific examples

[1869] For example, when a call comes in from an unidentified number to a terminal, the server automatically responds with "This call is being recorded. Please explain your purpose." The content of the call is analyzed, and if the call contains keywords such as "transfer" or "personal information," the emotion analysis means analyzes the user's emotional state and detects that the user is nervous. In this case, the system adds a message to the user saying, "Please remain calm." This allows the user to respond calmly.

[1870] Prompt Sentence Examples

[1871] If you receive a call from an unidentified number, check whether the call contains keywords such as "transfer" or "personal information." If the user is nervous, add a message to the call saying, "Please stay calm."

[1872] Responding to new fraud methods

[1873] System Configuration

[1874] Server: Uses web scraping techniques, news sites, and forum data to gather information on the latest fraudulent techniques and update the generative AI methods.

[1875] Generative AI methods: Retraining machine learning models to constantly update learning data and models based on new fraud techniques.

[1876] Sentiment analysis method: We use the sentiment analysis API to analyze the user's emotional state in real time and optimize the notification content.

[1877] Program processing overview

[1878] The server collects information on new fraud methods and updates the AI ​​generation method. The emotion analysis method analyzes the user's emotional state in real time and optimizes alerts to address the latest fraud methods. The AI ​​generation method uses the updated model to analyze the possibility of fraud.

[1879] Specific examples

[1880] For example, if a new "QR code fraud" is discovered, the server collects that information and reflects it in the generation AI means as learning data. Furthermore, the emotion analysis means analyzes the user's emotional state, allowing for appropriate responses to be provided in real time. For example, if a user is nervous about a new fraudulent technique, a message is displayed urging them to remain calm, warning them that "this link may be fraudulent, so do not click."

[1881] In this way, the present invention allows users to avoid the risk of fraud in advance and provides optimal responses according to their emotional state, thereby realizing a safe communication environment.

[1882] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1883] Email and SNS fraud detection function

[1884] Processing Steps

[1885] Step 1:

[1886] The server receives electronic messages. Specifically, it periodically checks emails and SNS messages using the Gmail API and Twitter API to retrieve new messages.

[1887] Input: New email or social media message

[1888] Output: Received electronic message

[1889] Step 2:

[1890] The server analyzes the content of the received electronic message using a generative AI method. The generative AI method (e.g., OpenAI GPT-3) is used to analyze the message text and assess its likelihood of fraud.

[1891] Input: Received electronic message

[1892] Data processing / computation: Detecting fraud patterns through natural language processing

[1893] Output: Fraud probability assessment result

[1894] Step 3:

[1895] The server analyzes the user's emotional state using emotion analysis tools, such as Microsoft's Azure Emotion API, which analyzes the user's facial expressions and voice in real time to estimate their emotional state.

[1896] Input: User's facial expression and voice data

[1897] Data processing / computation: Classifying emotional states using emotion recognition models

[1898] Output: User's emotional state

[1899] Step 4:

[1900] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[1901] Input: Fraud probability assessment, user emotional state

[1902] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[1903] Output: personalized warning message

[1904] Step 5:

[1905] The server notifies the user of the warning message. Appropriate warnings are sent to the user via smartphone push notifications or email.

[1906] Input: personalized warning message

[1907] Output: Message notified to the user

[1908] Automated telephone answering function

[1909] Processing Steps

[1910] Step 1:

[1911] Your device receives calls from blocked or unknown numbers. Use a VoIP service connected to your smartphone or landline to monitor whether there are any incoming calls.

[1912] Input: Calls from blocked or unknown numbers

[1913] Output: Incoming call notification

[1914] Step 2:

[1915] The server initiates an automated voice response using a generative AI method, using a speech recognition service such as Google Dialogflow to play a message such as "This call is being recorded, please state your purpose."

[1916] Input: Incoming call notification

[1917] Output: Playback of automated voice message

[1918] Step 3:

[1919] The server records the call, and the AI ​​generator analyzes the recording. The recorded voice data is converted into text using an NLP service and analyzed.

[1920] Input: Recorded call

[1921] Data processing / computation: speech-to-text conversion and text analysis

[1922] Output: Fraud probability assessment result

[1923] Step 4:

[1924] The server analyzes the user's emotional state using emotion analysis tools. It uses the Amazon Polly API to analyze the voice data during the call and evaluates the user's emotional state in real time.

[1925] Input: User's voice data

[1926] Data processing / calculation: Analyzing emotional states from voice data

[1927] Output: User's emotional state

[1928] Step 5:

[1929] The server optimizes the notification content based on the fraud probability assessment result and the user's emotional state. If it determines that there is a high probability of fraud, it creates a warning message according to the user's emotional state.

[1930] Input: Fraud probability assessment, user emotional state

[1931] Data processing / calculation: Generate notification messages based on evaluation results and emotion data

[1932] Output: personalized warning message

[1933] Step 6:

[1934] The server notifies the user of the warning message and recording contents, and sends appropriate warnings to the user via push notification or email.

[1935] Input: personalized warning message, recording

[1936] Output: Message and recording notified to the user

[1937] Responding to new fraud methods

[1938] Processing Steps

[1939] Step 1:

[1940] The server collects information about new fraud methods, using web scraping technology to retrieve the latest information from security news sites and forums.

[1941] Input: Information about a new fraud scheme

[1942] Output: Collected fraud data

[1943] Step 2:

[1944] The server updates the generative AI method and retrains the generative AI model to adapt to new fraud methods based on the collected information.

[1945] Input: Data on new fraud methods

[1946] Data processing / computation: Retraining AI models

[1947] Output: Updated generative AI model

[1948] Step 3:

[1949] The server uses emotion analysis to analyze the user's emotional state in real time, optimizing alerts to address the latest fraud techniques.

[1950] Input: User emotional state data

[1951] Data processing / calculation: Optimizing notification messages based on emotion data

[1952] Output: Optimized warning message

[1953] Step 4:

[1954] The server uses the updated generative AI model to analyze the possibility of fraud. The latest model analyzes received messages and call content to respond to new fraud methods.

[1955] Input: Received messages and calls

[1956] Data manipulation / calculation: Analysis based on new fraud methods

[1957] Output: Fraud probability assessment result

[1958] These processing steps allow users to proactively avoid the risk of fraud and receive the most appropriate response based on their emotional state.

[1959] (Application example 2)

[1960] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1961] In recent years, fraudulent activities have evolved on a daily basis, requiring immediate and reliable responses to prevent actual damage. However, no systems exist that take into account the user's emotional state, and fraud detection and responses are often inappropriate. In particular, in physical stores, it is difficult for staff to detect fraud while interacting with customers, and it is difficult to make calm decisions when the user is in a tense state. To solve these issues, a system is needed that utilizes wearable devices such as smart glasses to detect possible fraud in real time and provide optimal responses based on the user's emotions.

[1962] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means for analyzing possible fraudulent content, a means for creating an alert and notifying the user when it is determined that fraud is highly likely, a means for updating the generation AI to respond to new fraudulent methods, a means for analyzing the user's emotional state using an emotion engine and optimizing the notification content, and a means for displaying the alert on the smart glasses. This enables fraudulent acts in physical stores to be detected in real time and an optimal response based on emotions. Specifically, when staff are wearing smart glasses, the smart glasses analyze customer behavior and conversation content and display an effective warning message when possible fraud is detected. In addition, instructions are provided according to the staff's emotional state, allowing for a calm and prompt response.

[1963] "Generative AI" is a system that uses artificial intelligence techniques to analyze data and detect specific patterns and anomalies.

[1964] An "emotion engine" is a technology that analyzes a user's emotional state from their voice and images, and is a system that grasps emotions such as stress and tension in real time.

[1965] "Notification means" refers to a system or device that conveys warnings or instructions to users based on information detected by the generative AI or emotion engine.

[1966] "Update methods" are functions that allow systems and models to be updated with the latest data and methods, keeping them up to date at all times.

[1967] "Smart glasses" are wearable devices equipped with a display and communication functions, allowing users to visually receive and integrate information.

[1968] A "server" is a computer system that receives, processes, and transmits data over a network, and is used to manage and operate multiple functions in an integrated manner.

[1969] "Fraud detection" is the process of determining whether certain actions or words are potentially fraudulent, and is performed using generative AI.

[1970] The present invention is a system for detecting possible fraudulent activity in a brick-and-mortar store and providing an optimal response based on the emotional state of the user. Specific embodiments for carrying out the present invention will be described below.

[1971] System Configuration

[1972] 1. Hardware Configuration

[1973] Smart glasses: Equipped with a display, camera, and microphone, worn by staff.

[1974] Server: A computer with high-performance data processing capabilities, primarily responsible for data analysis and running generative AI models.

[1975] Communication method: Uses 5G networks that enable high-speed data communication.

[1976] 2. Software Configuration

[1977] Generative AI models, such as GPT-4 and BERT, can be used to detect potential fraud from audio and video data.

[1978] Emotion Engine: Analyzes staff emotional states in real time using Affectiva and Emotion AI.

[1979] Notification system: Software that notifies staff in real time of warnings and appropriate responses.

[1980] Program processing

[1981] The server receives the video and audio data sent from the smart glasses and analyzes it using a generative AI model. Specifically, it determines whether the customer's behavior or comments indicate the possibility of fraud. If the analyzed data indicates the possibility of fraud, the emotion engine analyzes the staff member's current emotional state. For example, if a staff member is nervous, the notification system will display a message on the smart glasses' display such as, "Please stay calm and treat the customer kindly."

[1982] Specific usage scenarios

[1983] Consider a scenario where a customer visits a brick-and-mortar store and wishes to make a large cash transaction. The smart glasses record the conversation with the customer, and the data is sent to a server. The server's generative AI model analyzes the customer's comments to determine whether they are likely to be fraudulent. For example, if keywords such as "large cash transaction" or "request for personal information" are detected, the emotion engine detects tension in the staff member's heart rate and facial expression. The notification system then displays a message on the smart glasses, such as "This is a possible fraud, so please ask the customer to present identification."

[1984] Prompt Sentence Examples

[1985] Analyze all video and audio recordings containing "large cash transactions" and "requests for personal information" to determine the possibility of fraud.

[1986] This invention enables fast and accurate fraud detection and response in brick-and-mortar stores, ensuring the safety of both customers and staff.

[1987] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1988] Step 1:

[1989] The smart glasses' cameras and microphones capture and record the customer's movements and conversations in real time. The input is the customer's video and audio data. The output is digital data that is sent to the server.

[1990] Step 2:

[1991] The server receives the video and audio data sent from the smart glasses. The input is the video and audio data from the smart glasses. The output is that this data is stored in the server.

[1992] Step 3:

[1993] The generative AI model in the server analyzes the received video and audio data. Specifically, it analyzes the content of the conversation and customer behavior patterns to detect the possibility of fraud. The input is the video and audio data stored in the server. The output is an analysis result indicating the possibility of fraud.

[1994] Step 4:

[1995] The server's emotion engine analyzes the emotional state of staff in real time. This is done by receiving and analyzing biometric information such as the staff's heart rate and facial expressions as input. The input is the staff's biometric information. The output is the analysis result regarding the staff's emotional state.

[1996] Step 5:

[1997] The server combines the analysis results of the generative AI model and the emotion engine and sends them to the notification system. The inputs are the analysis results indicating possible fraud and the analysis results regarding the emotional state of the staff. The output is an optimized warning message.

[1998] Step 6:

[1999] The notification system displays the alert message from the server on the display of the smart glasses. The input is the alert message sent from the server. The output is the specific alert message displayed on the smart glasses.

[2000] Step 7:

[2001] The staff member, who is the user, checks the warning message displayed on the smart glasses and takes appropriate action. The input is the warning message displayed on the smart glasses. The output is the specific action that the staff member takes for the customer.

[2002] These steps enable the system to detect potential fraud in real time in physical stores and provide optimal responses based on the emotional state of staff. For example, effective fraud detection can be achieved by analyzing all video and audio data containing "large cash transactions" and "requests for personal information" into a generative AI model and inputting prompt sentences to determine the likelihood of fraud.

[2003] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2004] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2005] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2006] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2007] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2008] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2009] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2010] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2011] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2012] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2013] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2014] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2015] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2016] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2017] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2018] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2019] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2020] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2021] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2022] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate...

Claims

1. A generative AI method for analyzing potentially fraudulent content; A means to generate alerts and notify users if fraud is deemed likely; A system that includes a means to update the generative AI to respond to new fraud techniques.

2. The system of claim 1 includes a means for the generation AI to respond to incoming calls from anonymous or unknown phone numbers with an automated voice, record and analyze the content of the call, and notify the user if it determines that there is a high possibility of fraud.

3. The system of claim 1 further includes a means for generating an alert for an email or SNS message and notifying the user if the generation AI analyzes the content of the email or SNS message and determines that the message may be fraudulent.

Images