System

Abstract

To provide an environment in which a user can safely perform online activities by providing a real-time phishing fraud detection and warning system using a generative model.SOLUTION: The phishing fraud detection system includes means for analyzing the received digital information using a generative model trained using knowledge of security experts and training data obtained from specialized information sources to detect characteristics of the fraudulent activity, means for generating an alert based on the analysis, and means for providing specific safety action advice to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, fraudulent activities via the Internet have been increasing rapidly, and phishing scams in particular have become victims of a wide variety of methods. While such fraudulent activities can have a significant impact on users' personal information and assets, existing countermeasures are unable to keep up with the ever-diversifying phishing methods. The present invention aims to create an environment where users can safely engage in online activities by providing a real-time phishing detection and warning system using generative models. [Means for solving the problem]

[0005] The present invention is a system that uses a generative model to analyze received digital information and detect characteristics of fraudulent activity. The system includes a means for issuing a warning based on the analysis results and a means for providing specific safety behavior advice to the user. The system also trains the generative model using training data obtained from the knowledge of security experts and specialized information sources, enabling it to respond to new fraudulent methods. Furthermore, by collecting feedback from users and improving the system's performance, the system provides a constantly evolving phishing fraud detection system that reflects the latest security information.

[0006] A "generative model" is a model that uses algorithms such as machine learning and deep learning to analyze data and generate new information and patterns.

[0007] "Analysis" is the process of breaking down digital information or data and extracting the information needed to understand it.

[0008] "Fraud signatures" refer to patterns or indicators common to phishing scams and other internet-based fraud.

[0009] A "warning" is information that provides advance notice and caution against specific risks or dangers.

[0010] "Digital information" refers to data such as text, images, audio, and video that is processed through a computer or other electronic device.

[0011] "Real time" refers to a state in which processing and response are carried out immediately at the moment an event occurs.

[0012] "Security expert knowledge" refers to insights and advice from experts with specialized knowledge and experience in the field of information security.

[0013] "Training data" is a data set consisting of known input-output pairs that is used to improve the performance of a machine learning model.

[0014] "User feedback" refers to opinions, evaluations, and information on areas for improvement provided by users of the system.

[0015] "Safety advice" refers to specific instructions and advice that help users avoid risks and act safely. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The system for implementing the present invention involves three entities: a server, a terminal, and a user. Details and specific examples of the processing performed by each entity will be explained below.

[0038] Server-side processing

[0039] Detecting email receipt

[0040] The server periodically checks the user's mailbox to see if any new emails have been received, using the mail client API.

[0041] Acquiring and analyzing email content

[0042] When the server detects a new email, it retrieves its contents and analyzes them using a generative model, which uses machine learning and deep learning algorithms to extract phishing features from the email body.

[0043] Phishing signature detection

[0044] It applies specific algorithms and rule-based models to detect phishing features from the analyzed email content, and generates a warning message if phishing features are detected.

[0045] Generating warning messages and advice

[0046] If a phishing scam is detected, the server generates a warning message for the user, including advice on safe behavior, which is reasonable and specific, such as "don't click on the link" or "go directly to the official website."

[0047] User Notification

[0048] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[0049] Terminal (client) side processing

[0050] Receiving new mail

[0051] The user's device periodically communicates with the mail server to receive new emails, which allows phishing scams to be detected and flagged before the user has a chance to check the email.

[0052] Communicating with the Server

[0053] The device receives warning messages and advice sent from the server, and the received data is displayed securely and instantly to the user.

[0054] Displaying a warning message

[0055] The received warning message will be displayed on the device, allowing users to immediately become aware of the risk of phishing scams.

[0056] User Roles

[0057] Checking warnings and taking safe actions

[0058] Users should check the warning messages and advice displayed on their devices and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email.

[0059] Providing feedback

[0060] Users can provide feedback on warning messages provided by the system, allowing the server to collect feedback and continuously improve the generative model.

[0061] Specific examples

[0062] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly."

[0063] Users should check the warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams.

[0064] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[0065] The processing flow will be explained below.

[0066] Server-side processing

[0067] Step 1:

[0068] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0069] Step 2:

[0070] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[0071] Step 3:

[0072] The server uses the generative model to analyze the email content, and the analysis process involves extracting text features from the email body using natural language processing (NLP) techniques.

[0073] Step 4:

[0074] The server applies an algorithm to detect fraudulent activity based on the analyzed feature information, and if it detects any phishing features, it records the results.

[0075] Step 5:

[0076] The server generates a warning message and advice on safe behavior based on the detection results, for example, "Phishing scam detected. Do not click on the link."

[0077] Step 6:

[0078] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[0079] Terminal side processing

[0080] Step 1:

[0081] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[0082] Step 2:

[0083] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0084] Step 3:

[0085] The terminal receives warning messages and advice sent by the server, and the received data is processed for secure and immediate display to the user.

[0086] Step 4:

[0087] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[0088] User Response

[0089] Step 1:

[0090] Users should check the warning message displayed on their device and follow the instructions, for example, by visiting the official website directly rather than clicking on the link in the email.

[0091] Step 2:

[0092] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model.

[0093] Step 3:

[0094] If users receive unclear warnings or advice, they can contact us for more information.

[0095] The above is a detailed description of the processing steps of each subject in the present invention. This series of processing enables phishing scams to be detected in real time and appropriate warnings and advice to be provided to users.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Many emails received by Internet users contain fraudulent activity, such as phishing scams. Therefore, unless users take appropriate measures against these emails, they are at high risk of personal information leaks and financial losses. Furthermore, conventional phishing detection systems lacked detection accuracy and immediacy, and were unable to completely prevent users from falling victim to fraud. Furthermore, it was difficult to respond quickly to new fraud methods. To solve these issues, a system is needed that can detect phishing scams with high accuracy and in real time, and provide users with prompt warnings and advice.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes means for analyzing information obtained using the generative model and detecting characteristics of phishing scams, means for issuing warnings based on the detection results, means for providing users with advice on safe behavior, means for periodically updating training data to deal with new fraudulent activities, means for collecting feedback from users to improve the accuracy of the generative model, and means for instantly notifying users of warning messages and advice to their devices. This makes it possible to detect phishing scams with high accuracy and in real time, and significantly reduce the risk of users becoming victims of fraudulent activities.

[0101] A "generative model" is a model that uses machine learning or deep learning algorithms to generate specific outputs from input data.

[0102] "Phishing" is a fraudulent activity that uses fake websites and emails to steal personal and financial information from users.

[0103] A "warning message" is a message that alerts users to the risk of detected phishing scams.

[0104] "Safety advice" is information that provides users with specific guidelines for action to avoid the risks posed by phishing scams.

[0105] "Training data" is a reference data set used to train a machine learning model, derived from the knowledge of security experts and expert information sources.

[0106] "Feedback" refers to opinions and ratings provided by users that are used to improve the performance of the system.

[0107] A "prompt sentence" is an input sentence that causes a generative AI model to generate an output.

[0108] "Immediate notification" refers to a means of communication that quickly conveys information about detected phishing scams to users.

[0109] "Digital information" refers to any information expressed in electronic format, such as email or online messages.

[0110] MODE FOR CARRYING OUT THE INVENTION

[0111] The system for implementing the present invention involves three main components: a server, a terminal, and a user. The specific software and hardware usage, data processing and calculation methods will be described in detail below.

[0112] Server-side processing

[0113] The server first periodically monitors the user's mailbox using the Gmail API and IMAP protocol. Specifically, it checks every 10 minutes to see if there is any new email. If a new email is detected through this process, the header information of that email is retrieved.

[0114] Next, the server retrieves the content of the received email. This process uses Python's Pandas library and natural language processing libraries (e.g., NLTK, Spacy). The retrieved email body and attachments are then used for data analysis.

[0115] Machine learning libraries such as Scikit-learn and TensorFlow are applied to detect phishing scam characteristics from the analyzed email content. For example, the frequency of occurrence of specific keywords in the email and URL domain checks are performed to evaluate the possibility of phishing scams.

[0116] If phishing features are detected, the server generates a warning message and advice on safe behavior. A generative AI model (e.g., GPT-3) is used to input a prompt and create a specific warning message.

[0117] The generated warning messages and advice are sent to the user's device using the SMTP protocol, and in some cases, Firebase Cloud Messaging is used to provide in-app notifications.

[0118] Terminal (client) side processing

[0119] A user's terminal receives new emails using email client software (e.g., Outlook, Thunderbird), and the email data is sent from the server via a protocol such as POP3 or IMAP and stored locally.

[0120] The device receives the warning message and advice sent from the server via the HTTPS protocol and displays it using Android's Notification Manager or iOS's Push Notification, allowing users to immediately recognize the risk and receive advice on how to act safely.

[0121] User Roles

[0122] Users should check the warning messages and advice displayed on their devices and take action to avoid the risks posed by phishing scams, such as visiting the official website directly instead of clicking on links in emails identified as phishing.

[0123] Additionally, users can provide feedback to the system using a dedicated form or in-app buttons, and the server uses this information to improve the accuracy of the generative model.

[0124] Specific examples

[0125] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the user's device receives this email, its contents are immediately sent to the server and analyzed using a generative AI model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly." The user can avoid the risk of phishing scams by checking this warning message and visiting the official website directly without clicking on the link.

[0126] Prompt Sentence Examples

[0127] "Generate a warning message in case of a phishing email pretending to be from a bank."

[0128] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Step 1:

[0131] The server periodically checks the user's mailbox using the Gmail API and IMAP protocol. This process detects new emails. The input data is the user's email account information, and the output is the header information of the new email. Specifically, the server checks every 10 minutes to see if new emails have arrived and retrieves the email header information.

[0132] Step 2:

[0133] When a new email is detected, the server retrieves the email's contents. The input data is the email's header information, and the output is the email body and the contents of any attachments. This process uses Python's Pandas library and a natural language processing library (e.g., NLTK, Spacy). Specifically, the email body is retrieved in text format, and data from attachments is also extracted.

[0134] Step 3:

[0135] The server applies machine learning models and deep learning algorithms to detect phishing scam characteristics from the email content it retrieves. The input data is the email body and attachment contents, and the output is the phishing scam detection results. Libraries such as Scikit-learn and TensorFlow are used for this processing. Specifically, it checks the frequency of occurrence of specific keywords in the email and the domain of the URL.

[0136] Step 4:

[0137] If the server detects characteristics of a phishing scam, it generates a warning message and advice on safe behavior. The input data is the phishing detection result, and the output is a warning message. A generative AI model (e.g., GPT-3) is used for this generation, and a specific warning message is created based on a prompt text. As a specific example, the prompt text used is, "Please generate a warning message in the event of a phishing email being received that pretends to be from a bank."

[0138] Step 5:

[0139] The server sends the generated warning message and advice to the user's device. The input data is the warning message, and the output is a notification displayed on the user's device. Notifications can be sent via email using the SMTP protocol or in-app notifications using Firebase Cloud Messaging. Specifically, the warning message and advice are sent to the user's device immediately.

[0140] Step 6:

[0141] The user's device receives the warning message and advice sent from the server and displays it immediately. The input data is the warning message received from the server, and the output is a notification that the user can see. This notification is sent using Android's Notification Manager or iOS's Push Notification. Specifically, the message "Phishing Attack Detected" is displayed in the notification area on the device, and the user can check the details.

[0142] Step 7:

[0143] The user checks the warning message and advice displayed on the device and takes safe action. The input data is the warning message displayed on the device, and the output is the user's action. For example, the user may take action by visiting the official website directly instead of clicking on the link in the email.

[0144] Step 8:

[0145] Users can provide feedback on the displayed warning message. The input data is the user's feedback, and the output is information recorded in the server's database. Feedback is sent using a dedicated form or a button within the app, and the server uses this information to improve the generative model. Specifically, it provides feedback to the system saying, "This warning was helpful."

[0146] (Application example 1)

[0147] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0148] In today's cyber environment, fraudulent activities such as phishing scams are on the rise, and methods for stealing personal and financial information via email are becoming more sophisticated. Therefore, there is a need for fast and effective systems that can help users recognize fraudulent activities and take safe actions. However, conventional phishing detection systems often lack practicality due to issues such as delayed warnings and users missing notifications. Furthermore, there is a lack of flexible systems that can respond to new fraudulent activities other than phishing scams.

[0149] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0150] In this invention, the server includes means for analyzing the obtained information using a generative model and detecting characteristics of fraudulent activity, means for issuing a warning about digital information that has characteristics of fraudulent activity, means for displaying the detected warning and advice on the display of the smart glasses, and means for providing advice on safe behavior to the user. This makes it possible to notify the user of phishing scams and other fraudulent activity in real time and provide visual alerts. It also provides advice on appropriate actions that the user should take immediately, minimizing the risk of harm.

[0151] A "generative model" is a model trained using machine learning or deep learning algorithms to identify and generate specific patterns or features.

[0152] "Fraudulent activity" refers to malicious actions or behaviors against users, such as phishing scams.

[0153] "Smart glasses" are a wearable eyeglass-type device equipped with a small computer for displaying digital information.

[0154] A "display" is a screen or display device that provides visual information to a user.

[0155] A "warning" is a message or display that informs users of the risk of fraudulent activity.

[0156] "Advice" is specific instructions or suggestions to encourage users to behave safely.

[0157] "Real-time" refers to a state in which data and information are processed immediately and provided without delay.

[0158] "Cloud" refers to resources such as data storage and computing resources provided over the Internet.

[0159] "Feedback" refers to the system usage experience and opinions provided by users, and is information used to improve system performance.

[0160] The system for implementing this invention involves three entities: a server, a terminal, and a user. The roles and specific processes of each entity will be explained below.

[0161] Server-side processing

[0162] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve the latest emails. When the server detects a new email, it retrieves its contents and analyzes them using a generative model. The generative model uses machine learning and deep learning algorithms to extract features of fraudulent activity, such as phishing scams, from the email body.

[0163] Specific algorithms and rule-based models are applied to detect fraudulent activity signatures from the analyzed email content. If fraudulent activity signatures are detected, a warning message is generated and includes advice on safe behavior. This advice is reasonable and specific, such as "do not click on links" or "go directly to the official website." The generated warning message and advice are sent to the user's smart glasses.

[0164] Processing on the device (smart glasses)

[0165] The smart glasses periodically communicate with the server to receive warning messages and advice, and the received data is immediately displayed in the user's field of view, allowing the user to immediately know about the risk of fraud.

[0166] User Roles

[0167] Users can check the warning messages and advice displayed on the smart glasses and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email. Users can also provide feedback on the warning messages provided by the system. The server collects the feedback and continuously improves the generative model.

[0168] Specific examples

[0169] For example, consider a case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the server receives the email, it begins analyzing it. Below is an example of a prompt sent to the generative AI model.

[0170] Prompt statement:

[0171] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[0172] The generative AI model receives this prompt and evaluates whether the email is a phishing scam. If phishing characteristics are detected, the server generates a warning message saying "Phishing scam detected. Do not click on the link" and advice "Please visit the official website directly," and sends these to the user's smart glasses. By checking the warning message displayed on the smart glasses and visiting the official website directly without clicking on the link, the user can avoid the risk of phishing scams.

[0173] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0174] Step 1:

[0175] The server periodically checks the user's mailbox to see if new emails have arrived. The input is direct external data: the user's email information, which is retrieved using the email client API. The output is a list of new emails.

[0176] Step 2:

[0177] When the server detects new emails, it retrieves their contents and stores them in the database. The input is a list of new emails, and the email contents are retrieved in text format via the API. The output is the retrieved email contents as text data.

[0178] Step 3:

[0179] The server sends the acquired email content to the generative AI model for analysis. The input is the text data of the email content, and the generative AI model extracts the characteristics of phishing scams. As a concrete example, the following prompt sentence is input:

[0180] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[0181] The output is the analysis result, which is a judgment result as to whether the email is a phishing scam or not.

[0182] Step 4:

[0183] The server generates a warning message if phishing scam characteristics are detected based on the analysis results of the generative AI model. The input is the analysis results, and if fraudulent activity is detected, it generates a warning message saying, "Phishing scam detected. Do not click on the link." The output is a warning message and specific advice on what to do.

[0184] Step 5:

[0185] The server sends the generated warning message and advice to the user's smart glasses. The input is the warning message and action advice, which are sent to the smart glasses via the network. The output is the warning message and advice displayed on the smart glasses.

[0186] Step 6:

[0187] The smart glasses receive warning messages and advice and display them in the user's field of view. The input is the warning messages and advice sent from the server, which are visualized in real time on the glasses' display. The output is information that allows the user to visually confirm the warning.

[0188] Step 7:

[0189] The user checks the warning messages and advice displayed on the smart glasses and follows the instructions. The input is the displayed warning messages and advice, and the user takes safe actions to avoid the risk of phishing scams. The output is the user's safe actions.

[0190] Step 8:

[0191] Users provide feedback on warning messages provided by the system. The input is the user's usage experience and opinions, and the server collects this feedback and stores it in a database. The output is data for improving the system's performance.

[0192] Step 9:

[0193] The server uses the collected feedback to update the generative model. The input is the user feedback data, which is used as training data for the generative AI model. The output is an improved generative AI model.

[0194] The above series of processes realizes a system that detects phishing scams in real time and provides warnings and advice to users.

[0195] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0196] The system for implementing the present invention involves four main entities: a server, a terminal, a user, and an emotion engine. Details and specific examples of the processing performed by each entity will be explained below.

[0197] Server-side processing

[0198] Detecting email receipt

[0199] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0200] Acquiring and analyzing email content

[0201] When the server detects a new email, it retrieves its contents. The email body and associated metadata are sent to a generative model for analysis. The generative model uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify signs of fraud.

[0202] Acquiring emotion data

[0203] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[0204] Phishing signature detection

[0205] The server analyzes the email content to detect signs of fraudulent activity such as phishing scams, records the results, and combines them with emotional data obtained from the emotion engine to generate a warning message based on the user's current emotional state.

[0206] Generating warning messages and advice

[0207] If a phishing scam is detected, the server generates a warning message and advice on safe behaviors according to the user's emotional state. For example, if the user is feeling stressed, the server will provide a more specific and gentle warning message.

[0208] User Notification

[0209] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[0210] Terminal side processing

[0211] Receiving new mail

[0212] The user's device periodically communicates with the mail server to receive new emails, using a standard protocol (such as IMAP or POP3) through the mail client.

[0213] Communicating with the Server

[0214] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0215] Receiving a warning message

[0216] The device receives warning messages and advice sent from the server, which incorporates data from the emotion engine and is based on the user's current emotional state.

[0217] Displaying a warning message

[0218] Display received warning messages to users, so that they are immediately aware of the risk of phishing.

[0219] User Response

[0220] Checking warnings and taking safe actions

[0221] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. The emotion engine also provides more detailed and gentler advice if the user is feeling stressed.

[0222] Providing feedback

[0223] Users can provide feedback on the warning messages and advice provided by the system, which is sent to the server and used to improve the generative model and emotion engine.

[0224] Specific examples

[0225] For example, consider the case where a user receives a phishing email pretending to be from a bank. The email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with a gentler advice saying, "We recommend that you visit the official website directly."

[0226] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[0227] The above is a detailed description of the processing steps of each entity in the system of the present invention. This series of processes enables real-time detection of phishing scams and provides appropriate warnings and advice that take into account the user's emotional state.

[0228] The processing flow will be explained below.

[0229] Server-side processing

[0230] Step 1:

[0231] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0232] Step 2:

[0233] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[0234] Step 3:

[0235] The server analyzes the email content using a generative model, which uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify phishing scam signatures.

[0236] Step 4:

[0237] The server applies an algorithm to detect phishing features based on the analyzed feature information, and if any phishing features are detected, records the results.

[0238] Step 5:

[0239] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[0240] Step 6:

[0241] The server generates warning messages and safety behavior advice based on the detection results. It considers the emotional data obtained from the emotion engine and prepares messages that match the user's current emotional state. If the user is feeling stressed, it provides warning messages with more specific and gentler wording.

[0242] Step 7:

[0243] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[0244] Terminal side processing

[0245] Step 1:

[0246] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[0247] Step 2:

[0248] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0249] Step 3:

[0250] The device receives warning messages and advice sent from the server, and the received data is based on the user's current emotional state.

[0251] Step 4:

[0252] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[0253] User Response

[0254] Step 1:

[0255] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. If users feel stressed, the emotion engine will provide more detailed and gentler advice.

[0256] Step 2:

[0257] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model and emotion engine.

[0258] Step 3:

[0259] If users receive unclear warnings or advice, they can contact us for more information.

[0260] Specific examples

[0261] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice saying, "We recommend that you visit the official website directly."

[0262] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[0263] Example 2

[0264] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0265] In modern society, phishing scams and other fraudulent activities are rapidly increasing, and users are exposed to sophisticated cyber attacks every day. However, conventional fraud detection systems cannot simply detect fraudulent patterns; they must also take into account the user's emotional state to respond appropriately. As a result, when users receive fraud warnings while feeling stressed or anxious, it becomes difficult for them to respond appropriately. This leaves a gap in effective measures to support users' safe online activities.

[0266] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity; means for acquiring the user's emotional state; means for issuing a warning based on the detection result and the user's emotional state; and means for providing the user with advice on safe behavior. The terminal includes: means for monitoring received digital information in real time and identifying digital information having characteristics of fraudulent activity; means for issuing a warning according to the user's emotional state for digital information identified as having characteristics of fraudulent activity; and means for providing the user with specific behavioral advice. This makes it possible to detect fraudulent activity in real time and provide appropriate warning messages and behavioral advice that take the user's emotional state into consideration.

[0267] A "generative model" is a model trained using a machine learning algorithm, which extracts meaningful features from input data and makes predictions.

[0268] "Fraud signatures" refer to patterns or signatures that identify malicious behavior intended to defraud or steal information.

[0269] "Means for issuing warnings" refers to technology that provides a function for notifying users of a warning message in response to detected fraudulent activity.

[0270] "Means for providing advice on safe behavior" refers to a feature that provides specific guidelines or advice for users to deal with fraudulent behavior.

[0271] "Means for acquiring the user's emotional state" refers to technology that estimates the user's emotional state at that time by analyzing the user's facial expressions, tone of voice, keystrokes, mouse movements, etc.

[0272] "Real-time monitoring" refers to the process of analyzing and processing data the moment it is generated.

[0273] "Digital information" refers to any information that is stored, transmitted, or processed in electronic form.

[0274] "Methods of collecting feedback" refers to the techniques and processes used to collect information such as user opinions, ratings, and usage experiences.

[0275] The system for implementing this invention involves four entities: a server, a terminal, a user, and an emotion engine. This system is designed to protect users from cyber attacks and fraudulent activities.

[0276] The server uses the following major hardware and software components:

[0277] Email client APIs: These include the Gmail API, Outlook API, etc., which allow the server to periodically check the user's mailbox to see if new emails have been received.

[0278] Generative AI model: Uses natural language processing (NLP) techniques to extract text features from email bodies and use approaches to detect fraudulent activity.

[0279] Emotion Engine API: Using technologies such as EmotionAPI, we identify the user's emotional state from their facial expressions, tone of voice, keystrokes, and mouse movements.

[0280] Database: Used to store and manage analysis results and emotion data.

[0281] The terminal works as follows:

[0282] Email client software: Uses standard protocols such as IMAP or POP3 to communicate with the email server and receive new email.

[0283] Communication module: Encrypts data using secure communication protocols such as TLS and communicates with the server in a secure manner.

[0284] UI component: Provides a user interface for displaying warning messages and advice to the user.

[0285] Users can receive warning messages and advice via their devices and take appropriate measures. Users can also provide feedback on the provided warning messages and advice, contributing to improving the system's performance.

[0286] The specific operation is explained below:

[0287] 1. The server periodically checks the user's mailbox using the mail client API to see if there are any new emails. For example, the check_new_mail() function is called every minute to see if there are any new emails.

[0288] 2. When a new email is detected, the email content is retrieved and the generative AI model is used to parse the email body and associated metadata.

[0289] 3. At the same time, the server calls the emotion engine API to obtain the user's emotion data and integrate it with the analysis results.

[0290] 4. The server detects signs of fraudulent activity from the analyzed email content and generates appropriate warning messages and advice by referring to the emotional data.

[0291] 5. The generated warning message and advice are sent from the server to the device using a secure protocol (TLS).

[0292] 6. The device displays the sent warning message to the user, and the user takes safe action based on it.

[0293] 7. Users can provide feedback on warning messages from their devices, which is sent to the server and used to improve system performance.

[0294] As a concrete example, the following prompt sentences can be fed into a generative AI model for analysis:

[0295] "Analyze the text of the email below and identify the characteristics of a phishing scam.

[0296] From: example@bank.com

[0297] Subject: Account verification required

[0298] Main text:

[0299] Dear Customer,

[0300] We have detected unusual activity on your account. Please click the link below to review your account information.

[0301] [Link to fake site]

[0302] thank you."

[0303] Using this prompt, the system can identify the characteristics of phishing scams and provide appropriate warnings and advice to users.

[0304] The above is a detailed description of the preferred embodiment of the invention. The system allows for real-time detection of fraudulent activity and provides warnings and advice based on emotional state.

[0305] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0306] Step 1:

[0307] The server periodically checks the user's mailbox to see if any new mail has been received.

[0308] Specifically, the server calls the check_new_mail() function to retrieve unread emails using a mail client API (e.g., Gmail API or Outlook API). The input is the API request, and the output is the ID of the new email or email metadata.

[0309] Step 2:

[0310] When the server detects new mail, it retrieves and analyzes its contents.

[0311] Specifically, the server retrieves the email body and associated metadata using the fetch_email_content(mail_id) function. The input is the email ID, and the output is the email body and metadata. This is then sent to the generative AI model, where text analysis is performed using the analyze_text_for_phishing(email_content) function. This analysis extracts features from the input email body and detects fraudulent activity.

[0312] Step 3:

[0313] While analyzing the email, the server also obtains the user's emotional data from the emotion engine.

[0314] Specifically, the server calls the emotion engine API (e.g., EmotionAPI) and retrieves emotion data using the fetch_emotion_data(user_id) function. The input is the user ID, and the output is the user's current emotional state data.

[0315] Step 4:

[0316] The server combines the analyzed email content with emotional data to detect phishing signatures.

[0317] Specifically, the server uses the detect_phishing(features, emotion_data) function to integrate the analysis results with the emotion data and obtain a fraud detection result. The input is the feature data and emotion data, and the output is the phishing detection result.

[0318] Step 5:

[0319] If a phishing scam is detected, the server generates a warning message and safety behavior advice.

[0320] Specifically, the server uses the generate_warning_message(phishing_detected, emotion_data) function to generate a warning message and advice based on the emotional state. The input is the phishing detection result and emotion data, and the output is a warning message and advice.

[0321] Step 6:

[0322] The server notifies the user's terminal of the generated warning message and advice.

[0323] Specifically, the server calls the send_notification(user_id, message) function to notify the user via email, SMS, or in-app notification. The input is the user ID and message, and the output is the result of sending the notification.

[0324] Step 7:

[0325] The user's device receives the new email and begins communicating with the server.

[0326] The specific operation involves the terminal receiving new email using a standard protocol such as IMAP or POP3 using the connect_to_mail_server() function and sending the content to the server. The input is the connection information for the mail server, and the output is the content of the new email.

[0327] Step 8:

[0328] The user's terminal receives the warning messages and advice sent from the server.

[0329] Specifically, the terminal receives an alert message from the server using the receive_notification() function. The input is a notification request, and the output is an alert message.

[0330] Step 9:

[0331] The user acknowledges the warning message and takes safe action.

[0332] The specific behavior includes the user reading the warning message displayed on the device, not clicking on the link in the phishing email, and directly visiting the official website. The input is the warning message, and the output is the user's behavior.

[0333] Step 10:

[0334] Users provide feedback on system warning messages and advice.

[0335] Specifically, the user uses the feedback function of the device and sends feedback to the server using the submit_feedback(feedback) function. The input is the user's feedback, and the output is the feedback sending result.

[0336] This is the flow of the system's program processing. This series of processes enables the system to detect phishing scams in real time and provide warning messages and advice on what to do that take into account the user's emotional state.

[0337] (Application example 2)

[0338] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0339] Conventional phishing detection systems issue warnings without considering the user's emotional state, which can cause excessive stress and confusion. Furthermore, the warning messages are uniform and lack advice to encourage users to take appropriate safety actions. Furthermore, they do not integrate with robots used in the home, limiting the means of notifying users. This presents a challenge for users, making it difficult for them to quickly understand the dangers of phishing scams and take appropriate action.

[0340] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity, means for evaluating the user's emotional state using an emotion engine, and means for generating and presenting appropriate warning messages and advice on safe behavior based on the detection results and the user's emotional state. This makes it possible to provide more effective and less stressful warning messages and advice that take the user's emotional state into consideration. In addition, by linking with a robot used in the home, warnings can be conveyed to the user through voice notifications and displays, allowing the user to respond quickly and appropriately.

[0341] A "generative model" is an algorithm that uses AI technology to extract features from input data and perform analysis and predictions.

[0342] The "emotion engine" is a system that evaluates a user's emotional state based on data such as the user's facial expressions, tone of voice, keystrokes, and mouse movements.

[0343] "Fraud signatures" are indicators and patterns that indicate phishing scams or other malicious activity.

[0344] An "appropriate warning message" is a message that informs users of the risk of fraudulent activities such as phishing scams and urges them to take specific action.

[0345] "Safety advice" is information that provides specific actions and measures that users should take when they encounter fraudulent activity.

[0346] "Real-time monitoring" is the process of reviewing and analyzing data as it is generated or received.

[0347] "Feedback" means opinions and information provided by users about the performance and usability of the system.

[0348] "Pattern recognition" is a technique for finding specific patterns or characteristics in data and performing analysis and judgments based on them.

[0349] "Audio Notifications" are a means of audibly conveying system-generated informational or warning messages to the user.

[0350] "Display notification" is a means of displaying system-generated informational or warning messages to the user on a display.

[0351] The system for realizing this application example is based on a robot used in the home, and includes a server, a terminal, a user, and an emotion engine. A specific implementation method of this system will be described below.

[0352] The server analyzes the features of the received data using a generative model. The generative model includes an algorithm that uses natural language processing techniques to extract features from the email body and detects signs of fraudulent activity. It also uses an emotion engine to evaluate the user's emotional state from facial expressions, tone of voice, keystrokes, mouse movements, etc. The emotion engine includes software for identifying the user's emotional state.

[0353] When a device receives an email, it immediately sends the contents of the email to a server and requests analysis. This data is encrypted using a secure communication protocol (e.g., TLS). Once the analysis is complete, the results are returned to the device, which then notifies the user of a warning message based on their emotional state and advice on safe behavior.

[0354] When the home robot receives a notification from the device, it communicates the information to the user through voice output and a display. Voice notifications are made using the robot's internal speaker, and warning messages and advice on safe behavior are displayed on the display.

[0355] For example, if a user receives a phishing email pretending to be from a bank, the content of the email is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice such as, "We recommend visiting the official website directly." These messages are communicated to the user aloud through the home robot's speaker, and similar messages are displayed on the display.

[0356] By checking the robot warning message and visiting the official website directly instead of clicking on the link, users can avoid the risk of phishing scams and stay safe online.

[0357] Example prompts to input to a generative AI model:

[0358] Detect phishing scams by analyzing the following email body:

[0359] ---

[0360] Subject: Important Notice

[0361] Body: Due to an update to our security system, please click the link below to update your authentication information.

[0362] Link: http: / / example.com / secure

[0363] Using this prompt, the generative AI model detects phishing scam characteristics within the email body and provides the basis for generating appropriate warning messages.

[0364] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0365] Step 1:

[0366] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve unread emails. The input is unread email data from the email server, and the output is the retrieved unread email information.

[0367] Step 2:

[0368] When the server detects a new email, it retrieves its contents. The retrieved email body and related metadata are sent to the generative model for analysis. The input is unread email information, and the output is the email body data for analysis.

[0369] Step 3:

[0370] The generative model uses natural language processing techniques to analyze the email body and extract features from the text. It then applies an algorithm to identify fraudulent activity features. The input is the email body data, and the output is fraud feature data.

[0371] Step 4:

[0372] While the server is analyzing the email, it also obtains the user's emotional data from the emotion engine. The emotion engine analyzes the user's facial expressions, tone of voice, keystrokes, mouse movements, etc. to identify their emotional state. The input is the user's behavioral data, and the output is emotional state data.

[0373] Step 5:

[0374] The server detects characteristics of fraudulent activities such as phishing scams from the analyzed email content and records the results. This is combined with emotional data obtained from the emotion engine to generate a warning message based on the user's emotional state. The input is fraudulent activity characteristic data and emotional state data, and the output is a warning message that takes emotions into consideration.

[0375] Step 6:

[0376] If a phishing scam is detected, the server generates a warning message and advice on safe behavior according to the user's emotional state. If the user is feeling stressed, it generates a gentle warning message and advises specific actions. The input is emotional state data and fraud feature data, and the output is a warning message and advice.

[0377] Step 7:

[0378] The terminal receives warning messages and advice sent from the server. This data is encrypted using a secure communication protocol (e.g., TLS). The input is the warning message and advice from the server, and the output is the received notification data.

[0379] Step 8:

[0380] The home robot receives warning messages from the device and conveys information to the user through voice output and a display. The input is notification data from the device, and the output is notification and display to the user.

[0381] Step 9:

[0382] The user checks the robot's warning message and follows the instructions to take safe action, thereby avoiding the risk of phishing scams. The input is the warning message from the robot, and the output is the user's actions to avoid risk.

[0383] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0384] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0385] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0386] [Second embodiment]

[0387] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0388] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0389] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0390] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0391] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0392] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0393] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0394] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0395] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0396] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0397] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0398] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0399] The system for implementing the present invention involves three entities: a server, a terminal, and a user. Details and specific examples of the processing performed by each entity will be explained below.

[0400] Server-side processing

[0401] Detecting email receipt

[0402] The server periodically checks the user's mailbox to see if any new emails have been received, using the mail client API.

[0403] Acquiring and analyzing email content

[0404] When the server detects a new email, it retrieves its contents and analyzes them using a generative model, which uses machine learning and deep learning algorithms to extract phishing features from the email body.

[0405] Phishing signature detection

[0406] It applies specific algorithms and rule-based models to detect phishing features from the analyzed email content, and generates a warning message if phishing features are detected.

[0407] Generating warning messages and advice

[0408] If a phishing scam is detected, the server generates a warning message for the user, including advice on safe behavior, which is reasonable and specific, such as "don't click on the link" or "go directly to the official website."

[0409] User Notification

[0410] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[0411] Terminal (client) side processing

[0412] Receiving new mail

[0413] The user's device periodically communicates with the mail server to receive new emails, which allows phishing scams to be detected and flagged before the user has a chance to check the email.

[0414] Communicating with the Server

[0415] The device receives warning messages and advice sent from the server, and the received data is displayed securely and instantly to the user.

[0416] Displaying a warning message

[0417] The received warning message will be displayed on the device, allowing users to immediately become aware of the risk of phishing scams.

[0418] User Roles

[0419] Checking warnings and taking safe actions

[0420] Users should check the warning messages and advice displayed on their devices and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email.

[0421] Providing feedback

[0422] Users can provide feedback on warning messages provided by the system, allowing the server to collect feedback and continuously improve the generative model.

[0423] Specific examples

[0424] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly."

[0425] Users should check the warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams.

[0426] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[0427] The processing flow will be explained below.

[0428] Server-side processing

[0429] Step 1:

[0430] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0431] Step 2:

[0432] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[0433] Step 3:

[0434] The server uses the generative model to analyze the email content, and the analysis process involves extracting text features from the email body using natural language processing (NLP) techniques.

[0435] Step 4:

[0436] The server applies an algorithm to detect fraudulent activity based on the analyzed feature information, and if it detects any phishing features, it records the results.

[0437] Step 5:

[0438] The server generates a warning message and advice on safe behavior based on the detection results, for example, "Phishing scam detected. Do not click on the link."

[0439] Step 6:

[0440] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[0441] Terminal side processing

[0442] Step 1:

[0443] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[0444] Step 2:

[0445] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0446] Step 3:

[0447] The terminal receives warning messages and advice sent by the server, and the received data is processed for secure and immediate display to the user.

[0448] Step 4:

[0449] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[0450] User Response

[0451] Step 1:

[0452] Users should check the warning message displayed on their device and follow the instructions, for example, by visiting the official website directly rather than clicking on the link in the email.

[0453] Step 2:

[0454] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model.

[0455] Step 3:

[0456] If users receive unclear warnings or advice, they can contact us for more information.

[0457] The above is a detailed description of the processing steps of each subject in the present invention. This series of processing enables phishing scams to be detected in real time and appropriate warnings and advice to be provided to users.

[0458] Example 1

[0459] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0460] Many emails received by Internet users contain fraudulent activity, such as phishing scams. Therefore, unless users take appropriate measures against these emails, they are at high risk of personal information leaks and financial losses. Furthermore, conventional phishing detection systems lacked detection accuracy and immediacy, and were unable to completely prevent users from falling victim to fraud. Furthermore, it was difficult to respond quickly to new fraud methods. To solve these issues, a system is needed that can detect phishing scams with high accuracy and in real time, and provide users with prompt warnings and advice.

[0461] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0462] In this invention, the server includes means for analyzing information obtained using the generative model and detecting characteristics of phishing scams, means for issuing warnings based on the detection results, means for providing users with advice on safe behavior, means for periodically updating training data to deal with new fraudulent activities, means for collecting feedback from users to improve the accuracy of the generative model, and means for instantly notifying users of warning messages and advice to their devices. This makes it possible to detect phishing scams with high accuracy and in real time, and significantly reduce the risk of users becoming victims of fraudulent activities.

[0463] A "generative model" is a model that uses machine learning or deep learning algorithms to generate specific outputs from input data.

[0464] "Phishing" is a fraudulent activity that uses fake websites and emails to steal personal and financial information from users.

[0465] A "warning message" is a message that alerts users to the risk of detected phishing scams.

[0466] "Safety advice" is information that provides users with specific guidelines for action to avoid the risks posed by phishing scams.

[0467] "Training data" is a reference data set used to train a machine learning model, derived from the knowledge of security experts and expert information sources.

[0468] "Feedback" refers to opinions and ratings provided by users that are used to improve the performance of the system.

[0469] A "prompt sentence" is an input sentence that causes a generative AI model to generate an output.

[0470] "Immediate notification" refers to a means of communication that quickly conveys information about detected phishing scams to users.

[0471] "Digital information" refers to any information expressed in electronic format, such as email or online messages.

[0472] MODE FOR CARRYING OUT THE INVENTION

[0473] The system for implementing the present invention involves three main components: a server, a terminal, and a user. The specific software and hardware usage, data processing and calculation methods will be described in detail below.

[0474] Server-side processing

[0475] The server first periodically monitors the user's mailbox using the Gmail API and IMAP protocol. Specifically, it checks every 10 minutes to see if there is any new email. If a new email is detected through this process, the header information of that email is retrieved.

[0476] Next, the server retrieves the content of the received email. This process uses Python's Pandas library and natural language processing libraries (e.g., NLTK, Spacy). The retrieved email body and attachments are then used for data analysis.

[0477] Machine learning libraries such as Scikit-learn and TensorFlow are applied to detect phishing scam characteristics from the analyzed email content. For example, the frequency of occurrence of specific keywords in the email and URL domain checks are performed to evaluate the possibility of phishing scams.

[0478] If phishing features are detected, the server generates a warning message and advice on safe behavior. A generative AI model (e.g., GPT-3) is used to input a prompt and create a specific warning message.

[0479] The generated warning messages and advice are sent to the user's device using the SMTP protocol, and in some cases, Firebase Cloud Messaging is used to provide in-app notifications.

[0480] Terminal (client) side processing

[0481] A user's terminal receives new emails using email client software (e.g., Outlook, Thunderbird), and the email data is sent from the server via a protocol such as POP3 or IMAP and stored locally.

[0482] The device receives the warning message and advice sent from the server via the HTTPS protocol and displays it using Android's Notification Manager or iOS's Push Notification, allowing users to immediately recognize the risk and receive advice on how to act safely.

[0483] User Roles

[0484] Users should check the warning messages and advice displayed on their devices and take action to avoid the risks posed by phishing scams, such as visiting the official website directly instead of clicking on links in emails identified as phishing.

[0485] Additionally, users can provide feedback to the system using a dedicated form or in-app buttons, and the server uses this information to improve the accuracy of the generative model.

[0486] Specific examples

[0487] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the user's device receives this email, its contents are immediately sent to the server and analyzed using a generative AI model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly." The user can avoid the risk of phishing scams by checking this warning message and visiting the official website directly without clicking on the link.

[0488] Prompt Sentence Examples

[0489] "Generate a warning message in case of a phishing email pretending to be from a bank."

[0490] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[0491] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0492] Step 1:

[0493] The server periodically checks the user's mailbox using the Gmail API and IMAP protocol. This process detects new emails. The input data is the user's email account information, and the output is the header information of the new email. Specifically, the server checks every 10 minutes to see if new emails have arrived and retrieves the email header information.

[0494] Step 2:

[0495] When a new email is detected, the server retrieves the email's contents. The input data is the email's header information, and the output is the email body and the contents of any attachments. This process uses Python's Pandas library and a natural language processing library (e.g., NLTK, Spacy). Specifically, the email body is retrieved in text format, and data from attachments is also extracted.

[0496] Step 3:

[0497] The server applies machine learning models and deep learning algorithms to detect phishing scam characteristics from the email content it retrieves. The input data is the email body and attachment contents, and the output is the phishing scam detection results. Libraries such as Scikit-learn and TensorFlow are used for this processing. Specifically, it checks the frequency of occurrence of specific keywords in the email and the domain of the URL.

[0498] Step 4:

[0499] If the server detects characteristics of a phishing scam, it generates a warning message and advice on safe behavior. The input data is the phishing detection result, and the output is a warning message. A generative AI model (e.g., GPT-3) is used for this generation, and a specific warning message is created based on a prompt text. As a specific example, the prompt text used is, "Please generate a warning message in the event of a phishing email being received that pretends to be from a bank."

[0500] Step 5:

[0501] The server sends the generated warning message and advice to the user's device. The input data is the warning message, and the output is a notification displayed on the user's device. Notifications can be sent via email using the SMTP protocol or in-app notifications using Firebase Cloud Messaging. Specifically, the warning message and advice are sent to the user's device immediately.

[0502] Step 6:

[0503] The user's device receives the warning message and advice sent from the server and displays it immediately. The input data is the warning message received from the server, and the output is a notification that the user can see. This notification is sent using Android's Notification Manager or iOS's Push Notification. Specifically, the message "Phishing Attack Detected" is displayed in the notification area on the device, and the user can check the details.

[0504] Step 7:

[0505] The user checks the warning message and advice displayed on the device and takes safe action. The input data is the warning message displayed on the device, and the output is the user's action. For example, the user may take action by visiting the official website directly instead of clicking on the link in the email.

[0506] Step 8:

[0507] Users can provide feedback on the displayed warning message. The input data is the user's feedback, and the output is information recorded in the server's database. Feedback is sent using a dedicated form or a button within the app, and the server uses this information to improve the generative model. Specifically, it provides feedback to the system saying, "This warning was helpful."

[0508] (Application example 1)

[0509] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0510] In today's cyber environment, fraudulent activities such as phishing scams are on the rise, and methods for stealing personal and financial information via email are becoming more sophisticated. Therefore, there is a need for fast and effective systems that can help users recognize fraudulent activities and take safe actions. However, conventional phishing detection systems often lack practicality due to issues such as delayed warnings and users missing notifications. Furthermore, there is a lack of flexible systems that can respond to new fraudulent activities other than phishing scams.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0512] In this invention, the server includes means for analyzing the obtained information using a generative model and detecting characteristics of fraudulent activity, means for issuing a warning about digital information that has characteristics of fraudulent activity, means for displaying the detected warning and advice on the display of the smart glasses, and means for providing advice on safe behavior to the user. This makes it possible to notify the user of phishing scams and other fraudulent activity in real time and provide visual alerts. It also provides advice on appropriate actions that the user should take immediately, minimizing the risk of harm.

[0513] A "generative model" is a model trained using machine learning or deep learning algorithms to identify and generate specific patterns or features.

[0514] "Fraudulent activity" refers to malicious actions or behaviors against users, such as phishing scams.

[0515] "Smart glasses" are a wearable eyeglass-type device equipped with a small computer for displaying digital information.

[0516] A "display" is a screen or display device that provides visual information to a user.

[0517] A "warning" is a message or display that informs users of the risk of fraudulent activity.

[0518] "Advice" is specific instructions or suggestions to encourage users to behave safely.

[0519] "Real-time" refers to a state in which data and information are processed immediately and provided without delay.

[0520] "Cloud" refers to resources such as data storage and computing resources provided over the Internet.

[0521] "Feedback" refers to the system usage experience and opinions provided by users, and is information used to improve system performance.

[0522] The system for implementing this invention involves three entities: a server, a terminal, and a user. The roles and specific processes of each entity will be explained below.

[0523] Server-side processing

[0524] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve the latest emails. When the server detects a new email, it retrieves its contents and analyzes them using a generative model. The generative model uses machine learning and deep learning algorithms to extract features of fraudulent activity, such as phishing scams, from the email body.

[0525] Specific algorithms and rule-based models are applied to detect fraudulent activity signatures from the analyzed email content. If fraudulent activity signatures are detected, a warning message is generated and includes advice on safe behavior. This advice is reasonable and specific, such as "do not click on links" or "go directly to the official website." The generated warning message and advice are sent to the user's smart glasses.

[0526] Processing on the device (smart glasses)

[0527] The smart glasses periodically communicate with the server to receive warning messages and advice, and the received data is immediately displayed in the user's field of view, allowing the user to immediately know about the risk of fraud.

[0528] User Roles

[0529] Users can check the warning messages and advice displayed on the smart glasses and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email. Users can also provide feedback on the warning messages provided by the system. The server collects the feedback and continuously improves the generative model.

[0530] Specific examples

[0531] For example, consider a case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the server receives the email, it begins analyzing it. Below is an example of a prompt sent to the generative AI model.

[0532] Prompt statement:

[0533] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[0534] The generative AI model receives this prompt and evaluates whether the email is a phishing scam. If phishing characteristics are detected, the server generates a warning message saying "Phishing scam detected. Do not click on the link" and advice "Please visit the official website directly," and sends these to the user's smart glasses. By checking the warning message displayed on the smart glasses and visiting the official website directly without clicking on the link, the user can avoid the risk of phishing scams.

[0535] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0536] Step 1:

[0537] The server periodically checks the user's mailbox to see if new emails have arrived. The input is direct external data: the user's email information, which is retrieved using the email client API. The output is a list of new emails.

[0538] Step 2:

[0539] When the server detects new emails, it retrieves their contents and stores them in the database. The input is a list of new emails, and the email contents are retrieved in text format via the API. The output is the retrieved email contents as text data.

[0540] Step 3:

[0541] The server sends the acquired email content to the generative AI model for analysis. The input is the text data of the email content, and the generative AI model extracts the characteristics of phishing scams. As a concrete example, the following prompt sentence is input:

[0542] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[0543] The output is the analysis result, which is a judgment result as to whether the email is a phishing scam or not.

[0544] Step 4:

[0545] The server generates a warning message if phishing scam characteristics are detected based on the analysis results of the generative AI model. The input is the analysis results, and if fraudulent activity is detected, it generates a warning message saying, "Phishing scam detected. Do not click on the link." The output is a warning message and specific advice on what to do.

[0546] Step 5:

[0547] The server sends the generated warning message and advice to the user's smart glasses. The input is the warning message and action advice, which are sent to the smart glasses via the network. The output is the warning message and advice displayed on the smart glasses.

[0548] Step 6:

[0549] The smart glasses receive warning messages and advice and display them in the user's field of view. The input is the warning messages and advice sent from the server, which are visualized in real time on the glasses' display. The output is information that allows the user to visually confirm the warning.

[0550] Step 7:

[0551] The user checks the warning messages and advice displayed on the smart glasses and follows the instructions. The input is the displayed warning messages and advice, and the user takes safe actions to avoid the risk of phishing scams. The output is the user's safe actions.

[0552] Step 8:

[0553] Users provide feedback on warning messages provided by the system. The input is the user's usage experience and opinions, and the server collects this feedback and stores it in a database. The output is data for improving the system's performance.

[0554] Step 9:

[0555] The server uses the collected feedback to update the generative model. The input is the user feedback data, which is used as training data for the generative AI model. The output is an improved generative AI model.

[0556] The above series of processes realizes a system that detects phishing scams in real time and provides warnings and advice to users.

[0557] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0558] The system for implementing the present invention involves four main entities: a server, a terminal, a user, and an emotion engine. Details and specific examples of the processing performed by each entity will be explained below.

[0559] Server-side processing

[0560] Detecting email receipt

[0561] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0562] Acquiring and analyzing email content

[0563] When the server detects a new email, it retrieves its contents. The email body and associated metadata are sent to a generative model for analysis. The generative model uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify signs of fraud.

[0564] Acquiring emotion data

[0565] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[0566] Phishing signature detection

[0567] The server analyzes the email content to detect signs of fraudulent activity such as phishing scams, records the results, and combines them with emotional data obtained from the emotion engine to generate a warning message based on the user's current emotional state.

[0568] Generating warning messages and advice

[0569] If a phishing scam is detected, the server generates a warning message and advice on safe behaviors according to the user's emotional state. For example, if the user is feeling stressed, the server will provide a more specific and gentle warning message.

[0570] User Notification

[0571] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[0572] Terminal side processing

[0573] Receiving new mail

[0574] The user's device periodically communicates with the mail server to receive new emails, using a standard protocol (such as IMAP or POP3) through the mail client.

[0575] Communicating with the Server

[0576] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0577] Receiving a warning message

[0578] The device receives warning messages and advice sent from the server, which incorporates data from the emotion engine and is based on the user's current emotional state.

[0579] Displaying a warning message

[0580] Display received warning messages to users, so that they are immediately aware of the risk of phishing.

[0581] User Response

[0582] Checking warnings and taking safe actions

[0583] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. The emotion engine also provides more detailed and gentler advice if the user is feeling stressed.

[0584] Providing feedback

[0585] Users can provide feedback on the warning messages and advice provided by the system, which is sent to the server and used to improve the generative model and emotion engine.

[0586] Specific examples

[0587] For example, consider the case where a user receives a phishing email pretending to be from a bank. The email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with a gentler advice saying, "We recommend that you visit the official website directly."

[0588] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[0589] The above is a detailed description of the processing steps of each entity in the system of the present invention. This series of processes enables real-time detection of phishing scams and provides appropriate warnings and advice that take into account the user's emotional state.

[0590] The processing flow will be explained below.

[0591] Server-side processing

[0592] Step 1:

[0593] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0594] Step 2:

[0595] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[0596] Step 3:

[0597] The server analyzes the email content using a generative model, which uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify phishing scam signatures.

[0598] Step 4:

[0599] The server applies an algorithm to detect phishing features based on the analyzed feature information, and if any phishing features are detected, records the results.

[0600] Step 5:

[0601] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[0602] Step 6:

[0603] The server generates warning messages and safety behavior advice based on the detection results. It considers the emotional data obtained from the emotion engine and prepares messages that match the user's current emotional state. If the user is feeling stressed, it provides warning messages with more specific and gentler wording.

[0604] Step 7:

[0605] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[0606] Terminal side processing

[0607] Step 1:

[0608] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[0609] Step 2:

[0610] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0611] Step 3:

[0612] The device receives warning messages and advice sent from the server, and the received data is based on the user's current emotional state.

[0613] Step 4:

[0614] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[0615] User Response

[0616] Step 1:

[0617] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. If users feel stressed, the emotion engine will provide more detailed and gentler advice.

[0618] Step 2:

[0619] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model and emotion engine.

[0620] Step 3:

[0621] If users receive unclear warnings or advice, they can contact us for more information.

[0622] Specific examples

[0623] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice saying, "We recommend that you visit the official website directly."

[0624] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[0625] Example 2

[0626] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0627] In modern society, phishing scams and other fraudulent activities are rapidly increasing, and users are exposed to sophisticated cyber attacks every day. However, conventional fraud detection systems cannot simply detect fraudulent patterns; they must also take into account the user's emotional state to respond appropriately. As a result, when users receive fraud warnings while feeling stressed or anxious, it becomes difficult for them to respond appropriately. This leaves a gap in effective measures to support users' safe online activities.

[0628] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity; means for acquiring the user's emotional state; means for issuing a warning based on the detection result and the user's emotional state; and means for providing the user with advice on safe behavior. The terminal includes: means for monitoring received digital information in real time and identifying digital information having characteristics of fraudulent activity; means for issuing a warning according to the user's emotional state for digital information identified as having characteristics of fraudulent activity; and means for providing the user with specific behavioral advice. This makes it possible to detect fraudulent activity in real time and provide appropriate warning messages and behavioral advice that take the user's emotional state into consideration.

[0629] A "generative model" is a model trained using a machine learning algorithm, which extracts meaningful features from input data and makes predictions.

[0630] "Fraud signatures" refer to patterns or signatures that identify malicious behavior intended to defraud or steal information.

[0631] "Means for issuing warnings" refers to technology that provides a function for notifying users of a warning message in response to detected fraudulent activity.

[0632] "Means for providing advice on safe behavior" refers to a feature that provides specific guidelines or advice for users to deal with fraudulent behavior.

[0633] "Means for acquiring the user's emotional state" refers to technology that estimates the user's emotional state at that time by analyzing the user's facial expressions, tone of voice, keystrokes, mouse movements, etc.

[0634] "Real-time monitoring" refers to the process of analyzing and processing data the moment it is generated.

[0635] "Digital information" refers to any information that is stored, transmitted, or processed in electronic form.

[0636] "Methods of collecting feedback" refers to the techniques and processes used to collect information such as user opinions, ratings, and usage experiences.

[0637] The system for implementing this invention involves four entities: a server, a terminal, a user, and an emotion engine. This system is designed to protect users from cyber attacks and fraudulent activities.

[0638] The server uses the following major hardware and software components:

[0639] Email client APIs: These include the Gmail API, Outlook API, etc., which allow the server to periodically check the user's mailbox to see if new emails have been received.

[0640] Generative AI model: Uses natural language processing (NLP) techniques to extract text features from email bodies and use approaches to detect fraudulent activity.

[0641] Emotion Engine API: Using technologies such as EmotionAPI, we identify the user's emotional state from their facial expressions, tone of voice, keystrokes, and mouse movements.

[0642] Database: Used to store and manage analysis results and emotion data.

[0643] The terminal works as follows:

[0644] Email client software: Uses standard protocols such as IMAP or POP3 to communicate with the email server and receive new email.

[0645] Communication module: Encrypts data using secure communication protocols such as TLS and communicates with the server in a secure manner.

[0646] UI component: Provides a user interface for displaying warning messages and advice to the user.

[0647] Users can receive warning messages and advice via their devices and take appropriate measures. Users can also provide feedback on the provided warning messages and advice, contributing to improving the system's performance.

[0648] The specific operation is explained below:

[0649] 1. The server periodically checks the user's mailbox using the mail client API to see if there are any new emails. For example, the check_new_mail() function is called every minute to see if there are any new emails.

[0650] 2. When a new email is detected, the email content is retrieved and the generative AI model is used to parse the email body and associated metadata.

[0651] 3. At the same time, the server calls the emotion engine API to obtain the user's emotion data and integrate it with the analysis results.

[0652] 4. The server detects signs of fraudulent activity from the analyzed email content and generates appropriate warning messages and advice by referring to the emotional data.

[0653] 5. The generated warning message and advice are sent from the server to the device using a secure protocol (TLS).

[0654] 6. The device displays the sent warning message to the user, and the user takes safe action based on it.

[0655] 7. Users can provide feedback on warning messages from their devices, which is sent to the server and used to improve system performance.

[0656] As a concrete example, the following prompt sentences can be fed into a generative AI model for analysis:

[0657] "Analyze the text of the email below and identify the characteristics of a phishing scam.

[0658] From: example@bank.com

[0659] Subject: Account verification required

[0660] Main text:

[0661] Dear Customer,

[0662] We have detected unusual activity on your account. Please click the link below to review your account information.

[0663] [Link to fake site]

[0664] thank you."

[0665] Using this prompt, the system can identify the characteristics of phishing scams and provide appropriate warnings and advice to users.

[0666] The above is a detailed description of the preferred embodiment of the invention. The system allows for real-time detection of fraudulent activity and provides warnings and advice based on emotional state.

[0667] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0668] Step 1:

[0669] The server periodically checks the user's mailbox to see if any new mail has been received.

[0670] Specifically, the server calls the check_new_mail() function to retrieve unread emails using a mail client API (e.g., Gmail API or Outlook API). The input is the API request, and the output is the ID of the new email or email metadata.

[0671] Step 2:

[0672] When the server detects new mail, it retrieves and analyzes its contents.

[0673] Specifically, the server retrieves the email body and associated metadata using the fetch_email_content(mail_id) function. The input is the email ID, and the output is the email body and metadata. This is then sent to the generative AI model, where text analysis is performed using the analyze_text_for_phishing(email_content) function. This analysis extracts features from the input email body and detects fraudulent activity.

[0674] Step 3:

[0675] While analyzing the email, the server also obtains the user's emotional data from the emotion engine.

[0676] Specifically, the server calls the emotion engine API (e.g., EmotionAPI) and retrieves emotion data using the fetch_emotion_data(user_id) function. The input is the user ID, and the output is the user's current emotional state data.

[0677] Step 4:

[0678] The server combines the analyzed email content with emotional data to detect phishing signatures.

[0679] Specifically, the server uses the detect_phishing(features, emotion_data) function to integrate the analysis results with the emotion data and obtain a fraud detection result. The input is the feature data and emotion data, and the output is the phishing detection result.

[0680] Step 5:

[0681] If a phishing scam is detected, the server generates a warning message and safety behavior advice.

[0682] Specifically, the server uses the generate_warning_message(phishing_detected, emotion_data) function to generate a warning message and advice based on the emotional state. The input is the phishing detection result and emotion data, and the output is a warning message and advice.

[0683] Step 6:

[0684] The server notifies the user's terminal of the generated warning message and advice.

[0685] Specifically, the server calls the send_notification(user_id, message) function to notify the user via email, SMS, or in-app notification. The input is the user ID and message, and the output is the result of sending the notification.

[0686] Step 7:

[0687] The user's device receives the new email and begins communicating with the server.

[0688] The specific operation involves the terminal receiving new email using a standard protocol such as IMAP or POP3 using the connect_to_mail_server() function and sending the content to the server. The input is the connection information for the mail server, and the output is the content of the new email.

[0689] Step 8:

[0690] The user's terminal receives the warning messages and advice sent from the server.

[0691] Specifically, the terminal receives an alert message from the server using the receive_notification() function. The input is a notification request, and the output is an alert message.

[0692] Step 9:

[0693] The user acknowledges the warning message and takes safe action.

[0694] The specific behavior includes the user reading the warning message displayed on the device, not clicking on the link in the phishing email, and directly visiting the official website. The input is the warning message, and the output is the user's behavior.

[0695] Step 10:

[0696] Users provide feedback on system warning messages and advice.

[0697] Specifically, the user uses the feedback function of the device and sends feedback to the server using the submit_feedback(feedback) function. The input is the user's feedback, and the output is the feedback sending result.

[0698] This is the flow of the system's program processing. This series of processes enables the system to detect phishing scams in real time and provide warning messages and advice on what to do that take into account the user's emotional state.

[0699] (Application example 2)

[0700] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] Conventional phishing detection systems issue warnings without considering the user's emotional state, which can cause excessive stress and confusion. Furthermore, the warning messages are uniform and lack advice to encourage users to take appropriate safety actions. Furthermore, they do not integrate with robots used in the home, limiting the means of notifying users. This presents a challenge for users, making it difficult for them to quickly understand the dangers of phishing scams and take appropriate action.

[0702] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity, means for evaluating the user's emotional state using an emotion engine, and means for generating and presenting appropriate warning messages and advice on safe behavior based on the detection results and the user's emotional state. This makes it possible to provide more effective and less stressful warning messages and advice that take the user's emotional state into consideration. In addition, by linking with a robot used in the home, warnings can be conveyed to the user through voice notifications and displays, allowing the user to respond quickly and appropriately.

[0703] A "generative model" is an algorithm that uses AI technology to extract features from input data and perform analysis and predictions.

[0704] The "emotion engine" is a system that evaluates a user's emotional state based on data such as the user's facial expressions, tone of voice, keystrokes, and mouse movements.

[0705] "Fraud signatures" are indicators and patterns that indicate phishing scams or other malicious activity.

[0706] An "appropriate warning message" is a message that informs users of the risk of fraudulent activities such as phishing scams and urges them to take specific action.

[0707] "Safety advice" is information that provides specific actions and measures that users should take when they encounter fraudulent activity.

[0708] "Real-time monitoring" is the process of reviewing and analyzing data as it is generated or received.

[0709] "Feedback" means opinions and information provided by users about the performance and usability of the system.

[0710] "Pattern recognition" is a technique for finding specific patterns or characteristics in data and performing analysis and judgments based on them.

[0711] "Audio Notifications" are a means of audibly conveying system-generated informational or warning messages to the user.

[0712] "Display notification" is a means of displaying system-generated informational or warning messages to the user on a display.

[0713] The system for realizing this application example is based on a robot used in the home, and includes a server, a terminal, a user, and an emotion engine. A specific implementation method of this system will be described below.

[0714] The server analyzes the features of the received data using a generative model. The generative model includes an algorithm that uses natural language processing techniques to extract features from the email body and detects signs of fraudulent activity. It also uses an emotion engine to evaluate the user's emotional state from facial expressions, tone of voice, keystrokes, mouse movements, etc. The emotion engine includes software for identifying the user's emotional state.

[0715] When a device receives an email, it immediately sends the contents of the email to a server and requests analysis. This data is encrypted using a secure communication protocol (e.g., TLS). Once the analysis is complete, the results are returned to the device, which then notifies the user of a warning message based on their emotional state and advice on safe behavior.

[0716] When the home robot receives a notification from the device, it communicates the information to the user through voice output and a display. Voice notifications are made using the robot's internal speaker, and warning messages and advice on safe behavior are displayed on the display.

[0717] For example, if a user receives a phishing email pretending to be from a bank, the content of the email is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice such as, "We recommend visiting the official website directly." These messages are communicated to the user aloud through the home robot's speaker, and similar messages are displayed on the display.

[0718] By checking the robot warning message and visiting the official website directly instead of clicking on the link, users can avoid the risk of phishing scams and stay safe online.

[0719] Example prompts to input to a generative AI model:

[0720] Detect phishing scams by analyzing the following email body:

[0721] ---

[0722] Subject: Important Notice

[0723] Body: Due to an update to our security system, please click the link below to update your authentication information.

[0724] Link: http: / / example.com / secure

[0725] Using this prompt, the generative AI model detects phishing scam characteristics within the email body and provides the basis for generating appropriate warning messages.

[0726] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0727] Step 1:

[0728] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve unread emails. The input is unread email data from the email server, and the output is the retrieved unread email information.

[0729] Step 2:

[0730] When the server detects a new email, it retrieves its contents. The retrieved email body and related metadata are sent to the generative model for analysis. The input is unread email information, and the output is the email body data for analysis.

[0731] Step 3:

[0732] The generative model uses natural language processing techniques to analyze the email body and extract features from the text. It then applies an algorithm to identify fraudulent activity features. The input is the email body data, and the output is fraud feature data.

[0733] Step 4:

[0734] While the server is analyzing the email, it also obtains the user's emotional data from the emotion engine. The emotion engine analyzes the user's facial expressions, tone of voice, keystrokes, mouse movements, etc. to identify their emotional state. The input is the user's behavioral data, and the output is emotional state data.

[0735] Step 5:

[0736] The server detects characteristics of fraudulent activities such as phishing scams from the analyzed email content and records the results. This is combined with emotional data obtained from the emotion engine to generate a warning message based on the user's emotional state. The input is fraudulent activity characteristic data and emotional state data, and the output is a warning message that takes emotions into consideration.

[0737] Step 6:

[0738] If a phishing scam is detected, the server generates a warning message and advice on safe behavior according to the user's emotional state. If the user is feeling stressed, it generates a gentle warning message and advises specific actions. The input is emotional state data and fraud feature data, and the output is a warning message and advice.

[0739] Step 7:

[0740] The terminal receives warning messages and advice sent from the server. This data is encrypted using a secure communication protocol (e.g., TLS). The input is the warning message and advice from the server, and the output is the received notification data.

[0741] Step 8:

[0742] The home robot receives warning messages from the device and conveys information to the user through voice output and a display. The input is notification data from the device, and the output is notification and display to the user.

[0743] Step 9:

[0744] The user checks the robot's warning message and follows the instructions to take safe action, thereby avoiding the risk of phishing scams. The input is the warning message from the robot, and the output is the user's actions to avoid risk.

[0745] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0746] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0747] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0748] [Third embodiment]

[0749] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0750] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0751] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0752] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0753] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0754] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0755] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0756] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0757] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0758] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0759] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0760] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0761] The system for implementing the present invention involves three entities: a server, a terminal, and a user. Details and specific examples of the processing performed by each entity will be explained below.

[0762] Server-side processing

[0763] Detecting email receipt

[0764] The server periodically checks the user's mailbox to see if any new emails have been received, using the mail client API.

[0765] Acquiring and analyzing email content

[0766] When the server detects a new email, it retrieves its contents and analyzes them using a generative model, which uses machine learning and deep learning algorithms to extract phishing features from the email body.

[0767] Phishing signature detection

[0768] It applies specific algorithms and rule-based models to detect phishing features from the analyzed email content, and generates a warning message if phishing features are detected.

[0769] Generating warning messages and advice

[0770] If a phishing scam is detected, the server generates a warning message for the user, including advice on safe behavior, which is reasonable and specific, such as "don't click on the link" or "go directly to the official website."

[0771] User Notification

[0772] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[0773] Terminal (client) side processing

[0774] Receiving new mail

[0775] The user's device periodically communicates with the mail server to receive new emails, which allows phishing scams to be detected and flagged before the user has a chance to check the email.

[0776] Communicating with the Server

[0777] The device receives warning messages and advice sent from the server, and the received data is displayed securely and instantly to the user.

[0778] Displaying a warning message

[0779] The received warning message will be displayed on the device, allowing users to immediately become aware of the risk of phishing scams.

[0780] User Roles

[0781] Checking warnings and taking safe actions

[0782] Users should check the warning messages and advice displayed on their devices and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email.

[0783] Providing feedback

[0784] Users can provide feedback on warning messages provided by the system, allowing the server to collect feedback and continuously improve the generative model.

[0785] Specific examples

[0786] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly."

[0787] Users should check the warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams.

[0788] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[0789] The processing flow will be explained below.

[0790] Server-side processing

[0791] Step 1:

[0792] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0793] Step 2:

[0794] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[0795] Step 3:

[0796] The server uses the generative model to analyze the email content, and the analysis process involves extracting text features from the email body using natural language processing (NLP) techniques.

[0797] Step 4:

[0798] The server applies an algorithm to detect fraudulent activity based on the analyzed feature information, and if it detects any phishing features, it records the results.

[0799] Step 5:

[0800] The server generates a warning message and advice on safe behavior based on the detection results, for example, "Phishing scam detected. Do not click on the link."

[0801] Step 6:

[0802] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[0803] Terminal side processing

[0804] Step 1:

[0805] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[0806] Step 2:

[0807] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0808] Step 3:

[0809] The terminal receives warning messages and advice sent by the server, and the received data is processed for secure and immediate display to the user.

[0810] Step 4:

[0811] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[0812] User Response

[0813] Step 1:

[0814] Users should check the warning message displayed on their device and follow the instructions, for example, by visiting the official website directly rather than clicking on the link in the email.

[0815] Step 2:

[0816] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model.

[0817] Step 3:

[0818] If users receive unclear warnings or advice, they can contact us for more information.

[0819] The above is a detailed description of the processing steps of each subject in the present invention. This series of processing enables phishing scams to be detected in real time and appropriate warnings and advice to be provided to users.

[0820] Example 1

[0821] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0822] Many emails received by Internet users contain fraudulent activity, such as phishing scams. Therefore, unless users take appropriate measures against these emails, they are at high risk of personal information leaks and financial losses. Furthermore, conventional phishing detection systems lacked detection accuracy and immediacy, and were unable to completely prevent users from falling victim to fraud. Furthermore, it was difficult to respond quickly to new fraud methods. To solve these issues, a system is needed that can detect phishing scams with high accuracy and in real time, and provide users with prompt warnings and advice.

[0823] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0824] In this invention, the server includes means for analyzing information obtained using the generative model and detecting characteristics of phishing scams, means for issuing warnings based on the detection results, means for providing users with advice on safe behavior, means for periodically updating training data to deal with new fraudulent activities, means for collecting feedback from users to improve the accuracy of the generative model, and means for instantly notifying users of warning messages and advice to their devices. This makes it possible to detect phishing scams with high accuracy and in real time, and significantly reduce the risk of users becoming victims of fraudulent activities.

[0825] A "generative model" is a model that uses machine learning or deep learning algorithms to generate specific outputs from input data.

[0826] "Phishing" is a fraudulent activity that uses fake websites and emails to steal personal and financial information from users.

[0827] A "warning message" is a message that alerts users to the risk of detected phishing scams.

[0828] "Safety advice" is information that provides users with specific guidelines for action to avoid the risks posed by phishing scams.

[0829] "Training data" is a reference data set used to train a machine learning model, derived from the knowledge of security experts and expert information sources.

[0830] "Feedback" refers to opinions and ratings provided by users that are used to improve the performance of the system.

[0831] A "prompt sentence" is an input sentence that causes a generative AI model to generate an output.

[0832] "Immediate notification" refers to a means of communication that quickly conveys information about detected phishing scams to users.

[0833] "Digital information" refers to any information expressed in electronic format, such as email or online messages.

[0834] MODE FOR CARRYING OUT THE INVENTION

[0835] The system for implementing the present invention involves three main components: a server, a terminal, and a user. The specific software and hardware usage, data processing and calculation methods will be described in detail below.

[0836] Server-side processing

[0837] The server first periodically monitors the user's mailbox using the Gmail API and IMAP protocol. Specifically, it checks every 10 minutes to see if there is any new email. If a new email is detected through this process, the header information of that email is retrieved.

[0838] Next, the server retrieves the content of the received email. This process uses Python's Pandas library and natural language processing libraries (e.g., NLTK, Spacy). The retrieved email body and attachments are then used for data analysis.

[0839] Machine learning libraries such as Scikit-learn and TensorFlow are applied to detect phishing scam characteristics from the analyzed email content. For example, the frequency of occurrence of specific keywords in the email and URL domain checks are performed to evaluate the possibility of phishing scams.

[0840] If phishing features are detected, the server generates a warning message and advice on safe behavior. A generative AI model (e.g., GPT-3) is used to input a prompt and create a specific warning message.

[0841] The generated warning messages and advice are sent to the user's device using the SMTP protocol, and in some cases, Firebase Cloud Messaging is used to provide in-app notifications.

[0842] Terminal (client) side processing

[0843] A user's terminal receives new emails using email client software (e.g., Outlook, Thunderbird), and the email data is sent from the server via a protocol such as POP3 or IMAP and stored locally.

[0844] The device receives the warning message and advice sent from the server via the HTTPS protocol and displays it using Android's Notification Manager or iOS's Push Notification, allowing users to immediately recognize the risk and receive advice on how to act safely.

[0845] User Roles

[0846] Users should check the warning messages and advice displayed on their devices and take action to avoid the risks posed by phishing scams, such as visiting the official website directly instead of clicking on links in emails identified as phishing.

[0847] Additionally, users can provide feedback to the system using a dedicated form or in-app buttons, and the server uses this information to improve the accuracy of the generative model.

[0848] Specific examples

[0849] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the user's device receives this email, its contents are immediately sent to the server and analyzed using a generative AI model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly." The user can avoid the risk of phishing scams by checking this warning message and visiting the official website directly without clicking on the link.

[0850] Prompt Sentence Examples

[0851] "Generate a warning message in case of a phishing email pretending to be from a bank."

[0852] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[0853] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0854] Step 1:

[0855] The server periodically checks the user's mailbox using the Gmail API and IMAP protocol. This process detects new emails. The input data is the user's email account information, and the output is the header information of the new email. Specifically, the server checks every 10 minutes to see if new emails have arrived and retrieves the email header information.

[0856] Step 2:

[0857] When a new email is detected, the server retrieves the email's contents. The input data is the email's header information, and the output is the email body and the contents of any attachments. This process uses Python's Pandas library and a natural language processing library (e.g., NLTK, Spacy). Specifically, the email body is retrieved in text format, and data from attachments is also extracted.

[0858] Step 3:

[0859] The server applies machine learning models and deep learning algorithms to detect phishing scam characteristics from the email content it retrieves. The input data is the email body and attachment contents, and the output is the phishing scam detection results. Libraries such as Scikit-learn and TensorFlow are used for this processing. Specifically, it checks the frequency of occurrence of specific keywords in the email and the domain of the URL.

[0860] Step 4:

[0861] If the server detects characteristics of a phishing scam, it generates a warning message and advice on safe behavior. The input data is the phishing detection result, and the output is a warning message. A generative AI model (e.g., GPT-3) is used for this generation, and a specific warning message is created based on a prompt text. As a specific example, the prompt text used is, "Please generate a warning message in the event of a phishing email being received that pretends to be from a bank."

[0862] Step 5:

[0863] The server sends the generated warning message and advice to the user's device. The input data is the warning message, and the output is a notification displayed on the user's device. Notifications can be sent via email using the SMTP protocol or in-app notifications using Firebase Cloud Messaging. Specifically, the warning message and advice are sent to the user's device immediately.

[0864] Step 6:

[0865] The user's device receives the warning message and advice sent from the server and displays it immediately. The input data is the warning message received from the server, and the output is a notification that the user can see. This notification is sent using Android's Notification Manager or iOS's Push Notification. Specifically, the message "Phishing Attack Detected" is displayed in the notification area on the device, and the user can check the details.

[0866] Step 7:

[0867] The user checks the warning message and advice displayed on the device and takes safe action. The input data is the warning message displayed on the device, and the output is the user's action. For example, the user may take action by visiting the official website directly instead of clicking on the link in the email.

[0868] Step 8:

[0869] Users can provide feedback on the displayed warning message. The input data is the user's feedback, and the output is information recorded in the server's database. Feedback is sent using a dedicated form or a button within the app, and the server uses this information to improve the generative model. Specifically, it provides feedback to the system saying, "This warning was helpful."

[0870] (Application example 1)

[0871] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0872] In today's cyber environment, fraudulent activities such as phishing scams are on the rise, and methods for stealing personal and financial information via email are becoming more sophisticated. Therefore, there is a need for fast and effective systems that can help users recognize fraudulent activities and take safe actions. However, conventional phishing detection systems often lack practicality due to issues such as delayed warnings and users missing notifications. Furthermore, there is a lack of flexible systems that can respond to new fraudulent activities other than phishing scams.

[0873] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0874] In this invention, the server includes means for analyzing the obtained information using a generative model and detecting characteristics of fraudulent activity, means for issuing a warning about digital information that has characteristics of fraudulent activity, means for displaying the detected warning and advice on the display of the smart glasses, and means for providing advice on safe behavior to the user. This makes it possible to notify the user of phishing scams and other fraudulent activity in real time and provide visual alerts. It also provides advice on appropriate actions that the user should take immediately, minimizing the risk of harm.

[0875] A "generative model" is a model trained using machine learning or deep learning algorithms to identify and generate specific patterns or features.

[0876] "Fraudulent activity" refers to malicious actions or behaviors against users, such as phishing scams.

[0877] "Smart glasses" are a wearable eyeglass-type device equipped with a small computer for displaying digital information.

[0878] A "display" is a screen or display device that provides visual information to a user.

[0879] A "warning" is a message or display that informs users of the risk of fraudulent activity.

[0880] "Advice" is specific instructions or suggestions to encourage users to behave safely.

[0881] "Real-time" refers to a state in which data and information are processed immediately and provided without delay.

[0882] "Cloud" refers to resources such as data storage and computing resources provided over the Internet.

[0883] "Feedback" refers to the system usage experience and opinions provided by users, and is information used to improve system performance.

[0884] The system for implementing this invention involves three entities: a server, a terminal, and a user. The roles and specific processes of each entity will be explained below.

[0885] Server-side processing

[0886] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve the latest emails. When the server detects a new email, it retrieves its contents and analyzes them using a generative model. The generative model uses machine learning and deep learning algorithms to extract features of fraudulent activity, such as phishing scams, from the email body.

[0887] Specific algorithms and rule-based models are applied to detect fraudulent activity signatures from the analyzed email content. If fraudulent activity signatures are detected, a warning message is generated and includes advice on safe behavior. This advice is reasonable and specific, such as "do not click on links" or "go directly to the official website." The generated warning message and advice are sent to the user's smart glasses.

[0888] Processing on the device (smart glasses)

[0889] The smart glasses periodically communicate with the server to receive warning messages and advice, and the received data is immediately displayed in the user's field of view, allowing the user to immediately know about the risk of fraud.

[0890] User Roles

[0891] Users can check the warning messages and advice displayed on the smart glasses and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email. Users can also provide feedback on the warning messages provided by the system. The server collects the feedback and continuously improves the generative model.

[0892] Specific examples

[0893] For example, consider a case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the server receives the email, it begins analyzing it. Below is an example of a prompt sent to the generative AI model.

[0894] Prompt statement:

[0895] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[0896] The generative AI model receives this prompt and evaluates whether the email is a phishing scam. If phishing characteristics are detected, the server generates a warning message saying "Phishing scam detected. Do not click on the link" and advice "Please visit the official website directly," and sends these to the user's smart glasses. By checking the warning message displayed on the smart glasses and visiting the official website directly without clicking on the link, the user can avoid the risk of phishing scams.

[0897] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0898] Step 1:

[0899] The server periodically checks the user's mailbox to see if new emails have arrived. The input is direct external data: the user's email information, which is retrieved using the email client API. The output is a list of new emails.

[0900] Step 2:

[0901] When the server detects new emails, it retrieves their contents and stores them in the database. The input is a list of new emails, and the email contents are retrieved in text format via the API. The output is the retrieved email contents as text data.

[0902] Step 3:

[0903] The server sends the acquired email content to the generative AI model for analysis. The input is the text data of the email content, and the generative AI model extracts the characteristics of phishing scams. As a concrete example, the following prompt sentence is input:

[0904] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[0905] The output is the analysis result, which is a judgment result as to whether the email is a phishing scam or not.

[0906] Step 4:

[0907] The server generates a warning message if phishing scam characteristics are detected based on the analysis results of the generative AI model. The input is the analysis results, and if fraudulent activity is detected, it generates a warning message saying, "Phishing scam detected. Do not click on the link." The output is a warning message and specific advice on what to do.

[0908] Step 5:

[0909] The server sends the generated warning message and advice to the user's smart glasses. The input is the warning message and action advice, which are sent to the smart glasses via the network. The output is the warning message and advice displayed on the smart glasses.

[0910] Step 6:

[0911] The smart glasses receive warning messages and advice and display them in the user's field of view. The input is the warning messages and advice sent from the server, which are visualized in real time on the glasses' display. The output is information that allows the user to visually confirm the warning.

[0912] Step 7:

[0913] The user checks the warning messages and advice displayed on the smart glasses and follows the instructions. The input is the displayed warning messages and advice, and the user takes safe actions to avoid the risk of phishing scams. The output is the user's safe actions.

[0914] Step 8:

[0915] Users provide feedback on warning messages provided by the system. The input is the user's usage experience and opinions, and the server collects this feedback and stores it in a database. The output is data for improving the system's performance.

[0916] Step 9:

[0917] The server uses the collected feedback to update the generative model. The input is the user feedback data, which is used as training data for the generative AI model. The output is an improved generative AI model.

[0918] The above series of processes realizes a system that detects phishing scams in real time and provides warnings and advice to users.

[0919] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0920] The system for implementing the present invention involves four main entities: a server, a terminal, a user, and an emotion engine. Details and specific examples of the processing performed by each entity will be explained below.

[0921] Server-side processing

[0922] Detecting email receipt

[0923] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0924] Acquiring and analyzing email content

[0925] When the server detects a new email, it retrieves its contents. The email body and associated metadata are sent to a generative model for analysis. The generative model uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify signs of fraud.

[0926] Acquiring emotion data

[0927] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[0928] Phishing signature detection

[0929] The server analyzes the email content to detect signs of fraudulent activity such as phishing scams, records the results, and combines them with emotional data obtained from the emotion engine to generate a warning message based on the user's current emotional state.

[0930] Generating warning messages and advice

[0931] If a phishing scam is detected, the server generates a warning message and advice on safe behaviors according to the user's emotional state. For example, if the user is feeling stressed, the server will provide a more specific and gentle warning message.

[0932] User Notification

[0933] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[0934] Terminal side processing

[0935] Receiving new mail

[0936] The user's device periodically communicates with the mail server to receive new emails, using a standard protocol (such as IMAP or POP3) through the mail client.

[0937] Communicating with the Server

[0938] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0939] Receiving a warning message

[0940] The device receives warning messages and advice sent from the server, which incorporates data from the emotion engine and is based on the user's current emotional state.

[0941] Displaying a warning message

[0942] Display received warning messages to users, so that they are immediately aware of the risk of phishing.

[0943] User Response

[0944] Checking warnings and taking safe actions

[0945] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. The emotion engine also provides more detailed and gentler advice if the user is feeling stressed.

[0946] Providing feedback

[0947] Users can provide feedback on the warning messages and advice provided by the system, which is sent to the server and used to improve the generative model and emotion engine.

[0948] Specific examples

[0949] For example, consider the case where a user receives a phishing email pretending to be from a bank. The email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with a gentler advice saying, "We recommend that you visit the official website directly."

[0950] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[0951] The above is a detailed description of the processing steps of each entity in the system of the present invention. This series of processes enables real-time detection of phishing scams and provides appropriate warnings and advice that take into account the user's emotional state.

[0952] The processing flow will be explained below.

[0953] Server-side processing

[0954] Step 1:

[0955] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[0956] Step 2:

[0957] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[0958] Step 3:

[0959] The server analyzes the email content using a generative model, which uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify phishing scam signatures.

[0960] Step 4:

[0961] The server applies an algorithm to detect phishing features based on the analyzed feature information, and if any phishing features are detected, records the results.

[0962] Step 5:

[0963] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[0964] Step 6:

[0965] The server generates warning messages and safety behavior advice based on the detection results. It considers the emotional data obtained from the emotion engine and prepares messages that match the user's current emotional state. If the user is feeling stressed, it provides warning messages with more specific and gentler wording.

[0966] Step 7:

[0967] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[0968] Terminal side processing

[0969] Step 1:

[0970] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[0971] Step 2:

[0972] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[0973] Step 3:

[0974] The device receives warning messages and advice sent from the server, and the received data is based on the user's current emotional state.

[0975] Step 4:

[0976] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[0977] User Response

[0978] Step 1:

[0979] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. If users feel stressed, the emotion engine will provide more detailed and gentler advice.

[0980] Step 2:

[0981] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model and emotion engine.

[0982] Step 3:

[0983] If users receive unclear warnings or advice, they can contact us for more information.

[0984] Specific examples

[0985] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice saying, "We recommend that you visit the official website directly."

[0986] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[0987] Example 2

[0988] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0989] In modern society, phishing scams and other fraudulent activities are rapidly increasing, and users are exposed to sophisticated cyber attacks every day. However, conventional fraud detection systems cannot simply detect fraudulent patterns; they must also take into account the user's emotional state to respond appropriately. As a result, when users receive fraud warnings while feeling stressed or anxious, it becomes difficult for them to respond appropriately. This leaves a gap in effective measures to support users' safe online activities.

[0990] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity; means for acquiring the user's emotional state; means for issuing a warning based on the detection result and the user's emotional state; and means for providing the user with advice on safe behavior. The terminal includes: means for monitoring received digital information in real time and identifying digital information having characteristics of fraudulent activity; means for issuing a warning according to the user's emotional state for digital information identified as having characteristics of fraudulent activity; and means for providing the user with specific behavioral advice. This makes it possible to detect fraudulent activity in real time and provide appropriate warning messages and behavioral advice that take the user's emotional state into consideration.

[0991] A "generative model" is a model trained using a machine learning algorithm, which extracts meaningful features from input data and makes predictions.

[0992] "Fraud signatures" refer to patterns or signatures that identify malicious behavior intended to defraud or steal information.

[0993] "Means for issuing warnings" refers to technology that provides a function for notifying users of a warning message in response to detected fraudulent activity.

[0994] "Means for providing advice on safe behavior" refers to a feature that provides specific guidelines or advice for users to deal with fraudulent behavior.

[0995] "Means for acquiring the user's emotional state" refers to technology that estimates the user's emotional state at that time by analyzing the user's facial expressions, tone of voice, keystrokes, mouse movements, etc.

[0996] "Real-time monitoring" refers to the process of analyzing and processing data the moment it is generated.

[0997] "Digital information" refers to any information that is stored, transmitted, or processed in electronic form.

[0998] "Methods of collecting feedback" refers to the techniques and processes used to collect information such as user opinions, ratings, and usage experiences.

[0999] The system for implementing this invention involves four entities: a server, a terminal, a user, and an emotion engine. This system is designed to protect users from cyber attacks and fraudulent activities.

[1000] The server uses the following major hardware and software components:

[1001] Email client APIs: These include the Gmail API, Outlook API, etc., which allow the server to periodically check the user's mailbox to see if new emails have been received.

[1002] Generative AI model: Uses natural language processing (NLP) techniques to extract text features from email bodies and use approaches to detect fraudulent activity.

[1003] Emotion Engine API: Using technologies such as EmotionAPI, we identify the user's emotional state from their facial expressions, tone of voice, keystrokes, and mouse movements.

[1004] Database: Used to store and manage analysis results and emotion data.

[1005] The terminal works as follows:

[1006] Email client software: Uses standard protocols such as IMAP or POP3 to communicate with the email server and receive new email.

[1007] Communication module: Encrypts data using secure communication protocols such as TLS and communicates with the server in a secure manner.

[1008] UI component: Provides a user interface for displaying warning messages and advice to the user.

[1009] Users can receive warning messages and advice via their devices and take appropriate measures. Users can also provide feedback on the provided warning messages and advice, contributing to improving the system's performance.

[1010] The specific operation is explained below:

[1011] 1. The server periodically checks the user's mailbox using the mail client API to see if there are any new emails. For example, the check_new_mail() function is called every minute to see if there are any new emails.

[1012] 2. When a new email is detected, the email content is retrieved and the generative AI model is used to parse the email body and associated metadata.

[1013] 3. At the same time, the server calls the emotion engine API to obtain the user's emotion data and integrate it with the analysis results.

[1014] 4. The server detects signs of fraudulent activity from the analyzed email content and generates appropriate warning messages and advice by referring to the emotional data.

[1015] 5. The generated warning message and advice are sent from the server to the device using a secure protocol (TLS).

[1016] 6. The device displays the sent warning message to the user, and the user takes safe action based on it.

[1017] 7. Users can provide feedback on warning messages from their devices, which is sent to the server and used to improve system performance.

[1018] As a concrete example, the following prompt sentences can be fed into a generative AI model for analysis:

[1019] "Analyze the text of the email below and identify the characteristics of a phishing scam.

[1020] From: example@bank.com

[1021] Subject: Account verification required

[1022] Main text:

[1023] Dear Customer,

[1024] We have detected unusual activity on your account. Please click the link below to review your account information.

[1025] [Link to fake site]

[1026] thank you."

[1027] Using this prompt, the system can identify the characteristics of phishing scams and provide appropriate warnings and advice to users.

[1028] The above is a detailed description of the preferred embodiment of the invention. The system allows for real-time detection of fraudulent activity and provides warnings and advice based on emotional state.

[1029] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1030] Step 1:

[1031] The server periodically checks the user's mailbox to see if any new mail has been received.

[1032] Specifically, the server calls the check_new_mail() function to retrieve unread emails using a mail client API (e.g., Gmail API or Outlook API). The input is the API request, and the output is the ID of the new email or email metadata.

[1033] Step 2:

[1034] When the server detects new mail, it retrieves and analyzes its contents.

[1035] Specifically, the server retrieves the email body and associated metadata using the fetch_email_content(mail_id) function. The input is the email ID, and the output is the email body and metadata. This is then sent to the generative AI model, where text analysis is performed using the analyze_text_for_phishing(email_content) function. This analysis extracts features from the input email body and detects fraudulent activity.

[1036] Step 3:

[1037] While analyzing the email, the server also obtains the user's emotional data from the emotion engine.

[1038] Specifically, the server calls the emotion engine API (e.g., EmotionAPI) and retrieves emotion data using the fetch_emotion_data(user_id) function. The input is the user ID, and the output is the user's current emotional state data.

[1039] Step 4:

[1040] The server combines the analyzed email content with emotional data to detect phishing signatures.

[1041] Specifically, the server uses the detect_phishing(features, emotion_data) function to integrate the analysis results with the emotion data and obtain a fraud detection result. The input is the feature data and emotion data, and the output is the phishing detection result.

[1042] Step 5:

[1043] If a phishing scam is detected, the server generates a warning message and safety behavior advice.

[1044] Specifically, the server uses the generate_warning_message(phishing_detected, emotion_data) function to generate a warning message and advice based on the emotional state. The input is the phishing detection result and emotion data, and the output is a warning message and advice.

[1045] Step 6:

[1046] The server notifies the user's terminal of the generated warning message and advice.

[1047] Specifically, the server calls the send_notification(user_id, message) function to notify the user via email, SMS, or in-app notification. The input is the user ID and message, and the output is the result of sending the notification.

[1048] Step 7:

[1049] The user's device receives the new email and begins communicating with the server.

[1050] The specific operation involves the terminal receiving new email using a standard protocol such as IMAP or POP3 using the connect_to_mail_server() function and sending the content to the server. The input is the connection information for the mail server, and the output is the content of the new email.

[1051] Step 8:

[1052] The user's terminal receives the warning messages and advice sent from the server.

[1053] Specifically, the terminal receives an alert message from the server using the receive_notification() function. The input is a notification request, and the output is an alert message.

[1054] Step 9:

[1055] The user acknowledges the warning message and takes safe action.

[1056] The specific behavior includes the user reading the warning message displayed on the device, not clicking on the link in the phishing email, and directly visiting the official website. The input is the warning message, and the output is the user's behavior.

[1057] Step 10:

[1058] Users provide feedback on system warning messages and advice.

[1059] Specifically, the user uses the feedback function of the device and sends feedback to the server using the submit_feedback(feedback) function. The input is the user's feedback, and the output is the feedback sending result.

[1060] This is the flow of the system's program processing. This series of processes enables the system to detect phishing scams in real time and provide warning messages and advice on what to do that take into account the user's emotional state.

[1061] (Application example 2)

[1062] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1063] Conventional phishing detection systems issue warnings without considering the user's emotional state, which can cause excessive stress and confusion. Furthermore, the warning messages are uniform and lack advice to encourage users to take appropriate safety actions. Furthermore, they do not integrate with robots used in the home, limiting the means of notifying users. This presents a challenge for users, making it difficult for them to quickly understand the dangers of phishing scams and take appropriate action.

[1064] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity, means for evaluating the user's emotional state using an emotion engine, and means for generating and presenting appropriate warning messages and advice on safe behavior based on the detection results and the user's emotional state. This makes it possible to provide more effective and less stressful warning messages and advice that take the user's emotional state into consideration. In addition, by linking with a robot used in the home, warnings can be conveyed to the user through voice notifications and displays, allowing the user to respond quickly and appropriately.

[1065] A "generative model" is an algorithm that uses AI technology to extract features from input data and perform analysis and predictions.

[1066] The "emotion engine" is a system that evaluates a user's emotional state based on data such as the user's facial expressions, tone of voice, keystrokes, and mouse movements.

[1067] "Fraud signatures" are indicators and patterns that indicate phishing scams or other malicious activity.

[1068] An "appropriate warning message" is a message that informs users of the risk of fraudulent activities such as phishing scams and urges them to take specific action.

[1069] "Safety advice" is information that provides specific actions and measures that users should take when they encounter fraudulent activity.

[1070] "Real-time monitoring" is the process of reviewing and analyzing data as it is generated or received.

[1071] "Feedback" means opinions and information provided by users about the performance and usability of the system.

[1072] "Pattern recognition" is a technique for finding specific patterns or characteristics in data and performing analysis and judgments based on them.

[1073] "Audio Notifications" are a means of audibly conveying system-generated informational or warning messages to the user.

[1074] "Display notification" is a means of displaying system-generated informational or warning messages to the user on a display.

[1075] The system for realizing this application example is based on a robot used in the home, and includes a server, a terminal, a user, and an emotion engine. A specific implementation method of this system will be described below.

[1076] The server analyzes the features of the received data using a generative model. The generative model includes an algorithm that uses natural language processing techniques to extract features from the email body and detects signs of fraudulent activity. It also uses an emotion engine to evaluate the user's emotional state from facial expressions, tone of voice, keystrokes, mouse movements, etc. The emotion engine includes software for identifying the user's emotional state.

[1077] When a device receives an email, it immediately sends the contents of the email to a server and requests analysis. This data is encrypted using a secure communication protocol (e.g., TLS). Once the analysis is complete, the results are returned to the device, which then notifies the user of a warning message based on their emotional state and advice on safe behavior.

[1078] When the home robot receives a notification from the device, it communicates the information to the user through voice output and a display. Voice notifications are made using the robot's internal speaker, and warning messages and advice on safe behavior are displayed on the display.

[1079] For example, if a user receives a phishing email pretending to be from a bank, the content of the email is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice such as, "We recommend visiting the official website directly." These messages are communicated to the user aloud through the home robot's speaker, and similar messages are displayed on the display.

[1080] By checking the robot warning message and visiting the official website directly instead of clicking on the link, users can avoid the risk of phishing scams and stay safe online.

[1081] Example prompts to input to a generative AI model:

[1082] Detect phishing scams by analyzing the following email body:

[1083] ---

[1084] Subject: Important Notice

[1085] Body: Due to an update to our security system, please click the link below to update your authentication information.

[1086] Link: http: / / example.com / secure

[1087] Using this prompt, the generative AI model detects phishing scam characteristics within the email body and provides the basis for generating appropriate warning messages.

[1088] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1089] Step 1:

[1090] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve unread emails. The input is unread email data from the email server, and the output is the retrieved unread email information.

[1091] Step 2:

[1092] When the server detects a new email, it retrieves its contents. The retrieved email body and related metadata are sent to the generative model for analysis. The input is unread email information, and the output is the email body data for analysis.

[1093] Step 3:

[1094] The generative model uses natural language processing techniques to analyze the email body and extract features from the text. It then applies an algorithm to identify fraudulent activity features. The input is the email body data, and the output is fraud feature data.

[1095] Step 4:

[1096] While the server is analyzing the email, it also obtains the user's emotional data from the emotion engine. The emotion engine analyzes the user's facial expressions, tone of voice, keystrokes, mouse movements, etc. to identify their emotional state. The input is the user's behavioral data, and the output is emotional state data.

[1097] Step 5:

[1098] The server detects characteristics of fraudulent activities such as phishing scams from the analyzed email content and records the results. This is combined with emotional data obtained from the emotion engine to generate a warning message based on the user's emotional state. The input is fraudulent activity characteristic data and emotional state data, and the output is a warning message that takes emotions into consideration.

[1099] Step 6:

[1100] If a phishing scam is detected, the server generates a warning message and advice on safe behavior according to the user's emotional state. If the user is feeling stressed, it generates a gentle warning message and advises specific actions. The input is emotional state data and fraud feature data, and the output is a warning message and advice.

[1101] Step 7:

[1102] The terminal receives warning messages and advice sent from the server. This data is encrypted using a secure communication protocol (e.g., TLS). The input is the warning message and advice from the server, and the output is the received notification data.

[1103] Step 8:

[1104] The home robot receives warning messages from the device and conveys information to the user through voice output and a display. The input is notification data from the device, and the output is notification and display to the user.

[1105] Step 9:

[1106] The user checks the robot's warning message and follows the instructions to take safe action, thereby avoiding the risk of phishing scams. The input is the warning message from the robot, and the output is the user's actions to avoid risk.

[1107] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1108] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1109] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1110] [Fourth embodiment]

[1111] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1112] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1113] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1114] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1115] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1116] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1117] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1118] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1119] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1120] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1121] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1122] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1123] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1124] The system for implementing the present invention involves three entities: a server, a terminal, and a user. Details and specific examples of the processing performed by each entity will be explained below.

[1125] Server-side processing

[1126] Detecting email receipt

[1127] The server periodically checks the user's mailbox to see if any new emails have been received, using the mail client API.

[1128] Acquiring and analyzing email content

[1129] When the server detects a new email, it retrieves its contents and analyzes them using a generative model, which uses machine learning and deep learning algorithms to extract phishing features from the email body.

[1130] Phishing signature detection

[1131] It applies specific algorithms and rule-based models to detect phishing features from the analyzed email content, and generates a warning message if phishing features are detected.

[1132] Generating warning messages and advice

[1133] If a phishing scam is detected, the server generates a warning message for the user, including advice on safe behavior, which is reasonable and specific, such as "don't click on the link" or "go directly to the official website."

[1134] User Notification

[1135] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[1136] Terminal (client) side processing

[1137] Receiving new mail

[1138] The user's device periodically communicates with the mail server to receive new emails, which allows phishing scams to be detected and flagged before the user has a chance to check the email.

[1139] Communicating with the Server

[1140] The device receives warning messages and advice sent from the server, and the received data is displayed securely and instantly to the user.

[1141] Displaying a warning message

[1142] The received warning message will be displayed on the device, allowing users to immediately become aware of the risk of phishing scams.

[1143] User Roles

[1144] Checking warnings and taking safe actions

[1145] Users should check the warning messages and advice displayed on their devices and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email.

[1146] Providing feedback

[1147] Users can provide feedback on warning messages provided by the system, allowing the server to collect feedback and continuously improve the generative model.

[1148] Specific examples

[1149] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly."

[1150] Users should check the warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams.

[1151] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[1152] The processing flow will be explained below.

[1153] Server-side processing

[1154] Step 1:

[1155] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[1156] Step 2:

[1157] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[1158] Step 3:

[1159] The server uses the generative model to analyze the email content, and the analysis process involves extracting text features from the email body using natural language processing (NLP) techniques.

[1160] Step 4:

[1161] The server applies an algorithm to detect fraudulent activity based on the analyzed feature information, and if it detects any phishing features, it records the results.

[1162] Step 5:

[1163] The server generates a warning message and advice on safe behavior based on the detection results, for example, "Phishing scam detected. Do not click on the link."

[1164] Step 6:

[1165] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[1166] Terminal side processing

[1167] Step 1:

[1168] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[1169] Step 2:

[1170] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[1171] Step 3:

[1172] The terminal receives warning messages and advice sent by the server, and the received data is processed for secure and immediate display to the user.

[1173] Step 4:

[1174] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[1175] User Response

[1176] Step 1:

[1177] Users should check the warning message displayed on their device and follow the instructions, for example, by visiting the official website directly rather than clicking on the link in the email.

[1178] Step 2:

[1179] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model.

[1180] Step 3:

[1181] If users receive unclear warnings or advice, they can contact us for more information.

[1182] The above is a detailed description of the processing steps of each subject in the present invention. This series of processing enables phishing scams to be detected in real time and appropriate warnings and advice to be provided to users.

[1183] Example 1

[1184] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1185] Many emails received by Internet users contain fraudulent activity, such as phishing scams. Therefore, unless users take appropriate measures against these emails, they are at high risk of personal information leaks and financial losses. Furthermore, conventional phishing detection systems lacked detection accuracy and immediacy, and were unable to completely prevent users from falling victim to fraud. Furthermore, it was difficult to respond quickly to new fraud methods. To solve these issues, a system is needed that can detect phishing scams with high accuracy and in real time, and provide users with prompt warnings and advice.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1187] In this invention, the server includes means for analyzing information obtained using the generative model and detecting characteristics of phishing scams, means for issuing warnings based on the detection results, means for providing users with advice on safe behavior, means for periodically updating training data to deal with new fraudulent activities, means for collecting feedback from users to improve the accuracy of the generative model, and means for instantly notifying users of warning messages and advice to their devices. This makes it possible to detect phishing scams with high accuracy and in real time, and significantly reduce the risk of users becoming victims of fraudulent activities.

[1188] A "generative model" is a model that uses machine learning or deep learning algorithms to generate specific outputs from input data.

[1189] "Phishing" is a fraudulent activity that uses fake websites and emails to steal personal and financial information from users.

[1190] A "warning message" is a message that alerts users to the risk of detected phishing scams.

[1191] "Safety advice" is information that provides users with specific guidelines for action to avoid the risks posed by phishing scams.

[1192] "Training data" is a reference data set used to train a machine learning model, derived from the knowledge of security experts and expert information sources.

[1193] "Feedback" refers to opinions and ratings provided by users that are used to improve the performance of the system.

[1194] A "prompt sentence" is an input sentence that causes a generative AI model to generate an output.

[1195] "Immediate notification" refers to a means of communication that quickly conveys information about detected phishing scams to users.

[1196] "Digital information" refers to any information expressed in electronic format, such as email or online messages.

[1197] MODE FOR CARRYING OUT THE INVENTION

[1198] The system for implementing the present invention involves three main components: a server, a terminal, and a user. The specific software and hardware usage, data processing and calculation methods will be described in detail below.

[1199] Server-side processing

[1200] The server first periodically monitors the user's mailbox using the Gmail API and IMAP protocol. Specifically, it checks every 10 minutes to see if there is any new email. If a new email is detected through this process, the header information of that email is retrieved.

[1201] Next, the server retrieves the content of the received email. This process uses Python's Pandas library and natural language processing libraries (e.g., NLTK, Spacy). The retrieved email body and attachments are then used for data analysis.

[1202] Machine learning libraries such as Scikit-learn and TensorFlow are applied to detect phishing scam characteristics from the analyzed email content. For example, the frequency of occurrence of specific keywords in the email and URL domain checks are performed to evaluate the possibility of phishing scams.

[1203] If phishing features are detected, the server generates a warning message and advice on safe behavior. A generative AI model (e.g., GPT-3) is used to input a prompt and create a specific warning message.

[1204] The generated warning messages and advice are sent to the user's device using the SMTP protocol, and in some cases, Firebase Cloud Messaging is used to provide in-app notifications.

[1205] Terminal (client) side processing

[1206] A user's terminal receives new emails using email client software (e.g., Outlook, Thunderbird), and the email data is sent from the server via a protocol such as POP3 or IMAP and stored locally.

[1207] The device receives the warning message and advice sent from the server via the HTTPS protocol and displays it using Android's Notification Manager or iOS's Push Notification, allowing users to immediately recognize the risk and receive advice on how to act safely.

[1208] User Roles

[1209] Users should check the warning messages and advice displayed on their devices and take action to avoid the risks posed by phishing scams, such as visiting the official website directly instead of clicking on links in emails identified as phishing.

[1210] Additionally, users can provide feedback to the system using a dedicated form or in-app buttons, and the server uses this information to improve the accuracy of the generative model.

[1211] Specific examples

[1212] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the user's device receives this email, its contents are immediately sent to the server and analyzed using a generative AI model. If the model detects characteristics of a phishing scam, the server notifies the user with a warning message saying, "Phishing scam detected. Do not click on the link," along with the advice, "Please visit the official website directly." The user can avoid the risk of phishing scams by checking this warning message and visiting the official website directly without clicking on the link.

[1213] Prompt Sentence Examples

[1214] "Generate a warning message in case of a phishing email pretending to be from a bank."

[1215] As described above, the system of the present invention improves safety on the Internet by detecting phishing scams in real time and providing appropriate warnings and advice to users.

[1216] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1217] Step 1:

[1218] The server periodically checks the user's mailbox using the Gmail API and IMAP protocol. This process detects new emails. The input data is the user's email account information, and the output is the header information of the new email. Specifically, the server checks every 10 minutes to see if new emails have arrived and retrieves the email header information.

[1219] Step 2:

[1220] When a new email is detected, the server retrieves the email's contents. The input data is the email's header information, and the output is the email body and the contents of any attachments. This process uses Python's Pandas library and a natural language processing library (e.g., NLTK, Spacy). Specifically, the email body is retrieved in text format, and data from attachments is also extracted.

[1221] Step 3:

[1222] The server applies machine learning models and deep learning algorithms to detect phishing scam characteristics from the email content it retrieves. The input data is the email body and attachment contents, and the output is the phishing scam detection results. Libraries such as Scikit-learn and TensorFlow are used for this processing. Specifically, it checks the frequency of occurrence of specific keywords in the email and the domain of the URL.

[1223] Step 4:

[1224] If the server detects characteristics of a phishing scam, it generates a warning message and advice on safe behavior. The input data is the phishing detection result, and the output is a warning message. A generative AI model (e.g., GPT-3) is used for this generation, and a specific warning message is created based on a prompt text. As a specific example, the prompt text used is, "Please generate a warning message in the event of a phishing email being received that pretends to be from a bank."

[1225] Step 5:

[1226] The server sends the generated warning message and advice to the user's device. The input data is the warning message, and the output is a notification displayed on the user's device. Notifications can be sent via email using the SMTP protocol or in-app notifications using Firebase Cloud Messaging. Specifically, the warning message and advice are sent to the user's device immediately.

[1227] Step 6:

[1228] The user's device receives the warning message and advice sent from the server and displays it immediately. The input data is the warning message received from the server, and the output is a notification that the user can see. This notification is sent using Android's Notification Manager or iOS's Push Notification. Specifically, the message "Phishing Attack Detected" is displayed in the notification area on the device, and the user can check the details.

[1229] Step 7:

[1230] The user checks the warning message and advice displayed on the device and takes safe action. The input data is the warning message displayed on the device, and the output is the user's action. For example, the user may take action by visiting the official website directly instead of clicking on the link in the email.

[1231] Step 8:

[1232] Users can provide feedback on the displayed warning message. The input data is the user's feedback, and the output is information recorded in the server's database. Feedback is sent using a dedicated form or a button within the app, and the server uses this information to improve the generative model. Specifically, it provides feedback to the system saying, "This warning was helpful."

[1233] (Application example 1)

[1234] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1235] In today's cyber environment, fraudulent activities such as phishing scams are on the rise, and methods for stealing personal and financial information via email are becoming more sophisticated. Therefore, there is a need for fast and effective systems that can help users recognize fraudulent activities and take safe actions. However, conventional phishing detection systems often lack practicality due to issues such as delayed warnings and users missing notifications. Furthermore, there is a lack of flexible systems that can respond to new fraudulent activities other than phishing scams.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1237] In this invention, the server includes means for analyzing the obtained information using a generative model and detecting characteristics of fraudulent activity, means for issuing a warning about digital information that has characteristics of fraudulent activity, means for displaying the detected warning and advice on the display of the smart glasses, and means for providing advice on safe behavior to the user. This makes it possible to notify the user of phishing scams and other fraudulent activity in real time and provide visual alerts. It also provides advice on appropriate actions that the user should take immediately, minimizing the risk of harm.

[1238] A "generative model" is a model trained using machine learning or deep learning algorithms to identify and generate specific patterns or features.

[1239] "Fraudulent activity" refers to malicious actions or behaviors against users, such as phishing scams.

[1240] "Smart glasses" are a wearable eyeglass-type device equipped with a small computer for displaying digital information.

[1241] A "display" is a screen or display device that provides visual information to a user.

[1242] A "warning" is a message or display that informs users of the risk of fraudulent activity.

[1243] "Advice" is specific instructions or suggestions to encourage users to behave safely.

[1244] "Real-time" refers to a state in which data and information are processed immediately and provided without delay.

[1245] "Cloud" refers to resources such as data storage and computing resources provided over the Internet.

[1246] "Feedback" refers to the system usage experience and opinions provided by users, and is information used to improve system performance.

[1247] The system for implementing this invention involves three entities: a server, a terminal, and a user. The roles and specific processes of each entity will be explained below.

[1248] Server-side processing

[1249] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve the latest emails. When the server detects a new email, it retrieves its contents and analyzes them using a generative model. The generative model uses machine learning and deep learning algorithms to extract features of fraudulent activity, such as phishing scams, from the email body.

[1250] Specific algorithms and rule-based models are applied to detect fraudulent activity signatures from the analyzed email content. If fraudulent activity signatures are detected, a warning message is generated and includes advice on safe behavior. This advice is reasonable and specific, such as "do not click on links" or "go directly to the official website." The generated warning message and advice are sent to the user's smart glasses.

[1251] Processing on the device (smart glasses)

[1252] The smart glasses periodically communicate with the server to receive warning messages and advice, and the received data is immediately displayed in the user's field of view, allowing the user to immediately know about the risk of fraud.

[1253] User Roles

[1254] Users can check the warning messages and advice displayed on the smart glasses and follow the instructions. For example, if a received email is determined to be a phishing scam, users can avoid the risk by visiting the official website directly rather than clicking on the link in the email. Users can also provide feedback on the warning messages provided by the system. The server collects the feedback and continuously improves the generative model.

[1255] Specific examples

[1256] For example, consider a case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles an official website. When the server receives the email, it begins analyzing it. Below is an example of a prompt sent to the generative AI model.

[1257] Prompt statement:

[1258] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[1259] The generative AI model receives this prompt and evaluates whether the email is a phishing scam. If phishing characteristics are detected, the server generates a warning message saying "Phishing scam detected. Do not click on the link" and advice "Please visit the official website directly," and sends these to the user's smart glasses. By checking the warning message displayed on the smart glasses and visiting the official website directly without clicking on the link, the user can avoid the risk of phishing scams.

[1260] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1261] Step 1:

[1262] The server periodically checks the user's mailbox to see if new emails have arrived. The input is direct external data: the user's email information, which is retrieved using the email client API. The output is a list of new emails.

[1263] Step 2:

[1264] When the server detects new emails, it retrieves their contents and stores them in the database. The input is a list of new emails, and the email contents are retrieved in text format via the API. The output is the retrieved email contents as text data.

[1265] Step 3:

[1266] The server sends the acquired email content to the generative AI model for analysis. The input is the text data of the email content, and the generative AI model extracts the characteristics of phishing scams. As a concrete example, the following prompt sentence is input:

[1267] "Dear customer, your bank account has been temporarily suspended. Please reactivate it by clicking the link below: fake-bank-link.example.com"

[1268] The output is the analysis result, which is a judgment result as to whether the email is a phishing scam or not.

[1269] Step 4:

[1270] The server generates a warning message if phishing scam characteristics are detected based on the analysis results of the generative AI model. The input is the analysis results, and if fraudulent activity is detected, it generates a warning message saying, "Phishing scam detected. Do not click on the link." The output is a warning message and specific advice on what to do.

[1271] Step 5:

[1272] The server sends the generated warning message and advice to the user's smart glasses. The input is the warning message and action advice, which are sent to the smart glasses via the network. The output is the warning message and advice displayed on the smart glasses.

[1273] Step 6:

[1274] The smart glasses receive warning messages and advice and display them in the user's field of view. The input is the warning messages and advice sent from the server, which are visualized in real time on the glasses' display. The output is information that allows the user to visually confirm the warning.

[1275] Step 7:

[1276] The user checks the warning messages and advice displayed on the smart glasses and follows the instructions. The input is the displayed warning messages and advice, and the user takes safe actions to avoid the risk of phishing scams. The output is the user's safe actions.

[1277] Step 8:

[1278] Users provide feedback on warning messages provided by the system. The input is the user's usage experience and opinions, and the server collects this feedback and stores it in a database. The output is data for improving the system's performance.

[1279] Step 9:

[1280] The server uses the collected feedback to update the generative model. The input is the user feedback data, which is used as training data for the generative AI model. The output is an improved generative AI model.

[1281] The above series of processes realizes a system that detects phishing scams in real time and provides warnings and advice to users.

[1282] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1283] The system for implementing the present invention involves four main entities: a server, a terminal, a user, and an emotion engine. Details and specific examples of the processing performed by each entity will be explained below.

[1284] Server-side processing

[1285] Detecting email receipt

[1286] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[1287] Acquiring and analyzing email content

[1288] When the server detects a new email, it retrieves its contents. The email body and associated metadata are sent to a generative model for analysis. The generative model uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify signs of fraud.

[1289] Acquiring emotion data

[1290] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[1291] Phishing signature detection

[1292] The server analyzes the email content to detect signs of fraudulent activity such as phishing scams, records the results, and combines them with emotional data obtained from the emotion engine to generate a warning message based on the user's current emotional state.

[1293] Generating warning messages and advice

[1294] If a phishing scam is detected, the server generates a warning message and advice on safe behaviors according to the user's emotional state. For example, if the user is feeling stressed, the server will provide a more specific and gentle warning message.

[1295] User Notification

[1296] Generated warning messages and advice are sent to the user's device via email, SMS, or in-app notifications.

[1297] Terminal side processing

[1298] Receiving new mail

[1299] The user's device periodically communicates with the mail server to receive new emails, using a standard protocol (such as IMAP or POP3) through the mail client.

[1300] Communicating with the Server

[1301] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[1302] Receiving a warning message

[1303] The device receives warning messages and advice sent from the server, which incorporates data from the emotion engine and is based on the user's current emotional state.

[1304] Displaying a warning message

[1305] Display received warning messages to users, so that they are immediately aware of the risk of phishing.

[1306] User Response

[1307] Checking warnings and taking safe actions

[1308] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. The emotion engine also provides more detailed and gentler advice if the user is feeling stressed.

[1309] Providing feedback

[1310] Users can provide feedback on the warning messages and advice provided by the system, which is sent to the server and used to improve the generative model and emotion engine.

[1311] Specific examples

[1312] For example, consider the case where a user receives a phishing email pretending to be from a bank. The email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with a gentler advice saying, "We recommend that you visit the official website directly."

[1313] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[1314] The above is a detailed description of the processing steps of each entity in the system of the present invention. This series of processes enables real-time detection of phishing scams and provides appropriate warnings and advice that take into account the user's emotional state.

[1315] The processing flow will be explained below.

[1316] Server-side processing

[1317] Step 1:

[1318] The server periodically checks the user's mailbox to see if new emails have arrived, and uses the email client API to retrieve unread emails.

[1319] Step 2:

[1320] When the server detects a new email, it retrieves its content and prepares to send the email body and its associated metadata to the generative model.

[1321] Step 3:

[1322] The server analyzes the email content using a generative model, which uses natural language processing (NLP) techniques to extract text features from the email body and applies algorithms to identify phishing scam signatures.

[1323] Step 4:

[1324] The server applies an algorithm to detect phishing features based on the analyzed feature information, and if any phishing features are detected, records the results.

[1325] Step 5:

[1326] The server analyzes the received email and simultaneously obtains the user's emotional data from the emotion engine, which identifies emotions from the user's facial expressions, tone of voice, keystrokes, mouse movements, etc., and provides the data.

[1327] Step 6:

[1328] The server generates warning messages and safety behavior advice based on the detection results. It considers the emotional data obtained from the emotion engine and prepares messages that match the user's current emotional state. If the user is feeling stressed, it provides warning messages with more specific and gentler wording.

[1329] Step 7:

[1330] The server will then notify the user of the generated warning messages and advice via email, SMS, or in-app notifications.

[1331] Terminal side processing

[1332] Step 1:

[1333] The device periodically communicates with the mail server to receive new emails, using standard protocols (such as IMAP or POP3) through the mail client.

[1334] Step 2:

[1335] The device sends the contents of the received email to the server and requests analysis. The data is encrypted using a secure communication protocol (e.g., TLS).

[1336] Step 3:

[1337] The device receives warning messages and advice sent from the server, and the received data is based on the user's current emotional state.

[1338] Step 4:

[1339] The device will then display the received warning message to the user, allowing them to immediately become aware of the risk of phishing scams.

[1340] User Response

[1341] Step 1:

[1342] Users should check the warning message displayed on their device and follow the instructions, for example, not clicking on the link in the email but visiting the official website directly. If users feel stressed, the emotion engine will provide more detailed and gentler advice.

[1343] Step 2:

[1344] After following the warnings and advice, users can provide feedback, which is sent to the server and used to improve the generative model and emotion engine.

[1345] Step 3:

[1346] If users receive unclear warnings or advice, they can contact us for more information.

[1347] Specific examples

[1348] For example, consider the case where a user receives a phishing email pretending to be from a bank. This email contains a fake link that resembles the official website. When the user's device receives the email, the content is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice saying, "We recommend that you visit the official website directly."

[1349] Users should acknowledge this warning message and go directly to the official website without clicking on the link, thereby avoiding the risk of phishing scams and ensuring safe online activities.

[1350] Example 2

[1351] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1352] In modern society, phishing scams and other fraudulent activities are rapidly increasing, and users are exposed to sophisticated cyber attacks every day. However, conventional fraud detection systems cannot simply detect fraudulent patterns; they must also take into account the user's emotional state to respond appropriately. As a result, when users receive fraud warnings while feeling stressed or anxious, it becomes difficult for them to respond appropriately. This leaves a gap in effective measures to support users' safe online activities.

[1353] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity; means for acquiring the user's emotional state; means for issuing a warning based on the detection result and the user's emotional state; and means for providing the user with advice on safe behavior. The terminal includes: means for monitoring received digital information in real time and identifying digital information having characteristics of fraudulent activity; means for issuing a warning according to the user's emotional state for digital information identified as having characteristics of fraudulent activity; and means for providing the user with specific behavioral advice. This makes it possible to detect fraudulent activity in real time and provide appropriate warning messages and behavioral advice that take the user's emotional state into consideration.

[1354] A "generative model" is a model trained using a machine learning algorithm, which extracts meaningful features from input data and makes predictions.

[1355] "Fraud signatures" refer to patterns or signatures that identify malicious behavior intended to defraud or steal information.

[1356] "Means for issuing warnings" refers to technology that provides a function for notifying users of a warning message in response to detected fraudulent activity.

[1357] "Means for providing advice on safe behavior" refers to a feature that provides specific guidelines or advice for users to deal with fraudulent behavior.

[1358] "Means for acquiring the user's emotional state" refers to technology that estimates the user's emotional state at that time by analyzing the user's facial expressions, tone of voice, keystrokes, mouse movements, etc.

[1359] "Real-time monitoring" refers to the process of analyzing and processing data the moment it is generated.

[1360] "Digital information" refers to any information that is stored, transmitted, or processed in electronic form.

[1361] "Methods of collecting feedback" refers to the techniques and processes used to collect information such as user opinions, ratings, and usage experiences.

[1362] The system for implementing this invention involves four entities: a server, a terminal, a user, and an emotion engine. This system is designed to protect users from cyber attacks and fraudulent activities.

[1363] The server uses the following major hardware and software components:

[1364] Email client APIs: These include the Gmail API, Outlook API, etc., which allow the server to periodically check the user's mailbox to see if new emails have been received.

[1365] Generative AI model: Uses natural language processing (NLP) techniques to extract text features from email bodies and use approaches to detect fraudulent activity.

[1366] Emotion Engine API: Using technologies such as EmotionAPI, we identify the user's emotional state from their facial expressions, tone of voice, keystrokes, and mouse movements.

[1367] Database: Used to store and manage analysis results and emotion data.

[1368] The terminal works as follows:

[1369] Email client software: Uses standard protocols such as IMAP or POP3 to communicate with the email server and receive new email.

[1370] Communication module: Encrypts data using secure communication protocols such as TLS and communicates with the server in a secure manner.

[1371] UI component: Provides a user interface for displaying warning messages and advice to the user.

[1372] Users can receive warning messages and advice via their devices and take appropriate measures. Users can also provide feedback on the provided warning messages and advice, contributing to improving the system's performance.

[1373] The specific operation is explained below:

[1374] 1. The server periodically checks the user's mailbox using the mail client API to see if there are any new emails. For example, the check_new_mail() function is called every minute to see if there are any new emails.

[1375] 2. When a new email is detected, the email content is retrieved and the generative AI model is used to parse the email body and associated metadata.

[1376] 3. At the same time, the server calls the emotion engine API to obtain the user's emotion data and integrate it with the analysis results.

[1377] 4. The server detects signs of fraudulent activity from the analyzed email content and generates appropriate warning messages and advice by referring to the emotional data.

[1378] 5. The generated warning message and advice are sent from the server to the device using a secure protocol (TLS).

[1379] 6. The device displays the sent warning message to the user, and the user takes safe action based on it.

[1380] 7. Users can provide feedback on warning messages from their devices, which is sent to the server and used to improve system performance.

[1381] As a concrete example, the following prompt sentences can be fed into a generative AI model for analysis:

[1382] "Analyze the text of the email below and identify the characteristics of a phishing scam.

[1383] From: example@bank.com

[1384] Subject: Account verification required

[1385] Main text:

[1386] Dear Customer,

[1387] We have detected unusual activity on your account. Please click the link below to review your account information.

[1388] [Link to fake site]

[1389] thank you."

[1390] Using this prompt, the system can identify the characteristics of phishing scams and provide appropriate warnings and advice to users.

[1391] The above is a detailed description of the preferred embodiment of the invention. The system allows for real-time detection of fraudulent activity and provides warnings and advice based on emotional state.

[1392] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1393] Step 1:

[1394] The server periodically checks the user's mailbox to see if any new mail has been received.

[1395] Specifically, the server calls the check_new_mail() function to retrieve unread emails using a mail client API (e.g., Gmail API or Outlook API). The input is the API request, and the output is the ID of the new email or email metadata.

[1396] Step 2:

[1397] When the server detects new mail, it retrieves and analyzes its contents.

[1398] Specifically, the server retrieves the email body and associated metadata using the fetch_email_content(mail_id) function. The input is the email ID, and the output is the email body and metadata. This is then sent to the generative AI model, where text analysis is performed using the analyze_text_for_phishing(email_content) function. This analysis extracts features from the input email body and detects fraudulent activity.

[1399] Step 3:

[1400] While analyzing the email, the server also obtains the user's emotional data from the emotion engine.

[1401] Specifically, the server calls the emotion engine API (e.g., EmotionAPI) and retrieves emotion data using the fetch_emotion_data(user_id) function. The input is the user ID, and the output is the user's current emotional state data.

[1402] Step 4:

[1403] The server combines the analyzed email content with emotional data to detect phishing signatures.

[1404] Specifically, the server uses the detect_phishing(features, emotion_data) function to integrate the analysis results with the emotion data and obtain a fraud detection result. The input is the feature data and emotion data, and the output is the phishing detection result.

[1405] Step 5:

[1406] If a phishing scam is detected, the server generates a warning message and safety behavior advice.

[1407] Specifically, the server uses the generate_warning_message(phishing_detected, emotion_data) function to generate a warning message and advice based on the emotional state. The input is the phishing detection result and emotion data, and the output is a warning message and advice.

[1408] Step 6:

[1409] The server notifies the user's terminal of the generated warning message and advice.

[1410] Specifically, the server calls the send_notification(user_id, message) function to notify the user via email, SMS, or in-app notification. The input is the user ID and message, and the output is the result of sending the notification.

[1411] Step 7:

[1412] The user's device receives the new email and begins communicating with the server.

[1413] The specific operation involves the terminal receiving new email using a standard protocol such as IMAP or POP3 using the connect_to_mail_server() function and sending the content to the server. The input is the connection information for the mail server, and the output is the content of the new email.

[1414] Step 8:

[1415] The user's terminal receives the warning messages and advice sent from the server.

[1416] Specifically, the terminal receives an alert message from the server using the receive_notification() function. The input is a notification request, and the output is an alert message.

[1417] Step 9:

[1418] The user acknowledges the warning message and takes safe action.

[1419] The specific behavior includes the user reading the warning message displayed on the device, not clicking on the link in the phishing email, and directly visiting the official website. The input is the warning message, and the output is the user's behavior.

[1420] Step 10:

[1421] Users provide feedback on system warning messages and advice.

[1422] Specifically, the user uses the feedback function of the device and sends feedback to the server using the submit_feedback(feedback) function. The input is the user's feedback, and the output is the feedback sending result.

[1423] This is the flow of the system's program processing. This series of processes enables the system to detect phishing scams in real time and provide warning messages and advice on what to do that take into account the user's emotional state.

[1424] (Application example 2)

[1425] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1426] Conventional phishing detection systems issue warnings without considering the user's emotional state, which can cause excessive stress and confusion. Furthermore, the warning messages are uniform and lack advice to encourage users to take appropriate safety actions. Furthermore, they do not integrate with robots used in the home, limiting the means of notifying users. This presents a challenge for users, making it difficult for them to quickly understand the dangers of phishing scams and take appropriate action.

[1427] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information obtained using a generative model and detecting characteristics of fraudulent activity, means for evaluating the user's emotional state using an emotion engine, and means for generating and presenting appropriate warning messages and advice on safe behavior based on the detection results and the user's emotional state. This makes it possible to provide more effective and less stressful warning messages and advice that take the user's emotional state into consideration. In addition, by linking with a robot used in the home, warnings can be conveyed to the user through voice notifications and displays, allowing the user to respond quickly and appropriately.

[1428] A "generative model" is an algorithm that uses AI technology to extract features from input data and perform analysis and predictions.

[1429] The "emotion engine" is a system that evaluates a user's emotional state based on data such as the user's facial expressions, tone of voice, keystrokes, and mouse movements.

[1430] "Fraud signatures" are indicators and patterns that indicate phishing scams or other malicious activity.

[1431] An "appropriate warning message" is a message that informs users of the risk of fraudulent activities such as phishing scams and urges them to take specific action.

[1432] "Safety advice" is information that provides specific actions and measures that users should take when they encounter fraudulent activity.

[1433] "Real-time monitoring" is the process of reviewing and analyzing data as it is generated or received.

[1434] "Feedback" means opinions and information provided by users about the performance and usability of the system.

[1435] "Pattern recognition" is a technique for finding specific patterns or characteristics in data and performing analysis and judgments based on them.

[1436] "Audio Notifications" are a means of audibly conveying system-generated informational or warning messages to the user.

[1437] "Display notification" is a means of displaying system-generated informational or warning messages to the user on a display.

[1438] The system for realizing this application example is based on a robot used in the home, and includes a server, a terminal, a user, and an emotion engine. A specific implementation method of this system will be described below.

[1439] The server analyzes the features of the received data using a generative model. The generative model includes an algorithm that uses natural language processing techniques to extract features from the email body and detects signs of fraudulent activity. It also uses an emotion engine to evaluate the user's emotional state from facial expressions, tone of voice, keystrokes, mouse movements, etc. The emotion engine includes software for identifying the user's emotional state.

[1440] When a device receives an email, it immediately sends the contents of the email to a server and requests analysis. This data is encrypted using a secure communication protocol (e.g., TLS). Once the analysis is complete, the results are returned to the device, which then notifies the user of a warning message based on their emotional state and advice on safe behavior.

[1441] When the home robot receives a notification from the device, it communicates the information to the user through voice output and a display. Voice notifications are made using the robot's internal speaker, and warning messages and advice on safe behavior are displayed on the display.

[1442] For example, if a user receives a phishing email pretending to be from a bank, the content of the email is immediately sent to the server and analyzed by the generative model and emotion engine. If the generative model detects the characteristics of a phishing scam and the emotion engine determines that the user's stress level is high, the server generates a warning message saying, "Phishing scam detected. Do not click on the link. Please be careful," along with gentle advice such as, "We recommend visiting the official website directly." These messages are communicated to the user aloud through the home robot's speaker, and similar messages are displayed on the display.

[1443] By checking the robot warning message and visiting the official website directly instead of clicking on the link, users can avoid the risk of phishing scams and stay safe online.

[1444] Example prompts to input to a generative AI model:

[1445] Detect phishing scams by analyzing the following email body:

[1446] ---

[1447] Subject: Important Notice

[1448] Body: Due to an update to our security system, please click the link below to update your authentication information.

[1449] Link: http: / / example.com / secure

[1450] Using this prompt, the generative AI model detects phishing scam characteristics within the email body and provides the basis for generating appropriate warning messages.

[1451] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1452] Step 1:

[1453] The server periodically checks the user's mailbox to see if new emails have been received. It uses the email client API to retrieve unread emails. The input is unread email data from the email server, and the output is the retrieved unread email information.

[1454] Step 2:

[1455] When the server detects a new email, it retrieves its contents. The retrieved email body and related metadata are sent to the generative model for analysis. The input is unread email information, and the output is the email body data for analysis.

[1456] Step 3:

[1457] The generative model uses natural language processing techniques to analyze the email body and extract features from the text. It then applies an algorithm to identify fraudulent activity features. The input is the email body data, and the output is fraud feature data.

[1458] Step 4:

[1459] While the server is analyzing the email, it also obtains the user's emotional data from the emotion engine. The emotion engine analyzes the user's facial expressions, tone of voice, keystrokes, mouse movements, etc. to identify their emotional state. The input is the user's behavioral data, and the output is emotional state data.

[1460] Step 5:

[1461] The server detects characteristics of fraudulent activities such as phishing scams from the analyzed email content and records the results. This is combined with emotional data obtained from the emotion engine to generate a warning message based on the user's emotional state. The input is fraudulent activity characteristic data and emotional state data, and the output is a warning message that takes emotions into consideration.

[1462] Step 6:

[1463] If a phishing scam is detected, the server generates a warning message and advice on safe behavior according to the user's emotional state. If the user is feeling stressed, it generates a gentle warning message and advises specific actions. The input is emotional state data and fraud feature data, and the output is a warning message and advice.

[1464] Step 7:

[1465] The terminal receives warning messages and advice sent from the server. This data is encrypted using a secure communication protocol (e.g., TLS). The input is the warning message and advice from the server, and the output is the received notification data.

[1466] Step 8:

[1467] The home robot receives warning messages from the device and conveys information to the user through voice output and a display. The input is notification data from the device, and the output is notification and display to the user.

[1468] Step 9:

[1469] The user checks the robot's warning message and follows the instructions to take safe action, thereby avoiding the risk of phishing scams. The input is the warning message from the robot, and the output is the user's actions to avoid risk.

[1470] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1471] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1472] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1473] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1474] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1475] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1476] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1477] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1478] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1479] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1480] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1481] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1482] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1483] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1484] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1485] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1486] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1487] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1488] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1489] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1490] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1491] The following is further disclosed regarding the above embodiment.

[1492] (Claim 1)

[1493] Using a generative model,

[1494] Analyze the information obtained,

[1495] means for detecting characteristics of fraudulent activity;

[1496] a means for issuing an alert based on the detection result;

[1497] a means of providing users with advice on safe behavior;

[1498] A system including:

[1499] (Claim 2)

[1500] Monitor received digital information in real time,

[1501] means for identifying digital information characteristic of fraud;

[1502] a means for issuing a warning about digital information having the characteristics of fraudulent activity that have been determined;

[1503] a means of providing users with specific behavioral advice;

[1504] 10. The system of claim 1, comprising:

[1505] (Claim 3)

[1506] training a generative model using training data obtained from security experts and expert information sources;

[1507] A way to update to address new fraud methods, and

[1508] a means of gathering user feedback and improving the system's performance;

[1509] 10. The system of claim 1, comprising:

[1510] "Example 1"

[1511] (Claim 1)

[1512] A means for analyzing the obtained information using a generative model to detect characteristics of phishing scams;

[1513] a means for issuing an alert based on the detection result;

[1514] a means of providing users with advice on safe behavior;

[1515] A means of regularly updating training data to address new fraudulent activity; and

[1516] A means to collect user feedback and improve the accuracy of the generative model;

[1517] A means of instantly sending warning messages and advice to the user's device;

[1518] A system including:

[1519] (Claim 2)

[1520] A means for monitoring received digital information in real time and identifying digital information characteristic of phishing scams;

[1521] a means for issuing a warning about digital information that has been identified as having characteristics of a phishing scam;

[1522] a means of providing users with specific behavioral advice;

[1523] A means by which the server periodically checks the user's mailbox for new digital information;

[1524] A means of using prompt statements to automatically generate specific warning messages;

[1525] 10. The system of claim 1, comprising:

[1526] (Claim 3)

[1527] training a generative model using training data obtained from security experts and expert information sources;

[1528] A way to update to address new fraud methods, and

[1529] a means of gathering user feedback and improving the system's performance;

[1530] a means for using a secure communication protocol to notify a user of a warning message;

[1531] 10. The system of claim 1, comprising:

[1532] "Application Example 1"

[1533] (Claim 1)

[1534] Using a generative model,

[1535] Analyze the information obtained,

[1536] means for detecting characteristics of fraudulent activity;

[1537] A means for issuing warnings about digital information that has characteristics of fraudulent activity;

[1538] a means of providing users with advice on safe behavior;

[1539] means for displaying the detected warnings and advice on a display of the smart glasses;

[1540] A system including:

[1541] (Claim 2)

[1542] Monitor received digital information in real time,

[1543] means for identifying digital information characteristic of fraud;

[1544] a means for issuing a warning about digital information having the characteristics of fraudulent activity that have been determined;

[1545] a means of providing users with specific behavioral advice;

[1546] 10. The system of claim 1, comprising:

[1547] (Claim 3)

[1548] training a generative model using training data obtained from security experts and expert information sources;

[1549] A way to update to address new fraud methods, and

[1550] a means of gathering user feedback and improving the system's performance;

[1551] 10. The system of claim 1, comprising:

[1552] "Example 2: Combining Emotion Engines"

[1553] (Claim 1)

[1554] Using a generative model,

[1555] Analyze the information obtained,

[1556] means for detecting characteristics of fraudulent activity;

[1557] a means for obtaining an emotional state of a user;

[1558] means for issuing an alert based on the detection results and the user's emotional state;

[1559] a means of providing users with advice on safe behavior;

[1560] A system including:

[1561] (Claim 2)

[1562] Monitor received digital information in real time,

[1563] means for identifying digital information characteristic of fraud;

[1564] A means for issuing a warning according to the emotional state of the user about the digital information having the characteristics of fraudulent activity that has been determined;

[1565] a means of providing users with specific behavioral advice;

[1566] 10. The system of claim 1, comprising:

[1567] (Claim 3)

[1568] training a generative model using training data obtained from security expertise and information sources;

[1569] A way to update to address new fraud methods, and

[1570] a means of gathering user feedback and improving the system's performance;

[1571] 10. The system of claim 1, comprising:

[1572] "Application example 2 when combining emotion engines"

[1573] (Claim 1)

[1574] Using a generative model,

[1575] Analyze the information obtained,

[1576] means for detecting characteristics of fraudulent activity;

[1577] means for assessing the emotional state of the user using an emotion engine;

[1578] means for generating and presenting appropriate warning messages and safety behavior advice based on the detection results and the user's emotional state;

[1579] A system including:

[1580] (Claim 2)

[1581] Monitor received digital information in real time,

[1582] means for identifying digital information characteristic of fraud;

[1583] a means for acquiring emotional data of a user using an emotion engine and warning the user of the identified characteristics of fraudulent activity in a format corresponding to the user's emotional state;

[1584] a means of providing users with specific, emotionally sensitive behavioral advice;

[1585] 10. The system of claim 1, comprising:

[1586] (Claim 3)

[1587] training a generative model using training data obtained from security experts and expert information sources;

[1588] A way to update to address new fraud methods, and

[1589] a means of collecting user feedback and sentiment data to improve system performance and user experience;

[1590] 10. The system of claim 1, comprising: [Explanation of symbols]

[1591] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. Using a generative model, Analyze the information obtained, means for detecting characteristics of fraudulent activity; a means for issuing an alert based on the detection result; a means of providing users with advice on safe behavior; A system including:

2. Monitor received digital information in real time, means for identifying digital information characteristic of fraud; a means for issuing a warning about digital information having the characteristics of fraudulent activity that have been determined; a means of providing users with specific behavioral advice; The system of claim 1 , comprising:

3. training a generative model using training data obtained from security experts and expert information sources; A way to update to address new fraud methods, and a means of gathering user feedback and improving the system's performance; The system of claim 1 , comprising:

Images