system

Abstract

We provide the system. [Solution] A means for acquiring received data and analyzing the metadata and content data of said data, A means for inputting the analyzed data into a machine learning model and evaluating it, A means for classifying data based on the aforementioned evaluation and automatically processing data classified into a specific category, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern digital society, there are still many suspicious emails, and in particular, the problem remains that a large number of users, including the elderly, suffer financial damage. Conventional countermeasures such as spam filters and manual confirmation by users have limitations, and there is a need for a system that can quickly and effectively determine suspicious emails by a more advanced automated method and prevent damage.

Means for Solving the Problems

[0005] This invention analyzes the metadata and content of received email data and uses a machine learning model to evaluate whether or not the email is suspicious. Based on the evaluation results, it includes means for automatically classifying emails and deleting or isolating emails deemed suspicious. It also notifies the terminal of suspicious emails and continuously learns from the evaluation of new data to improve detection accuracy. In this way, it effectively prevents damage caused by suspicious emails without requiring any action from the user.

[0006] "Received data" refers to all information that a server or client retrieves from an external system, including email metadata and body text.

[0007] "Metadata" refers to attribute information associated with emails and digital data, including data-specific information such as the sender, date and time of receipt, and subject.

[0008] A "machine learning model" refers to a collection of algorithms that learn patterns and features based on large amounts of data and then use that knowledge to make predictions and classifications on new data.

[0009] "Evaluating" refers to the act of applying judgment criteria based on analyzed data and determining the degree of suspicion using specific conditions or standards.

[0010] "Classifying" refers to the process of sorting data into different groups or categories based on certain criteria.

[0011] "To process automatically" refers to a series of operations that a program or system performs according to certain conditions without requiring human intervention.

[0012] A "terminal" refers to a device such as a computer or smartphone that is directly operated by the user, and it has the role of receiving notifications from the server.

[0013] "Continuous learning" refers to the process of incorporating new data and feedback to improve the predictive ability of a model over time. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0015] Next, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The suspicious email elimination agent system according to the present invention provides a program that protects users from suspicious emails by having a server receive email data and analyze its metadata and main information. The server evaluates received emails using a machine learning model and quickly identifies and processes suspicious emails based on this evaluation. Specifically, when the server receives new emails in a user's email account, it analyzes their contents. During the analysis, the sender's address, subject line patterns, links and phrases in the email body are identified.

[0036] The server inputs the analyzed data into a machine learning model and scores the email based on its characteristics. If the score is high, the email is judged to be suspicious, and the server automatically deletes or quarantines it. For example, if you receive an email from an unknown sender saying "Your account is not secure," the server moves the email to a quarantine folder and notifies your device of the warning. The user receives the notification on their device and can prevent harm by not clicking on the link in the email, confirming that their account is secure.

[0037] This invention allows the server to continuously update training data and improve the accuracy of the machine learning model based on feedback, enabling it to adapt to newly emerging suspicious emails. As a result, users can use email with peace of mind, and damage from phishing scams and other fraudulent activities can be significantly reduced.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The server retrieves new email data from the user's mail server. It downloads messages from the user's account using an API or the IMAP or POP3 protocol.

[0041] Step 2:

[0042] The server analyzes the metadata and body content of the email it receives. It extracts the sender's name, sender's address, subject, etc. from the email header, and extracts important phrases from the body text.

[0043] Step 3:

[0044] The server inputs the analysis results into a machine learning model, which scores the suspiciousness of the email. The model uses patterns learned from past suspicious email data to evaluate the risk of the current email.

[0045] Step 4:

[0046] The server classifies emails based on their scores. They are categorized as "safe," "suspicious," or "dubious," with emails that score particularly high being flagged as "dubious."

[0047] Step 5:

[0048] The server automatically processes emails classified as "suspicious." Depending on the settings, these emails are either moved to a quarantine folder or permanently deleted.

[0049] Step 6:

[0050] The device receives a notification from the server and alerts the user that their email has been flagged. The notification on the device is displayed as a pop-up or banner.

[0051] Step 7:

[0052] After the server completes processing, it saves the received email data and its evaluation results to a training dataset. This data is then used to improve the accuracy of the machine learning model in the next training process.

[0053] (Example 1)

[0054] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0055] With the advancement of information and communication technology, fraudulent emails and scams are on the rise, making it easier for users to receive suspicious emails. This problem poses serious risks such as the leakage of personal information and unauthorized access, so there is a need for effective means to detect suspicious emails and protect users.

[0056] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0057] In this invention, the server includes means for acquiring received data and analyzing the metadata and content data of the data; means for inputting the analyzed data into a generating AI model for evaluation; and means for generating dynamic prompt sentences for learning new features and updating the generating AI model. This enables rapid and highly accurate identification of suspicious emails and response to constantly evolving phishing techniques.

[0058] "Received data" refers to information and communications sent to the server from external sources, including emails and messages that are subject to analysis and evaluation.

[0059] "Metadata" refers to information other than the content of the data itself, such as the sender's address, subject, and date and time of receipt.

[0060] "Content data" refers to the actual content of an email or message, including text, links, and attachments.

[0061] A "generative AI model" refers to an algorithm-based system that uses machine learning to recognize data patterns and identify suspicious emails.

[0062] A "dynamic prompt" is an instruction generated by the server to update the learning process of a generative AI model, and it refers to data guidelines for incorporating new features into the model's responses.

[0063] "Evaluation" refers to the process of determining suspiciousness through scoring and analysis performed by a generating AI model on received data.

[0064] The suspicious email elimination agent system according to the present invention provides advanced detection technology for securely managing incoming emails in email services used by users. This technology is based on a dedicated program that runs on a server and aims to quickly identify phishing emails and fraudulent emails by combining email analysis and machine learning.

[0065] This system is specifically server-centric. The server first retrieves newly received emails from the user's email account. At this stage, it analyzes the email's metadata, such as the sender's address and subject, as well as content data, including the text and links within the email body. The results of this analysis are then input into a generative AI model built on cloud services such as Azure® and AWS®. The generative AI model used here has learned from a large amount of data and possesses specialized algorithms to extract the characteristics of suspicious emails.

[0066] The analyzed data is evaluated by a generative AI model. Based on the evaluation results, emails are classified into specific categories, such as "trustworthy," "suspicious," or "spam." In this process, the server constantly generates new dynamic prompts to reflect the characteristics of the latest emails and updates the model's learning. For example, when a new phishing technique emerges, a prompt such as "Identify the characteristics of recent phishing campaigns and update the model" might be generated.

[0067] Emails deemed suspicious within the classified emails are automatically isolated by the server from the user's regular mailbox. The server then issues a warning to the user's device, notifying them that the suspicious email has been handled and providing information on safety measures. By reviewing this notification, users can take precautions to avoid opening suspicious emails and protect themselves from potential risks.

[0068] In this way, the system of the present invention provides users with an environment in which they can use email without anxiety and enables a rapid response to increasingly sophisticated fraudulent activities.

[0069] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0070] Step 1:

[0071] The server receives new emails in the user's email account. The input includes metadata (sender address, subject) and content data (body text, links) of the received email. This data is then analyzed. Specifically, the server checks whether the email sender is known, whether the subject contains distinctive phrases, and whether there are links in the body. This analysis allows for an initial assessment of whether the email is suspicious.

[0072] Step 2:

[0073] The server inputs the information obtained from the analysis into a generating AI model. The input data is then sent to a scoring process to evaluate the suspiciousness of the emails. In this process, a model that has learned the characteristics of known suspicious emails analyzes the content and patterns of the emails. As output, a score is generated for each email. Specifically, it quantifies how high the risk is based on the format of the links included in the email and the degree of matching of the metadata.

[0074] Step 3:

[0075] The server classifies emails into specific categories based on scores generated by the AI ​​model. High scores classify emails as "suspicious" or "spam," while low scores classify them as "trustworthy." Specifically, emails with scores above a certain level are automatically moved to a quarantine folder, separating them from other messages. The input for this step is the scoring result, and the output is the email classification result.

[0076] Step 4:

[0077] The server sends a notification to the user's device based on the categorization results. For emails deemed suspicious, a warning message is displayed on the user's screen. The input is the classification result from the previous step, and the output is the warning notification. Specifically, the user is provided with a notification such as, "A new suspicious email has been detected. Please check the details."

[0078] Step 5:

[0079] The server generates dynamic prompts based on the characteristics of newly discovered suspicious emails. To update the generating AI model, the server creates a prompt such as "Learn new features to identify recent phishing emails." This prompt will be used in the next model training. The input for this step is data of new suspicious emails, and the output is the prompt. Specifically, the server prepares guidelines for improving the accuracy of the model.

[0080] (Application Example 1)

[0081] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0082] Protecting users quickly and effectively from suspicious emails is a critical challenge in today's information society. However, existing systems are time-consuming to identify and respond to suspicious emails, potentially leaving users vulnerable to harm. This invention aims to solve these problems by rapidly detecting suspicious emails and providing users with immediate warnings.

[0083] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0084] In this invention, the server includes means for acquiring received information and analyzing metadata and content information of said information; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific category; and means for displaying the information classified into a specific category in real time on a smart device. This enables users to receive prompt warnings of suspicious emails and take immediate and secure action.

[0085] "Received information" refers to all data that a server or device acquires from external sources, and in particular includes email metadata and content information.

[0086] "Metadata" refers to additional information about emails other than their content, such as the sender's address, subject, and date and time of sending.

[0087] "Content information" refers to the actual information content, such as text and links, included in the body of an email.

[0088] "Means of analysis" refers to processes and algorithms for analyzing the metadata and content information of received information and extracting suspicious features.

[0089] A "machine learning model" is a program that learns from past data and identifies the characteristics of suspicious emails, and is used for evaluating emails.

[0090] "Means of evaluation" refers to the process of using machine learning models to determine whether the analyzed information is suspicious or not.

[0091] "Specific categories" refer to groups of emails, such as suspicious emails and regular emails, that are classified through analysis and evaluation of received information.

[0092] "Automated processing methods" refer to processes that perform actions such as deletion or isolation on information classified into specific categories without user intervention.

[0093] A "smart device" is a portable device equipped with information processing and communication functions, and specifically refers to smartphones and tablet devices.

[0094] "Means of displaying information in real time" refers to the process of visually presenting information categorized into a specific category on a device immediately.

[0095] This invention realizes a suspicious email detection and warning system on smart devices. The server acquires incoming information from external sources and analyzes its metadata and content information. Python or email processing libraries can be used to analyze the sender address, subject, and links within the email body. The analyzed information is input into a machine learning model, which uses libraries such as scikit-learn to evaluate whether the email is suspicious.

[0096] The server categorizes information into suspicious emails and normal emails based on its evaluation. Emails deemed suspicious with a high score are automatically moved to a quarantine folder or deleted. This information is notified to the user's smart device in real time. This allows the user to receive an immediate warning and take a safe action. Smartphones are typically equipped with iOS or Android® OS, and compatible apps are generally developed using Swift or Kotlin.

[0097] For example, when a user has a question like, "Is this email suspicious?", the application analyzes the email in the background and notifies the user that "This email may be suspicious." This helps the user when making important decisions.

[0098] Examples of prompts to input into a generative AI model:

[0099] "Please analyze the subject lines and sender information of today's new emails to detect suspicious emails."

[0100] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0101] Step 1:

[0102] The server retrieves received information from the mail server. The input is an email, and the output is the email's metadata (sender address, subject, date and time sent) and content information (body, links). Specifically, the server connects to the mail server using the IMAP protocol and searches for unread emails.

[0103] Step 2:

[0104] The server analyzes the received information. The input here is the metadata and content information obtained in step 1, and the output is analysis data for extracting suspicious features. Specifically, text analysis is used to analyze whether there are specific patterns in the email sender or subject line, or whether there are suspicious links or phrases in the body of the email.

[0105] Step 3:

[0106] The server inputs the analyzed data into a machine learning model for evaluation. The input is the analyzed data from step 2, and the output is a score indicating whether the email is suspicious or not. Specifically, the scikit-learn library is used to perform scoring with a pre-trained model. A higher score indicates that the email is suspicious.

[0107] Step 4:

[0108] The server classifies and processes information based on the evaluation results. The input is the evaluation score from step 3, and the output is the action indicating whether the suspicious email will be moved to a quarantine folder or deleted. Specifically, it implements a function to move emails that exceed a certain score threshold to a quarantine folder.

[0109] Step 5:

[0110] The device displays a warning to the user in real time. The input is the result of the processing in step 4, and the output is the warning message displayed to the user. Specifically, the smartphone app uses push notifications to immediately inform the user that the email is suspicious.

[0111] Step 6:

[0112] The user reviews the warning and decides how to respond to the suspicious email. The input is the warning message from step 5, and the output is selected as an action for the user to handle the email safely. The user receives a notification and takes action, such as not clicking on links in the suspicious email.

[0113] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0114] This invention enhances the user experience by combining a system that effectively detects and processes suspicious emails with an emotion engine that recognizes user emotions in real time. First, the server receives email data and analyzes its metadata and content. The analyzed data is scored by a machine learning model and classified according to its level of suspiciousness. While suspicious emails are quarantined or deleted through this process, this invention further adds a process in which the emotion engine analyzes the user's emotions.

[0115] The server uses an emotion engine to monitor the emotional impact that emails categorized into specific groups have on users. It captures in real time how users react when they receive certain email notifications on their devices, and if a negative reaction is detected, it can adjust the frequency and display method of notifications. For example, if a device notifies a user that "an important email has been identified as suspicious," the emotion engine will instruct the device to soften the notification to avoid causing the user emotional stress.

[0116] Furthermore, the emotion engine collects user feedback as training data, contributing to improved accuracy in emotion recognition. This allows the server to predict user emotions more accurately and optimize the methods and content of email notifications. The present invention not only processes suspicious emails but also aims to reduce the user's mental burden and provide a safer and more comfortable communication environment.

[0117] The following describes the processing flow.

[0118] Step 1:

[0119] The server receives new email data from the user's mail server. It retrieves the email's metadata (sender, date and time, subject, etc.) and content data, and converts it into a format that can be parsed within the program.

[0120] Step 2:

[0121] The server inputs the acquired email data into a machine learning model to evaluate its suspiciousness. The model uses past learning results to assign a score to the email and determine whether it is malicious or not.

[0122] Step 3:

[0123] The server classifies emails as "safe," "suspicious," or "dubious" based on a score. Emails deemed suspicious are moved to a separate folder or deleted.

[0124] Step 4:

[0125] The server receives the results of categorization and activates the sentiment engine. It evaluates in real time how emails categorized as "suspicious," in particular, affect the user's emotions.

[0126] Step 5:

[0127] The terminal receives instructions from the server and sends notifications to the user based on the content and status of the email. The notifications are adjusted to reduce the user's emotional response to the email content (e.g., stress or anxiety).

[0128] Step 6:

[0129] Based on the user's emotional response, the emotion engine sends feedback to the server to optimize future notifications and the user interface. This makes notification methods more effective and comfortable for the user.

[0130] Step 7:

[0131] The server incorporates feedback from the emotion engine into its training data, continuously improving the accuracy of email suspicion detection and availability notifications. This allows the system as a whole to provide a safer and less stressful environment for users.

[0132] (Example 2)

[0133] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0134] Conventional suspicious email detection systems typically focused on isolating or deleting suspicious emails, but they did not consider the emotional impact on users. Therefore, there was a need for measures to mitigate the stress and discomfort users experienced when receiving emails deemed suspicious. Furthermore, there was a need to provide a safe and comfortable communication environment that considered both the detection of suspicious emails and the user's emotional state.

[0135] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0136] In this invention, the server includes means for acquiring received data and analyzing attribute information and content data of the data; means for inputting the analyzed data into a machine learning model and evaluating it; means for classifying the data based on the evaluation and automatically processing the data classified into a specific category; means for using an emotion analysis engine to analyze user reactions; and means for adjusting the frequency and display method of notifications according to the user's emotional state. This makes it possible to reduce the mental burden on users when dealing with suspicious emails and to realize a safe and comfortable communication environment that further improves the user experience.

[0137] "Received data" refers to a collection of information and signals taken into the system from an external source, and is the digital data format used for its analysis.

[0138] "Attribute information" refers to metadata associated with data, including information such as the sender, recipient, and date and time of transmission.

[0139] "Content data" refers to the actual content other than metadata, and in the case of an email, it refers to the content of the body and attachments.

[0140] "Analysis" is a process that aims to break down and examine data in order to detect its meaning and patterns.

[0141] A "machine learning model" is an algorithmic framework used to make predictions and classifications based on input data, learning from large amounts of data and applying the results.

[0142] "Evaluation" is the process of quantifying the importance and characteristics of analyzed data and measuring its performance and value.

[0143] A "category" is a classification criterion that indicates a group of data that share specific characteristics, and is used to streamline data organization and processing.

[0144] "Automated processing" refers to a method of managing and processing data without human intervention, based on pre-defined rules or algorithms.

[0145] An "emotion analysis engine" refers to a technical tool that infers and analyzes a user's emotions from their voice, facial expressions, and other data.

[0146] A "notification" is a means of communication used to inform a user of specific information or circumstances, and can take the form of visual or auditory communication.

[0147] This invention provides a system for detecting suspicious communications, analyzing users' emotional responses, and improving the user experience. This system is implemented through the interaction of a server, a terminal, and a user.

[0148] The server acquires the received communication data and analyzes its attribute information and content. Natural language processing libraries such as NLTK and spaCy are used for this analysis. The analyzed data is then evaluated using machine learning platforms such as TENSORFLOW® and Sci-kit Learn. This allows the data to be classified into specific categories based on its suspicious nature.

[0149] Next, the device receives a notification from the server and activates the sentiment analysis engine. The sentiment analysis engine uses a generative AI model to analyze the user's reactions in real time. Specifically, it utilizes the user's facial expressions and voice data acquired through the device's camera and microphone. Furthermore, for sentiment analysis, generative AI models such as those from OpenAI (registered trademark) are used.

[0150] If a user shows a negative reaction to data categorized into a specific group, the device adjusts the frequency and display method of notifications based on instructions from the sentiment analysis engine. This reduces user stress and ensures a more comfortable user experience.

[0151] For example, if a device sends a notification to a user stating that "certain information has been deemed suspicious," and the user finds this notification stressful, the notification content will be changed to softer language and the notification sound will be reduced to show consideration for the user's feelings.

[0152] A concrete example of a prompt message is: "Consider a system that recognizes user emotions in real time and optimizes the notification method for suspicious communications. Please come up with ideas for devising notification methods that will not cause stress to the user."

[0153] In summary, the present invention aims to provide a safer and more comfortable communication environment by detecting suspicious emails while also considering the user's emotional response.

[0154] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0155] Step 1:

[0156] The server retrieves received data from the mail server. This input includes various data such as email headers, sender, recipient, and message body. The server uses this input data to parse metadata and content using natural language processing libraries (such as NLTK or spaCy). This analysis tokenizes the data, converting each element into a format that is easier to parse. The output is the parsed structured data.

[0157] Step 2:

[0158] The server inputs the analyzed data into a machine learning model (e.g., TensorFlow, Sci-kit Learn) to calculate a suspiciousness score. Based on the input data, it evaluates the data's characteristics using a specific algorithm. This process classifies the data as either a normal email or a suspicious email. The output includes the score and the classification result.

[0159] Step 3:

[0160] The server automatically processes email data evaluated based on specific criteria. Specifically, highly suspicious emails are moved to a quarantine folder or permanently deleted. This process ensures email security and protects the user's communication environment. A list of emails classified as safe is generated as output.

[0161] Step 4:

[0162] The device receives notifications sent from the server and activates the emotion analysis engine. The input is the email notification information from the server. The device captures the user's real-time facial expressions and voice and analyzes this data using a generative AI model. The output is the analysis result indicating the user's emotional state.

[0163] Step 5:

[0164] The device adjusts the display method and frequency of notifications based on the sentiment analysis results. Specifically, if the user indicates a negative emotion, it will soften the tone of the notification message on the screen and lower the notification sound. This adjustment reduces user stress and provides a more comfortable experience. The output is the notification method optimized after the adjustments.

[0165] Step 6:

[0166] Users provide feedback on notifications. The device sends this feedback information to the server. The server uses this as training data for a machine learning model and performs continuous learning. The output is an improved model that reflects the feedback. This process improves the overall accuracy of the system and the user experience.

[0167] (Application Example 2)

[0168] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0169] One problem with email is that suspicious emails can cause psychological distress to users. Receiving suspicious emails can cause users to feel stressed or anxious. Furthermore, traditional email filtering systems may mistakenly classify important emails as suspicious, leading to the risk of missing crucial information. In addition, frequent email notifications can become a daily annoyance, detracting from the user experience. To solve these problems, there is a need for a system that incorporates a function to recognize user emotions in real time and adjust notification methods accordingly.

[0170] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0171] In this invention, the server includes means for acquiring received data and analyzing the indicator information and content information of the data; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific area; and means for analyzing the user's emotional state in real time and adjusting the notification method according to the emotional state. This makes it possible to appropriately manage notifications of suspicious emails while taking into account the user's emotions, thereby reducing psychological burden and providing an environment in which important information is not missed.

[0172] "Received data" refers to the set of information that the system acquires from an external source and processes.

[0173] "Metric information" refers to metadata and attribute data included in received data, which serves as supplementary information for understanding the nature and characteristics of that data.

[0174] "Content information" refers to essential data such as text, numbers, and images contained within the received data.

[0175] A "machine learning model" is a mathematical model that uses algorithms to learn patterns in data and perform predictions and classifications.

[0176] "Real-time analysis" is a process that performs analysis immediately at the time intervals in which data is generated, and obtains results instantly.

[0177] A "specific domain" refers to a data category classified based on evaluations using machine learning.

[0178] "Automatic processing" means that the system manipulates or manages data according to established criteria without requiring human intervention.

[0179] "Emotional state" refers to the user's psychological and emotional condition, and this information influences their actions and responses.

[0180] "Notification methods" refer to the methods and formats used to convey information to users, including sound, visual effects, and text.

[0181] This invention provides a system that recognizes user emotions in real time and appropriately adjusts the notification method for suspicious emails. The system receives, analyzes, evaluates, and classifies data, and measures emotions and adjusts notifications accordingly.

[0182] The server first acquires the received data and then analyzes its metric and content information in detail. This process uses the data stream provided by the email client and employs analysis software to decompose important metadata and content information. The analyzed information is then input into a machine learning model. Here, machine learning frameworks such as TensorFlow are used to evaluate the suspiciousness of the data and classify it into specific areas.

[0183] Next, the system analyzes the user's emotional state in real time based on the classified data. This emotion recognition utilizes tools such as the Google Cloud Natural Language API, and sensors may be installed on the device to capture the user's facial expressions and reactions. If the system determines that the user's emotional state is negative, the server adjusts the notification method and softens the tone of the suspicious email notification. This aims to improve the user experience.

[0184] For example, when a user receives an email that has been flagged as suspicious, the server adjusts its system to reduce the user's psychological burden by sending a notification with gentle wording such as, "Please handle this email with care."

[0185] An example of a prompt might be, "Analyze the user's facial expression data regarding this email and suggest ways to adjust the notification tone if there is a stress response." Based on this prompt, the AI ​​model will customize the optimal notification method based on human emotions.

[0186] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0187] Step 1:

[0188] The server receives data from the email client. It obtains email metadata and content information as input data. Based on this information, the server analyzes the metadata (e.g., sender, subject, sending time, etc.) and extracts the content data. Initial spam filtering is performed through the analysis of the metadata.

[0189] Step 2:

[0190] The server inputs the analyzed metadata and content information into a machine learning model. The model is trained using TensorFlow and determines the suspiciousness of the data. The model outputs a suspiciousness score, and based on that score, the data is classified into different categories (e.g., spam, normal, caution).

[0191] Step 3:

[0192] The server automatically processes data based on its suspiciousness score. Suspicious data classified into specific areas is either isolated or deleted. Information deemed normal is sent to the user's terminal. This process is performed automatically by the system without any user intervention.

[0193] Step 4:

[0194] The user's terminal receives email data from the server and analyzes the user's emotional state in real time. Sensors and cameras are used to monitor the user's reactions (e.g., facial expressions and voice tone), and the Google Cloud Natural Language API is used to analyze emotions. The results are output as an emotional state.

[0195] Step 5:

[0196] The server adjusts notification methods based on the user's emotional state. If negative emotions are detected, the server generates a prompt to soften the notification and determines the appropriate notification method. For example, it might send a gentle notification message such as "Caution is advised, please proceed cautiously" to the user's terminal.

[0197] Step 6:

[0198] The server collects user feedback and uses it to continuously improve the system. The feedback data is used to retrain machine learning models and improve the accuracy of the sentiment engine. This allows the system to continuously improve the user experience.

[0199] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0200] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0201] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0202] [Second Embodiment]

[0203] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0204] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0205] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0206] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0207] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0208] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0209] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0210] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0211] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0212] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0213] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0214] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0215] The suspicious email elimination agent system according to the present invention provides a program that protects users from suspicious emails by having a server receive email data and analyze its metadata and main information. The server evaluates received emails using a machine learning model and quickly identifies and processes suspicious emails based on this evaluation. Specifically, when the server receives new emails in a user's email account, it analyzes their contents. During the analysis, the sender's address, subject line patterns, links and phrases in the email body are identified.

[0216] The server inputs the analyzed data into a machine learning model and scores the email based on its characteristics. If the score is high, the email is judged to be suspicious, and the server automatically deletes or quarantines it. For example, if you receive an email from an unknown sender saying "Your account is not secure," the server moves the email to a quarantine folder and notifies your device of the warning. The user receives the notification on their device and can prevent harm by not clicking on the link in the email, confirming that their account is secure.

[0217] This invention allows the server to continuously update training data and improve the accuracy of the machine learning model based on feedback, enabling it to adapt to newly emerging suspicious emails. As a result, users can use email with peace of mind, and damage from phishing scams and other fraudulent activities can be significantly reduced.

[0218] The following describes the processing flow.

[0219] Step 1:

[0220] The server retrieves new email data from the user's mail server. It downloads messages from the user's account using an API or the IMAP or POP3 protocol.

[0221] Step 2:

[0222] The server analyzes the metadata and body content of the email it receives. It extracts the sender's name, sender's address, subject, etc., from the email header, and extracts important phrases from the text of the email body.

[0223] Step 3:

[0224] The server inputs the analysis results into a machine learning model, which scores the suspiciousness of the email. The model uses patterns learned from past suspicious email data to evaluate the risk of the current email.

[0225] Step 4:

[0226] The server classifies emails based on their scores. They are categorized as "safe," "suspicious," or "dubious," with emails that score particularly high being flagged as "dubious."

[0227] Step 5:

[0228] The server automatically processes emails classified as "suspicious." Depending on the settings, these emails are either moved to a quarantine folder or permanently deleted.

[0229] Step 6:

[0230] The device receives a notification from the server and alerts the user that their email has been flagged. The notification on the device is displayed as a pop-up or banner.

[0231] Step 7:

[0232] After the server completes processing, it saves the received email data and its evaluation results to a training dataset. This data is then used to improve the accuracy of the machine learning model in the next training process.

[0233] (Example 1)

[0234] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0235] With the advancement of information and communication technology, fraudulent emails and scams are on the rise, making it easier for users to receive suspicious emails. This problem poses serious risks such as the leakage of personal information and unauthorized access, so there is a need for effective means to detect suspicious emails and protect users.

[0236] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0237] In this invention, the server includes means for acquiring received data and analyzing the metadata and content data of the data; means for inputting the analyzed data into a generating AI model for evaluation; and means for generating dynamic prompt sentences for learning new features and updating the generating AI model. This enables rapid and highly accurate identification of suspicious emails and response to constantly evolving phishing techniques.

[0238] "Received data" refers to information and communications sent to the server from external sources, including emails and messages that are subject to analysis and evaluation.

[0239] "Metadata" refers to information other than the content of the data itself, such as the sender's address, subject, and date and time of receipt.

[0240] "Content data" refers to the actual content of an email or message, including text, links, and attachments.

[0241] A "generative AI model" refers to an algorithm-based system that uses machine learning to recognize data patterns and identify suspicious emails.

[0242] A "dynamic prompt" is an instruction generated by the server to update the learning process of a generative AI model, and it refers to data guidelines for incorporating new features into the model's responses.

[0243] "Evaluation" refers to the process of determining suspiciousness through scoring and analysis performed by a generating AI model on received data.

[0244] The suspicious email elimination agent system according to the present invention provides advanced detection technology for securely managing incoming emails in email services used by users. This technology is based on a dedicated program that runs on a server and aims to quickly identify phishing emails and fraudulent emails by combining email analysis and machine learning.

[0245] This system is specifically server-centric. The server first retrieves newly received emails from the user's email account. At this stage, it analyzes the email's metadata, such as the sender's address and subject, as well as content data, including the text and links within the email body. The results of this analysis are then input into a generative AI model built on cloud services such as Azure and AWS. The generative AI model used here has learned from a large amount of data and possesses specialized algorithms to extract characteristics of suspicious emails.

[0246] The analyzed data is evaluated by a generative AI model. Based on the evaluation results, emails are classified into specific categories, such as "trustworthy," "suspicious," or "spam." In this process, the server constantly generates new dynamic prompts to reflect the characteristics of the latest emails and updates the model's learning. For example, when a new phishing technique emerges, a prompt such as "Identify the characteristics of recent phishing campaigns and update the model" might be generated.

[0247] Emails deemed suspicious within the classified emails are automatically isolated by the server from the user's regular mailbox. The server then issues a warning to the user's device, notifying them that the suspicious email has been handled and providing information on safety measures. By reviewing this notification, users can take precautions to avoid opening suspicious emails and protect themselves from potential risks.

[0248] In this way, the system of the present invention provides users with an environment in which they can use email without anxiety and enables a rapid response to increasingly sophisticated fraudulent activities.

[0249] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0250] Step 1:

[0251] The server receives new emails in the user's email account. The input includes metadata (sender address, subject) and content data (body text, links) of the received email. This data is then analyzed. Specifically, the server checks whether the email sender is known, whether the subject contains distinctive phrases, and whether there are links in the body. This analysis allows for an initial assessment of whether the email is suspicious.

[0252] Step 2:

[0253] The server inputs the information obtained from the analysis into a generating AI model. The input data is then sent to a scoring process to evaluate the suspiciousness of the emails. In this process, a model that has learned the characteristics of known suspicious emails analyzes the content and patterns of the emails. As output, a score is generated for each email. Specifically, it quantifies how high the risk is based on the format of the links included in the email and the degree of matching of the metadata.

[0254] Step 3:

[0255] The server classifies emails into specific categories based on scores generated by the AI ​​model. High scores classify emails as "suspicious" or "spam," while low scores classify them as "trustworthy." Specifically, emails with scores above a certain level are automatically moved to a quarantine folder, separating them from other messages. The input for this step is the scoring result, and the output is the email classification result.

[0256] Step 4:

[0257] The server sends a notification to the user's device based on the categorization results. For emails deemed suspicious, a warning message is displayed on the user's screen. The input is the classification result from the previous step, and the output is the warning notification. Specifically, the user is provided with a notification such as, "A new suspicious email has been detected. Please check the details."

[0258] Step 5:

[0259] The server generates dynamic prompts based on the characteristics of newly discovered suspicious emails. To update the generating AI model, the server creates a prompt such as "Learn new features to identify recent phishing emails." This prompt will be used in the next model training. The input for this step is data of new suspicious emails, and the output is the prompt. Specifically, the server prepares guidelines for improving the accuracy of the model.

[0260] (Application Example 1)

[0261] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0262] Protecting users quickly and effectively from suspicious emails is a critical challenge in today's information society. However, existing systems are time-consuming to identify and respond to suspicious emails, potentially leaving users vulnerable to harm. This invention aims to solve these problems by rapidly detecting suspicious emails and providing users with immediate warnings.

[0263] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0264] In this invention, the server includes means for acquiring received information and analyzing metadata and content information of said information; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific category; and means for displaying the information classified into a specific category in real time on a smart device. This enables users to receive prompt warnings of suspicious emails and take immediate and secure action.

[0265] "Received information" refers to all data that a server or device acquires from external sources, and in particular includes email metadata and content information.

[0266] "Metadata" refers to additional information about emails other than their content, such as the sender's address, subject, and date and time of sending.

[0267] "Content information" refers to the actual information content, such as text and links, included in the body of an email.

[0268] "Means of analysis" refers to processes and algorithms for analyzing the metadata and content information of received information and extracting suspicious features.

[0269] A "machine learning model" is a program that learns from past data and identifies the characteristics of suspicious emails, and is used for evaluating emails.

[0270] "Means of evaluation" refers to the process of using machine learning models to determine whether the analyzed information is suspicious or not.

[0271] "Specific categories" refer to groups of emails, such as suspicious emails and regular emails, that are classified through analysis and evaluation of received information.

[0272] "Automated processing methods" refer to processes that perform actions such as deletion or isolation on information classified into specific categories without user intervention.

[0273] A "smart device" is a portable device equipped with information processing and communication functions, and specifically refers to smartphones and tablet devices.

[0274] "Means of displaying information in real time" refers to the process of visually presenting information categorized into a specific category on a device immediately.

[0275] This invention realizes a suspicious email detection and warning system on smart devices. The server acquires incoming information from external sources and analyzes its metadata and content information. Python or email processing libraries can be used to analyze the sender address, subject, and links within the email body. The analyzed information is input into a machine learning model, which uses libraries such as scikit-learn to evaluate whether the email is suspicious.

[0276] The server categorizes information into suspicious emails and normal emails based on its evaluation. Emails deemed suspicious with a high score are automatically moved to a quarantine folder or deleted. This information is notified to the user's smart device in real time. This allows the user to receive an immediate warning and take a safe action. Smartphones are equipped with iOS or Android OS, and compatible apps are typically developed using Swift or Kotlin.

[0277] For example, when a user has a question like, "Is this email suspicious?", the application analyzes the email in the background and notifies the user that "This email may be suspicious." This helps the user when making important decisions.

[0278] Examples of prompt sentences to be input into the generative AI model:

[0279] "Analyze the subject line and sender information of today's incoming emails and detect suspicious emails."

[0280] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0281] Step 1:

[0282] The server acquires the received information from the mail server. The input is an email, and as the output, meta-information of the email (sender address, subject line, sending date and time) and content information (body text, link) are obtained. As a specific operation, the server connects to the mail server using the IMAP protocol and searches for unread emails.

[0283] Step 2:

[0284] The server analyzes the received information it has acquired. Here, the input is the meta-information and content information obtained in Step 1, and as the output, analysis data for extracting suspicious features is generated. Specifically, it is analyzed using text analysis processing whether there are specific patterns in the sender or subject line of the email, or whether there are any suspicious links or phrases in the body text.

[0285] Step 3:

[0286] The server inputs the analysis data into the machine learning model for evaluation. The input is the analysis data from Step 2, and the output is a score indicating whether the email is suspicious. As a specific operation, scoring is performed using a pre-trained model with the scikit-learn library. If the score is high, it is determined to be a suspicious email.

[0287] Step 4:

[0288] The server classifies and processes information based on the evaluation results. The input is the evaluation score from step 3, and the output is the action indicating whether the suspicious email will be moved to a quarantine folder or deleted. Specifically, it implements a function to move emails that exceed a certain score threshold to a quarantine folder.

[0289] Step 5:

[0290] The device displays a warning to the user in real time. The input is the result of the processing in step 4, and the output is the warning message displayed to the user. Specifically, the smartphone app uses push notifications to immediately inform the user that the email is suspicious.

[0291] Step 6:

[0292] The user reviews the warning and decides how to respond to the suspicious email. The input is the warning message from step 5, and the output is selected as an action for the user to handle the email safely. The user receives a notification and takes action, such as not clicking on links in the suspicious email.

[0293] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0294] This invention enhances the user experience by combining a system that effectively detects and processes suspicious emails with an emotion engine that recognizes user emotions in real time. First, the server receives email data and analyzes its metadata and content. The analyzed data is scored by a machine learning model and classified according to its level of suspiciousness. While suspicious emails are quarantined or deleted through this process, this invention further adds a process in which the emotion engine analyzes the user's emotions.

[0295] The server uses an emotion engine to monitor the emotional impact that emails categorized into specific groups have on users. It captures in real time how users react when they receive certain email notifications on their devices, and if a negative reaction is detected, it can adjust the frequency and display method of notifications. For example, if a device notifies a user that "an important email has been identified as suspicious," the emotion engine will instruct the device to soften the notification to avoid causing the user emotional stress.

[0296] Furthermore, the emotion engine collects user feedback as training data, contributing to improved accuracy in emotion recognition. This allows the server to predict user emotions more accurately and optimize the methods and content of email notifications. The present invention not only processes suspicious emails but also aims to reduce the user's mental burden and provide a safer and more comfortable communication environment.

[0297] The following describes the processing flow.

[0298] Step 1:

[0299] The server receives new email data from the user's mail server. It retrieves the email's metadata (sender, date and time, subject, etc.) and content data, and converts it into a format that can be parsed within the program.

[0300] Step 2:

[0301] The server inputs the acquired email data into a machine learning model to evaluate its suspiciousness. The model uses past learning results to assign a score to the email and determine whether it is malicious or not.

[0302] Step 3:

[0303] The server classifies emails as "safe," "suspicious," or "dubious" based on a score. Emails deemed suspicious are moved to a separate folder or deleted.

[0304] Step 4:

[0305] The server receives the results classified into categories and activates the emotion engine. It evaluates in real time how emails classified into specific categories, especially those classified as "suspicious", affect the user's emotions.

[0306] Step 5:

[0307] The terminal receives instructions from the server and notifies the user according to the content and status of the email. The notification is adjusted to reduce the user's emotional reaction (e.g., stress or anxiety) to the content of the email.

[0308] Step 6:

[0309] Based on the user's emotional reaction, the emotion engine sends feedback to the server to optimize future notifications and the user interface. This makes the notification method more effective and comfortable for the user.

[0310] Step 7:

[0311] The server incorporates the feedback from the emotion engine into the learning data and continuously improves the accuracy of the suspicious email determination and the free time notification method. This enables the system to provide a safer and stress-free environment for the user as a whole.

[0312] (Example 2)

[0313] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0314] Conventional suspicious email detection systems typically focused on isolating or deleting suspicious emails, but they did not consider the emotional impact on users. Therefore, there was a need for measures to mitigate the stress and discomfort users experienced when receiving emails deemed suspicious. Furthermore, there was a need to provide a safe and comfortable communication environment that considered both the detection of suspicious emails and the user's emotional state.

[0315] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0316] In this invention, the server includes means for acquiring received data and analyzing attribute information and content data of the data; means for inputting the analyzed data into a machine learning model and evaluating it; means for classifying the data based on the evaluation and automatically processing the data classified into a specific category; means for using an emotion analysis engine to analyze user reactions; and means for adjusting the frequency and display method of notifications according to the user's emotional state. This makes it possible to reduce the mental burden on users when dealing with suspicious emails and to realize a safe and comfortable communication environment that further improves the user experience.

[0317] "Received data" refers to a collection of information and signals taken into the system from an external source, and is the digital data format used for its analysis.

[0318] "Attribute information" refers to metadata associated with data, including information such as the sender, recipient, and date and time of transmission.

[0319] "Content data" refers to the actual content other than metadata, and in the case of an email, it refers to the content of the body and attachments.

[0320] "Analysis" is a process that aims to break down and examine data in order to detect its meaning and patterns.

[0321] A "machine learning model" is an algorithmic framework used to make predictions and classifications based on input data, learning from large amounts of data and applying the results.

[0322] "Evaluation" is the process of quantifying the importance and characteristics of analyzed data and measuring its performance and value.

[0323] A "category" is a classification criterion that indicates a group of data that share specific characteristics, and is used to streamline data organization and processing.

[0324] "Automated processing" refers to a method of managing and processing data without human intervention, based on pre-defined rules or algorithms.

[0325] An "emotion analysis engine" refers to a technical tool that infers and analyzes a user's emotions from their voice, facial expressions, and other data.

[0326] A "notification" is a means of communication used to inform a user of specific information or circumstances, and can take the form of visual or auditory communication.

[0327] This invention provides a system for detecting suspicious communications, analyzing users' emotional responses, and improving the user experience. This system is implemented through the interaction of a server, a terminal, and a user.

[0328] The server acquires the received communication data and analyzes its attribute information and content. Natural language processing libraries such as NLTK and spaCy are used for this analysis. The analyzed data is then evaluated using machine learning platforms such as TensorFlow and Sci-kit Learn. This allows the data to be classified into specific categories based on its suspicious nature.

[0329] Next, the device receives a notification from the server and activates the sentiment analysis engine. The sentiment analysis engine uses a generative AI model to analyze the user's reactions in real time. Specifically, it utilizes the user's facial expressions and voice data acquired through the device's camera and microphone. For example, OpenAI's generative AI model is used for sentiment analysis.

[0330] If a user shows a negative reaction to data categorized into a specific group, the device adjusts the frequency and display method of notifications based on instructions from the sentiment analysis engine. This reduces user stress and ensures a more comfortable user experience.

[0331] For example, if a device sends a notification to a user stating that "certain information has been deemed suspicious," and the user finds this notification stressful, the notification content will be changed to softer language and the notification sound will be reduced to show consideration for the user's feelings.

[0332] A concrete example of a prompt message is: "Consider a system that recognizes user emotions in real time and optimizes the notification method for suspicious communications. Please come up with ideas for devising notification methods that will not cause stress to the user."

[0333] In summary, the present invention aims to provide a safer and more comfortable communication environment by detecting suspicious emails while also considering the user's emotional response.

[0334] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0335] Step 1:

[0336] The server retrieves received data from the mail server. This input includes various data such as email headers, sender, recipient, and message body. The server uses this input data to parse metadata and content using natural language processing libraries (such as NLTK or spaCy). This analysis tokenizes the data, converting each element into a format that is easier to parse. The output is the parsed structured data.

[0337] Step 2:

[0338] The server inputs the analyzed data into a machine learning model (e.g., TensorFlow, Sci-kit Learn) to calculate a suspiciousness score. Based on the input data, it evaluates the data's characteristics using a specific algorithm. This process classifies the data as either a normal email or a suspicious email. The output includes the score and the classification result.

[0339] Step 3:

[0340] The server automatically processes email data evaluated based on specific criteria. Specifically, highly suspicious emails are moved to a quarantine folder or permanently deleted. This process ensures email security and protects the user's communication environment. A list of emails classified as safe is generated as output.

[0341] Step 4:

[0342] The device receives notifications sent from the server and activates the emotion analysis engine. The input is the email notification information from the server. The device captures the user's real-time facial expressions and voice and analyzes this data using a generative AI model. The output is the analysis result indicating the user's emotional state.

[0343] Step 5:

[0344] The device adjusts the display method and frequency of notifications based on the sentiment analysis results. Specifically, if the user indicates a negative emotion, it will soften the tone of the notification message on the screen and lower the notification sound. This adjustment reduces user stress and provides a more comfortable experience. The output is the notification method optimized after the adjustments.

[0345] Step 6:

[0346] Users provide feedback on notifications. The device sends this feedback information to the server. The server uses this as training data for a machine learning model and performs continuous learning. The output is an improved model that reflects the feedback. This process improves the overall accuracy of the system and the user experience.

[0347] (Application Example 2)

[0348] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0349] One problem with email is that suspicious emails can cause psychological distress to users. Receiving suspicious emails can cause users to feel stressed or anxious. Furthermore, traditional email filtering systems may mistakenly classify important emails as suspicious, leading to the risk of missing crucial information. In addition, frequent email notifications can become a daily annoyance, detracting from the user experience. To solve these problems, there is a need for a system that incorporates a function to recognize user emotions in real time and adjust notification methods accordingly.

[0350] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0351] In this invention, the server includes means for acquiring received data and analyzing the indicator information and content information of the data; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific area; and means for analyzing the user's emotional state in real time and adjusting the notification method according to the emotional state. This makes it possible to appropriately manage notifications of suspicious emails while taking into account the user's emotions, thereby reducing psychological burden and providing an environment in which important information is not missed.

[0352] "Received data" refers to the set of information that the system acquires from an external source and processes.

[0353] "Metric information" refers to metadata and attribute data included in received data, which serves as supplementary information for understanding the nature and characteristics of that data.

[0354] "Content information" refers to essential data such as text, numbers, and images contained within the received data.

[0355] A "machine learning model" is a mathematical model that uses algorithms to learn patterns in data and perform predictions and classifications.

[0356] "Real-time analysis" is a process that performs analysis immediately at the time intervals in which data is generated, and obtains results instantly.

[0357] A "specific domain" refers to a data category classified based on evaluations using machine learning.

[0358] "Automatic processing" means that the system manipulates or manages data according to established criteria without requiring human intervention.

[0359] "Emotional state" refers to the user's psychological and emotional condition, and this information influences their actions and responses.

[0360] "Notification methods" refer to the methods and formats used to convey information to users, including sound, visual effects, and text.

[0361] This invention provides a system that recognizes user emotions in real time and appropriately adjusts the notification method for suspicious emails. The system receives, analyzes, evaluates, and classifies data, and measures emotions and adjusts notifications accordingly.

[0362] The server first acquires the received data and then analyzes its metric and content information in detail. This process uses the data stream provided by the email client and employs analysis software to decompose important metadata and content information. The analyzed information is then input into a machine learning model. Here, machine learning frameworks such as TensorFlow are used to evaluate the suspiciousness of the data and classify it into specific areas.

[0363] Next, the system analyzes the user's emotional state in real time based on the classified data. This emotion recognition utilizes tools such as the Google Cloud Natural Language API, and sensors may be installed on the device to capture the user's facial expressions and reactions. If the system determines that the user's emotional state is negative, the server adjusts the notification method and softens the tone of the suspicious email notification. This aims to improve the user experience.

[0364] For example, when a user receives an email that has been flagged as suspicious, the server adjusts its system to reduce the user's psychological burden by sending a notification with gentle wording such as, "Please handle this email with care."

[0365] An example of a prompt might be, "Analyze the user's facial expression data regarding this email and suggest ways to adjust the notification tone if there is a stress response." Based on this prompt, the AI ​​model will customize the optimal notification method based on human emotions.

[0366] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0367] Step 1:

[0368] The server receives data from the email client. It obtains email metadata and content information as input data. Based on this information, the server analyzes the metadata (e.g., sender, subject, sending time, etc.) and extracts the content data. Initial spam filtering is performed through the analysis of the metadata.

[0369] Step 2:

[0370] The server inputs the analyzed metadata and content information into a machine learning model. The model is trained using TensorFlow and determines the suspiciousness of the data. The model outputs a suspiciousness score, and based on that score, the data is classified into different categories (e.g., spam, normal, caution).

[0371] Step 3:

[0372] The server automatically processes data based on its suspiciousness score. Suspicious data classified into specific areas is either isolated or deleted. Information deemed normal is sent to the user's terminal. This process is performed automatically by the system without any user intervention.

[0373] Step 4:

[0374] The user's terminal receives email data from the server and analyzes the user's emotional state in real time. Sensors and cameras are used to monitor the user's reactions (e.g., facial expressions and voice tone), and the Google Cloud Natural Language API is used to analyze emotions. The results are output as an emotional state.

[0375] Step 5:

[0376] The server adjusts notification methods based on the user's emotional state. If negative emotions are detected, the server generates a prompt to soften the notification and determines the appropriate notification method. For example, it might send a gentle notification message such as "Caution is advised, please proceed cautiously" to the user's terminal.

[0377] Step 6:

[0378] The server collects user feedback and uses it to continuously improve the system. The feedback data is used to retrain machine learning models and improve the accuracy of the sentiment engine. This allows the system to continuously improve the user experience.

[0379] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0380] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0381] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0382] [Third Embodiment]

[0383] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0384] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0385] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0386] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0387] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0388] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0389] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0390] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0391] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0392] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0393] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0394] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0395] The suspicious email elimination agent system according to the present invention provides a program that protects users from suspicious emails by having a server receive email data and analyze its metadata and main information. The server evaluates received emails using a machine learning model and quickly identifies and processes suspicious emails based on this evaluation. Specifically, when the server receives new emails in a user's email account, it analyzes their contents. During the analysis, the sender's address, subject line patterns, links and phrases in the email body are identified.

[0396] The server inputs the analyzed data into a machine learning model and scores the email based on its characteristics. If the score is high, the email is judged to be suspicious, and the server automatically deletes or quarantines it. For example, if you receive an email from an unknown sender saying "Your account is not secure," the server moves the email to a quarantine folder and notifies your device of the warning. The user receives the notification on their device and can prevent harm by not clicking on the link in the email, confirming that their account is secure.

[0397] This invention allows the server to continuously update training data and improve the accuracy of the machine learning model based on feedback, enabling it to adapt to newly emerging suspicious emails. As a result, users can use email with peace of mind, and damage from phishing scams and other fraudulent activities can be significantly reduced.

[0398] The following describes the processing flow.

[0399] Step 1:

[0400] The server retrieves new email data from the user's mail server. It downloads messages from the user's account using an API or the IMAP or POP3 protocol.

[0401] Step 2:

[0402] The server analyzes the metadata and body content of the email it receives. It extracts the sender's name, sender's address, subject, etc., from the email header, and extracts important phrases from the text of the email body.

[0403] Step 3:

[0404] The server inputs the analysis results into a machine learning model, which scores the suspiciousness of the email. The model uses patterns learned from past suspicious email data to evaluate the risk of the current email.

[0405] Step 4:

[0406] The server classifies emails based on their scores. They are categorized as "safe," "suspicious," or "dubious," with emails that score particularly high being flagged as "dubious."

[0407] Step 5:

[0408] The server automatically processes emails classified as "suspicious." Depending on the settings, these emails are either moved to a quarantine folder or permanently deleted.

[0409] Step 6:

[0410] The device receives a notification from the server and alerts the user that their email has been flagged. The notification on the device is displayed as a pop-up or banner.

[0411] Step 7:

[0412] After the server completes processing, it saves the received email data and its evaluation results to a training dataset. This data is then used to improve the accuracy of the machine learning model in the next training process.

[0413] (Example 1)

[0414] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0415] With the advancement of information and communication technology, fraudulent emails and scams are on the rise, making it easier for users to receive suspicious emails. This problem poses serious risks such as the leakage of personal information and unauthorized access, so there is a need for effective means to detect suspicious emails and protect users.

[0416] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0417] In this invention, the server includes means for acquiring received data and analyzing the metadata and content data of the data; means for inputting the analyzed data into a generating AI model for evaluation; and means for generating dynamic prompt sentences for learning new features and updating the generating AI model. This enables rapid and highly accurate identification of suspicious emails and response to constantly evolving phishing techniques.

[0418] "Received data" refers to information and communications sent to the server from external sources, including emails and messages that are subject to analysis and evaluation.

[0419] "Metadata" refers to information other than the content of the data itself, such as the sender's address, subject, and date and time of receipt.

[0420] "Content data" refers to the actual content of an email or message, including text, links, and attachments.

[0421] A "generative AI model" refers to an algorithm-based system that uses machine learning to recognize data patterns and identify suspicious emails.

[0422] A "dynamic prompt" is an instruction generated by the server to update the learning process of a generative AI model, and it refers to data guidelines for incorporating new features into the model's responses.

[0423] "Evaluation" refers to the process of determining suspiciousness through scoring and analysis performed by a generating AI model on received data.

[0424] The suspicious email elimination agent system according to the present invention provides advanced detection technology for securely managing incoming emails in email services used by users. This technology is based on a dedicated program that runs on a server and aims to quickly identify phishing emails and fraudulent emails by combining email analysis and machine learning.

[0425] This system is specifically server-centric. The server first retrieves newly received emails from the user's email account. At this stage, it analyzes the email's metadata, such as the sender's address and subject, as well as content data, including the text and links within the email body. The results of this analysis are then input into a generative AI model built on cloud services such as Azure and AWS. The generative AI model used here has learned from a large amount of data and possesses specialized algorithms to extract characteristics of suspicious emails.

[0426] The analyzed data is evaluated by a generative AI model. Based on the evaluation results, emails are classified into specific categories, such as "trustworthy," "suspicious," or "spam." In this process, the server constantly generates new dynamic prompts to reflect the characteristics of the latest emails and updates the model's learning. For example, when a new phishing technique emerges, a prompt such as "Identify the characteristics of recent phishing campaigns and update the model" might be generated.

[0427] Emails deemed suspicious within the classified emails are automatically isolated by the server from the user's regular mailbox. The server then issues a warning to the user's device, notifying them that the suspicious email has been handled and providing information on safety measures. By reviewing this notification, users can take precautions to avoid opening suspicious emails and protect themselves from potential risks.

[0428] In this way, the system of the present invention provides users with an environment in which they can use email without anxiety and enables a rapid response to increasingly sophisticated fraudulent activities.

[0429] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0430] Step 1:

[0431] The server receives new emails in the user's email account. The input includes metadata (sender address, subject) and content data (body text, links) of the received email. This data is then analyzed. Specifically, the server checks whether the email sender is known, whether the subject contains distinctive phrases, and whether there are links in the body. This analysis allows for an initial assessment of whether the email is suspicious.

[0432] Step 2:

[0433] The server inputs the information obtained from the analysis into a generating AI model. The input data is then sent to a scoring process to evaluate the suspiciousness of the emails. In this process, a model that has learned the characteristics of known suspicious emails analyzes the content and patterns of the emails. As output, a score is generated for each email. Specifically, it quantifies how high the risk is based on the format of the links included in the email and the degree of matching of the metadata.

[0434] Step 3:

[0435] The server classifies emails into specific categories based on scores generated by the AI ​​model. High scores classify emails as "suspicious" or "spam," while low scores classify them as "trustworthy." Specifically, emails with scores above a certain level are automatically moved to a quarantine folder, separating them from other messages. The input for this step is the scoring result, and the output is the email classification result.

[0436] Step 4:

[0437] The server sends a notification to the user's device based on the categorization results. For emails deemed suspicious, a warning message is displayed on the user's screen. The input is the classification result from the previous step, and the output is the warning notification. Specifically, the user is provided with a notification such as, "A new suspicious email has been detected. Please check the details."

[0438] Step 5:

[0439] The server generates dynamic prompts based on the characteristics of newly discovered suspicious emails. To update the generating AI model, the server creates a prompt such as "Learn new features to identify recent phishing emails." This prompt will be used in the next model training. The input for this step is data of new suspicious emails, and the output is the prompt. Specifically, the server prepares guidelines for improving the accuracy of the model.

[0440] (Application Example 1)

[0441] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0442] Protecting users quickly and effectively from suspicious emails is a critical challenge in today's information society. However, existing systems are time-consuming to identify and respond to suspicious emails, potentially leaving users vulnerable to harm. This invention aims to solve these problems by rapidly detecting suspicious emails and providing users with immediate warnings.

[0443] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0444] In this invention, the server includes means for acquiring received information and analyzing metadata and content information of said information; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific category; and means for displaying the information classified into a specific category in real time on a smart device. This enables users to receive prompt warnings of suspicious emails and take immediate and secure action.

[0445] "Received information" refers to all data that a server or device acquires from external sources, and in particular includes email metadata and content information.

[0446] "Metadata" refers to additional information about emails other than their content, such as the sender's address, subject, and date and time of sending.

[0447] "Content information" refers to the actual information content, such as text and links, included in the body of an email.

[0448] "Means of analysis" refers to processes and algorithms for analyzing the metadata and content information of received information and extracting suspicious features.

[0449] A "machine learning model" is a program that learns from past data and identifies the characteristics of suspicious emails, and is used for evaluating emails.

[0450] "Means of evaluation" refers to the process of using machine learning models to determine whether the analyzed information is suspicious or not.

[0451] "Specific categories" refer to groups of emails, such as suspicious emails and regular emails, that are classified through analysis and evaluation of received information.

[0452] "Automated processing methods" refer to processes that perform actions such as deletion or isolation on information classified into specific categories without user intervention.

[0453] A "smart device" is a portable device equipped with information processing and communication functions, and specifically refers to smartphones and tablet devices.

[0454] "Means of displaying information in real time" refers to the process of visually presenting information categorized into a specific category on a device immediately.

[0455] This invention realizes a suspicious email detection and warning system on smart devices. The server acquires incoming information from external sources and analyzes its metadata and content information. Python or email processing libraries can be used to analyze the sender address, subject, and links within the email body. The analyzed information is input into a machine learning model, which uses libraries such as scikit-learn to evaluate whether the email is suspicious.

[0456] The server categorizes information into suspicious emails and normal emails based on its evaluation. Emails deemed suspicious with a high score are automatically moved to a quarantine folder or deleted. This information is notified to the user's smart device in real time. This allows the user to receive an immediate warning and take a safe action. Smartphones are equipped with iOS or Android OS, and compatible apps are typically developed using Swift or Kotlin.

[0457] For example, when a user has a question like, "Is this email suspicious?", the application analyzes the email in the background and notifies the user that "This email may be suspicious." This helps the user when making important decisions.

[0458] Examples of prompts to input into a generative AI model:

[0459] "Please analyze the subject lines and sender information of today's new emails to detect suspicious emails."

[0460] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0461] Step 1:

[0462] The server retrieves received information from the mail server. The input is an email, and the output is the email's metadata (sender address, subject, date and time sent) and content information (body, links). Specifically, the server connects to the mail server using the IMAP protocol and searches for unread emails.

[0463] Step 2:

[0464] The server analyzes the received information. The input here is the metadata and content information obtained in step 1, and the output is analysis data for extracting suspicious features. Specifically, text analysis is used to analyze whether there are specific patterns in the email sender or subject line, or whether there are suspicious links or phrases in the body of the email.

[0465] Step 3:

[0466] The server inputs the analyzed data into a machine learning model for evaluation. The input is the analyzed data from step 2, and the output is a score indicating whether the email is suspicious or not. Specifically, the scikit-learn library is used to perform scoring with a pre-trained model. A higher score indicates that the email is suspicious.

[0467] Step 4:

[0468] The server classifies and processes information based on the evaluation results. The input is the evaluation score from step 3, and the output is the action indicating whether the suspicious email will be moved to a quarantine folder or deleted. Specifically, it implements a function to move emails that exceed a certain score threshold to a quarantine folder.

[0469] Step 5:

[0470] The device displays a warning to the user in real time. The input is the result of the processing in step 4, and the output is the warning message displayed to the user. Specifically, the smartphone app uses push notifications to immediately inform the user that the email is suspicious.

[0471] Step 6:

[0472] The user reviews the warning and decides what to do about the suspicious email. The input is the warning message from step 5, and the output is selected as an action for the user to handle the email safely. The user receives a notification and takes action, such as not clicking on links in the suspicious email.

[0473] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0474] This invention enhances the user experience by combining a system that effectively detects and processes suspicious emails with an emotion engine that recognizes user emotions in real time. First, the server receives email data and analyzes its metadata and content. The analyzed data is scored by a machine learning model and classified according to its level of suspiciousness. While suspicious emails are quarantined or deleted through this process, this invention further adds a process in which the emotion engine analyzes the user's emotions.

[0475] The server uses an emotion engine to monitor the emotional impact that emails categorized into specific groups have on users. It captures in real time how users react when they receive certain email notifications on their devices, and if a negative reaction is detected, it can adjust the frequency and display method of notifications. For example, if a device notifies a user that "an important email has been identified as suspicious," the emotion engine will instruct the device to soften the notification to avoid causing the user emotional stress.

[0476] Furthermore, the emotion engine collects user feedback as training data, contributing to improved accuracy in emotion recognition. This allows the server to predict user emotions more accurately and optimize the methods and content of email notifications. The present invention not only processes suspicious emails but also aims to reduce the user's mental burden and provide a safer and more comfortable communication environment.

[0477] The following describes the processing flow.

[0478] Step 1:

[0479] The server receives new email data from the user's mail server. It retrieves the email's metadata (sender, date and time, subject, etc.) and content data, and converts it into a format that can be parsed within the program.

[0480] Step 2:

[0481] The server inputs the acquired email data into a machine learning model to evaluate its suspiciousness. The model uses past learning results to assign a score to the email and determine whether it is malicious or not.

[0482] Step 3:

[0483] The server classifies emails as "safe," "suspicious," or "dubious" based on a score. Emails deemed suspicious are moved to a separate folder or deleted.

[0484] Step 4:

[0485] The server receives the results of categorization and activates the sentiment engine. It evaluates in real time how emails categorized as "suspicious," in particular, affect the user's emotions.

[0486] Step 5:

[0487] The terminal receives instructions from the server and sends notifications to the user based on the content and status of the email. The notifications are adjusted to reduce the user's emotional response to the email content (e.g., stress or anxiety).

[0488] Step 6:

[0489] Based on the user's emotional response, the emotion engine sends feedback to the server to optimize future notifications and the user interface. This makes notification methods more effective and comfortable for the user.

[0490] Step 7:

[0491] The server incorporates feedback from the emotion engine into its training data, continuously improving the accuracy of email suspicion detection and availability notifications. This allows the system as a whole to provide a safer and less stressful environment for users.

[0492] (Example 2)

[0493] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0494] Conventional suspicious email detection systems typically focused on isolating or deleting suspicious emails, but they did not consider the emotional impact on users. Therefore, there was a need for measures to mitigate the stress and discomfort users experienced when receiving emails deemed suspicious. Furthermore, there was a need to provide a safe and comfortable communication environment that considered both the detection of suspicious emails and the user's emotional state.

[0495] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0496] In this invention, the server includes means for acquiring received data and analyzing attribute information and content data of the data; means for inputting the analyzed data into a machine learning model and evaluating it; means for classifying the data based on the evaluation and automatically processing the data classified into a specific category; means for using an emotion analysis engine to analyze user reactions; and means for adjusting the frequency and display method of notifications according to the user's emotional state. This makes it possible to reduce the mental burden on users when dealing with suspicious emails and to realize a safe and comfortable communication environment that further improves the user experience.

[0497] "Received data" refers to a collection of information and signals taken into the system from an external source, and is the digital data format used for its analysis.

[0498] "Attribute information" refers to metadata associated with data, including information such as the sender, recipient, and date and time of transmission.

[0499] "Content data" refers to the actual content other than metadata, and in the case of an email, it refers to the content of the body and attachments.

[0500] "Analysis" is a process that aims to break down and examine data in order to detect its meaning and patterns.

[0501] A "machine learning model" is an algorithmic framework used to make predictions and classifications based on input data, learning from large amounts of data and applying the results.

[0502] "Evaluation" is the process of quantifying the importance and characteristics of analyzed data and measuring its performance and value.

[0503] A "category" is a classification criterion that indicates a group of data that share specific characteristics, and is used to streamline data organization and processing.

[0504] "Automated processing" refers to a method of managing and processing data without human intervention, based on pre-defined rules or algorithms.

[0505] An "emotion analysis engine" refers to a technical tool that infers and analyzes a user's emotions from their voice, facial expressions, and other data.

[0506] A "notification" is a means of communication used to inform a user of specific information or circumstances, and can take the form of visual or auditory communication.

[0507] This invention provides a system for detecting suspicious communications, analyzing users' emotional responses, and improving the user experience. This system is implemented through the interaction of a server, a terminal, and a user.

[0508] The server acquires the received communication data and analyzes its attribute information and content. Natural language processing libraries such as NLTK and spaCy are used for this analysis. The analyzed data is then evaluated using machine learning platforms such as TensorFlow and Sci-kit Learn. This allows the data to be classified into specific categories based on its suspicious nature.

[0509] Next, the device receives a notification from the server and activates the sentiment analysis engine. The sentiment analysis engine uses a generative AI model to analyze the user's reactions in real time. Specifically, it utilizes the user's facial expressions and voice data acquired through the device's camera and microphone. For example, OpenAI's generative AI model is used for sentiment analysis.

[0510] If a user shows a negative reaction to data categorized into a specific group, the device adjusts the frequency and display method of notifications based on instructions from the sentiment analysis engine. This reduces user stress and ensures a more comfortable user experience.

[0511] For example, if a device sends a notification to a user stating that "certain information has been deemed suspicious," and the user finds this notification stressful, the notification content will be changed to softer language and the notification sound will be reduced to show consideration for the user's feelings.

[0512] A concrete example of a prompt message is: "Consider a system that recognizes user emotions in real time and optimizes the notification method for suspicious communications. Please come up with ideas for devising notification methods that will not cause stress to the user."

[0513] In summary, the present invention aims to provide a safer and more comfortable communication environment by detecting suspicious emails while also considering the user's emotional response.

[0514] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0515] Step 1:

[0516] The server retrieves received data from the mail server. This input includes various data such as email headers, sender, recipient, and message body. The server uses this input data to parse metadata and content using natural language processing libraries (such as NLTK or spaCy). This analysis tokenizes the data, converting each element into a format that is easier to parse. The output is the parsed structured data.

[0517] Step 2:

[0518] The server inputs the analyzed data into a machine learning model (e.g., TensorFlow, Sci-kit Learn) to calculate a suspiciousness score. Based on the input data, it evaluates the data's characteristics using a specific algorithm. This process classifies the data as either a normal email or a suspicious email. The output includes the score and the classification result.

[0519] Step 3:

[0520] The server automatically processes email data evaluated based on specific criteria. Specifically, highly suspicious emails are moved to a quarantine folder or permanently deleted. This process ensures email security and protects the user's communication environment. A list of emails classified as safe is generated as output.

[0521] Step 4:

[0522] The device receives notifications sent from the server and activates the emotion analysis engine. The input is the email notification information from the server. The device captures the user's real-time facial expressions and voice and analyzes this data using a generative AI model. The output is the analysis result indicating the user's emotional state.

[0523] Step 5:

[0524] The device adjusts the display method and frequency of notifications based on the sentiment analysis results. Specifically, if the user indicates a negative emotion, it will soften the tone of the notification message on the screen and lower the notification sound. This adjustment reduces user stress and provides a more comfortable experience. The output is the notification method optimized after the adjustments.

[0525] Step 6:

[0526] Users provide feedback on notifications. The device sends this feedback information to the server. The server uses this as training data for a machine learning model and performs continuous learning. The output is an improved model that reflects the feedback. This process improves the overall accuracy of the system and the user experience.

[0527] (Application Example 2)

[0528] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0529] One problem with email is that suspicious emails can cause psychological distress to users. Receiving suspicious emails can cause users to feel stressed or anxious. Furthermore, traditional email filtering systems may mistakenly classify important emails as suspicious, leading to the risk of missing crucial information. In addition, frequent email notifications can become a daily annoyance, detracting from the user experience. To solve these problems, there is a need for a system that incorporates a function to recognize user emotions in real time and adjust notification methods accordingly.

[0530] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0531] In this invention, the server includes means for acquiring received data and analyzing the indicator information and content information of the data; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific area; and means for analyzing the user's emotional state in real time and adjusting the notification method according to the emotional state. This makes it possible to appropriately manage notifications of suspicious emails while taking into account the user's emotions, thereby reducing psychological burden and providing an environment in which important information is not missed.

[0532] "Received data" refers to the set of information that the system acquires from an external source and processes.

[0533] "Metric information" refers to metadata and attribute data included in received data, which serves as supplementary information for understanding the nature and characteristics of that data.

[0534] "Content information" refers to essential data such as text, numbers, and images contained within the received data.

[0535] A "machine learning model" is a mathematical model that uses algorithms to learn patterns in data and perform predictions and classifications.

[0536] "Real-time analysis" is a process that performs analysis immediately at the time intervals in which data is generated, and obtains results instantly.

[0537] A "specific domain" refers to a data category classified based on evaluation using machine learning.

[0538] "Automatic processing" means that the system manipulates or manages data according to established criteria without requiring human intervention.

[0539] "Emotional state" refers to the user's psychological and emotional condition, and this information influences their actions and responses.

[0540] "Notification methods" refer to the methods and formats used to convey information to users, including sound, visual effects, and text.

[0541] This invention provides a system that recognizes user emotions in real time and appropriately adjusts the notification method for suspicious emails. The system receives, analyzes, evaluates, and classifies data, and measures emotions and adjusts notifications accordingly.

[0542] The server first acquires the received data and then analyzes its metric and content information in detail. This process uses the data stream provided by the email client and employs analysis software to decompose important metadata and content information. The analyzed information is then input into a machine learning model. Here, machine learning frameworks such as TensorFlow are used to evaluate the suspiciousness of the data and classify it into specific areas.

[0543] Next, the system analyzes the user's emotional state in real time based on the classified data. This emotion recognition utilizes tools such as the Google Cloud Natural Language API, and sensors may be installed on the device to capture the user's facial expressions and reactions. If the system determines that the user's emotional state is negative, the server adjusts the notification method and softens the tone of the suspicious email notification. This aims to improve the user experience.

[0544] For example, when a user receives an email that has been flagged as suspicious, the server adjusts its system to reduce the user's psychological burden by sending a notification with gentle wording such as, "Please handle this email with care."

[0545] An example of a prompt might be, "Analyze the user's facial expression data regarding this email and suggest ways to adjust the notification tone if there is a stress response." Based on this prompt, the AI ​​model will customize the optimal notification method based on human emotions.

[0546] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0547] Step 1:

[0548] The server receives data from the email client. It obtains email metadata and content information as input data. Based on this information, the server analyzes the metadata (e.g., sender, subject, sending time, etc.) and extracts the content data. Initial spam filtering is performed through the analysis of the metadata.

[0549] Step 2:

[0550] The server inputs the analyzed metadata and content information into a machine learning model. The model is trained using TensorFlow and determines the suspiciousness of the data. The model outputs a suspiciousness score, and based on that score, the data is classified into different categories (e.g., spam, normal, caution).

[0551] Step 3:

[0552] The server automatically processes data based on its suspiciousness score. Suspicious data classified into specific areas is either isolated or deleted. Information deemed normal is sent to the user's terminal. This process is performed automatically by the system without any user intervention.

[0553] Step 4:

[0554] The user's terminal receives email data from the server and analyzes the user's emotional state in real time. Sensors and cameras are used to monitor the user's reactions (e.g., facial expressions and voice tone), and the Google Cloud Natural Language API is used to analyze emotions. The results are output as an emotional state.

[0555] Step 5:

[0556] The server adjusts notification methods based on the user's emotional state. If negative emotions are detected, the server generates a prompt to soften the notification and determines the appropriate notification method. For example, it might send a gentle notification message such as "Caution is advised, please proceed cautiously" to the user's terminal.

[0557] Step 6:

[0558] The server collects user feedback and uses it to continuously improve the system. The feedback data is used to retrain machine learning models and improve the accuracy of the sentiment engine. This allows the system to continuously improve the user experience.

[0559] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0560] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0561] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0562] [Fourth Embodiment]

[0563] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0564] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0565] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0566] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0567] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0568] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0569] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0570] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0571] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0572] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0573] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0574] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0575] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0576] The suspicious email elimination agent system according to the present invention provides a program that protects users from suspicious emails by having a server receive email data and analyze its metadata and main information. The server evaluates received emails using a machine learning model and quickly identifies and processes suspicious emails based on this evaluation. Specifically, when the server receives new emails in a user's email account, it analyzes their contents. During the analysis, the sender's address, subject line patterns, links and phrases in the email body are identified.

[0577] The server inputs the analyzed data into a machine learning model and scores the email based on its characteristics. If the score is high, the email is judged to be suspicious, and the server automatically deletes or quarantines it. For example, if you receive an email from an unknown sender saying "Your account is not secure," the server moves the email to a quarantine folder and notifies your device of the warning. The user receives the notification on their device and can prevent harm by not clicking on the link in the email, confirming that their account is secure.

[0578] This invention allows the server to continuously update training data and improve the accuracy of the machine learning model based on feedback, enabling it to adapt to newly emerging suspicious emails. As a result, users can use email with peace of mind, and damage from phishing scams and other fraudulent activities can be significantly reduced.

[0579] The following describes the processing flow.

[0580] Step 1:

[0581] The server retrieves new email data from the user's mail server. It downloads messages from the user's account using an API or the IMAP or POP3 protocol.

[0582] Step 2:

[0583] The server analyzes the metadata and body content of the email it receives. It extracts the sender's name, sender's address, subject, etc., from the email header, and extracts important phrases from the text of the email body.

[0584] Step 3:

[0585] The server inputs the analysis results into a machine learning model, which scores the suspiciousness of the email. The model uses patterns learned from past suspicious email data to evaluate the risk of the current email.

[0586] Step 4:

[0587] The server classifies emails based on their scores. They are categorized as "safe," "suspicious," or "dubious," with emails that score particularly high being flagged as "dubious."

[0588] Step 5:

[0589] The server automatically processes emails classified as "suspicious." Depending on the settings, these emails are either moved to a quarantine folder or permanently deleted.

[0590] Step 6:

[0591] The device receives a notification from the server and alerts the user that their email has been flagged. The notification on the device is displayed as a pop-up or banner.

[0592] Step 7:

[0593] After the server completes processing, it saves the received email data and its evaluation results to a training dataset. This data is then used to improve the accuracy of the machine learning model in the next training process.

[0594] (Example 1)

[0595] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0596] With the advancement of information and communication technology, fraudulent emails and scams are on the rise, making it easier for users to receive suspicious emails. This problem poses serious risks such as the leakage of personal information and unauthorized access, so there is a need for effective means to detect suspicious emails and protect users.

[0597] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0598] In this invention, the server includes means for acquiring received data and analyzing the metadata and content data of the data; means for inputting the analyzed data into a generating AI model for evaluation; and means for generating dynamic prompt sentences for learning new features and updating the generating AI model. This enables rapid and highly accurate identification of suspicious emails and response to constantly evolving phishing techniques.

[0599] "Received data" refers to information and communications sent to the server from external sources, including emails and messages that are subject to analysis and evaluation.

[0600] "Metadata" refers to information other than the content of the data itself, such as the sender's address, subject, and date and time of receipt.

[0601] "Content data" refers to the actual content of an email or message, including text, links, and attachments.

[0602] A "generative AI model" refers to an algorithm-based system that uses machine learning to recognize data patterns and identify suspicious emails.

[0603] A "dynamic prompt" is an instruction generated by the server to update the learning process of a generative AI model, and it refers to data guidelines for incorporating new features into the model's responses.

[0604] "Evaluation" refers to the process of determining suspiciousness through scoring and analysis performed by a generating AI model on received data.

[0605] The suspicious email elimination agent system according to the present invention provides advanced detection technology for securely managing incoming emails in email services used by users. This technology is based on a dedicated program that runs on a server and aims to quickly identify phishing emails and fraudulent emails by combining email analysis and machine learning.

[0606] This system is specifically server-centric. The server first retrieves newly received emails from the user's email account. At this stage, it analyzes the email's metadata, such as the sender's address and subject, as well as content data, including the text and links within the email body. The results of this analysis are then input into a generative AI model built on cloud services such as Azure and AWS. The generative AI model used here has learned from a large amount of data and possesses specialized algorithms to extract characteristics of suspicious emails.

[0607] The analyzed data is evaluated by a generative AI model. Based on the evaluation results, emails are classified into specific categories, such as "trustworthy," "suspicious," or "spam." In this process, the server constantly generates new dynamic prompts to reflect the characteristics of the latest emails and updates the model's learning. For example, when a new phishing technique emerges, a prompt such as "Identify the characteristics of recent phishing campaigns and update the model" might be generated.

[0608] Emails deemed suspicious within the classified emails are automatically isolated by the server from the user's regular mailbox. The server then issues a warning to the user's device, notifying them that the suspicious email has been handled and providing information on safety measures. By reviewing this notification, users can take precautions to avoid opening suspicious emails and protect themselves from potential risks.

[0609] In this way, the system of the present invention provides users with an environment in which they can use email without anxiety and enables a rapid response to increasingly sophisticated fraudulent activities.

[0610] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0611] Step 1:

[0612] The server receives new emails in the user's email account. The input includes metadata (sender address, subject) and content data (body text, links) of the received email. This data is then analyzed. Specifically, the server checks whether the email sender is known, whether the subject contains distinctive phrases, and whether there are links in the body. This analysis allows for an initial assessment of whether the email is suspicious.

[0613] Step 2:

[0614] The server inputs the information obtained from the analysis into a generating AI model. The input data is then sent to a scoring process to evaluate the suspiciousness of the emails. In this process, a model that has learned the characteristics of known suspicious emails analyzes the content and patterns of the emails. As output, a score is generated for each email. Specifically, it quantifies how high the risk is based on the format of the links included in the email and the degree of matching of the metadata.

[0615] Step 3:

[0616] The server classifies emails into specific categories based on scores generated by the AI ​​model. High scores classify emails as "suspicious" or "spam," while low scores classify them as "trustworthy." Specifically, emails with scores above a certain level are automatically moved to a quarantine folder, separating them from other messages. The input for this step is the scoring result, and the output is the email classification result.

[0617] Step 4:

[0618] The server sends a notification to the user's device based on the categorization results. For emails deemed suspicious, a warning message is displayed on the user's screen. The input is the classification result from the previous step, and the output is the warning notification. Specifically, the user is provided with a notification such as, "A new suspicious email has been detected. Please check the details."

[0619] Step 5:

[0620] The server generates dynamic prompts based on the characteristics of newly discovered suspicious emails. To update the generating AI model, the server creates a prompt such as "Learn new features to identify recent phishing emails." This prompt will be used in the next model training. The input for this step is data of new suspicious emails, and the output is the prompt. Specifically, the server prepares guidelines for improving the accuracy of the model.

[0621] (Application Example 1)

[0622] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0623] Protecting users quickly and effectively from suspicious emails is a critical challenge in today's information society. However, existing systems are time-consuming to identify and respond to suspicious emails, potentially leaving users vulnerable to harm. This invention aims to solve these problems by rapidly detecting suspicious emails and providing users with immediate warnings.

[0624] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0625] In this invention, the server includes means for acquiring received information and analyzing metadata and content information of said information; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific category; and means for displaying the information classified into a specific category in real time on a smart device. This enables users to receive prompt warnings of suspicious emails and take immediate and secure action.

[0626] "Received information" refers to all data that a server or device acquires from external sources, and in particular includes email metadata and content information.

[0627] "Metadata" refers to additional information about emails other than their content, such as the sender's address, subject, and date and time of sending.

[0628] "Content information" refers to the actual information content, such as text and links, included in the body of an email.

[0629] "Means of analysis" refers to processes and algorithms for analyzing the metadata and content information of received information and extracting suspicious features.

[0630] A "machine learning model" is a program that learns from past data and identifies the characteristics of suspicious emails, and is used for evaluating emails.

[0631] "Means of evaluation" refers to the process of using machine learning models to determine whether the analyzed information is suspicious or not.

[0632] "Specific categories" refer to groups of emails, such as suspicious emails and regular emails, that are classified through analysis and evaluation of received information.

[0633] "Automated processing methods" refer to processes that perform actions such as deletion or isolation on information classified into specific categories without user intervention.

[0634] A "smart device" is a portable device equipped with information processing and communication functions, and specifically refers to smartphones and tablet devices.

[0635] "Means of displaying information in real time" refers to the process of visually presenting information categorized into a specific category on a device immediately.

[0636] This invention realizes a suspicious email detection and warning system on smart devices. The server acquires incoming information from external sources and analyzes its metadata and content information. Python or email processing libraries can be used to analyze the sender address, subject, and links within the email body. The analyzed information is input into a machine learning model, which uses libraries such as scikit-learn to evaluate whether the email is suspicious.

[0637] The server categorizes information into suspicious emails and normal emails based on its evaluation. Emails deemed suspicious with a high score are automatically moved to a quarantine folder or deleted. This information is notified to the user's smart device in real time. This allows the user to receive an immediate warning and take a safe action. Smartphones are equipped with iOS or Android OS, and compatible apps are typically developed using Swift or Kotlin.

[0638] For example, when a user has a question like, "Is this email suspicious?", the application analyzes the email in the background and notifies the user that "This email may be suspicious." This helps the user when making important decisions.

[0639] Examples of prompts to input into a generative AI model:

[0640] "Please analyze the subject lines and sender information of today's new emails to detect suspicious emails."

[0641] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0642] Step 1:

[0643] The server retrieves received information from the mail server. The input is an email, and the output is the email's metadata (sender address, subject, date and time sent) and content information (body, links). Specifically, the server connects to the mail server using the IMAP protocol and searches for unread emails.

[0644] Step 2:

[0645] The server analyzes the received information. The input here is the metadata and content information obtained in step 1, and the output is analysis data for extracting suspicious features. Specifically, text analysis is used to analyze whether there are specific patterns in the email sender or subject line, or whether there are suspicious links or phrases in the body of the email.

[0646] Step 3:

[0647] The server inputs the analyzed data into a machine learning model for evaluation. The input is the analyzed data from step 2, and the output is a score indicating whether the email is suspicious or not. Specifically, the scikit-learn library is used to perform scoring with a pre-trained model. A higher score indicates that the email is suspicious.

[0648] Step 4:

[0649] The server classifies and processes information based on the evaluation results. The input is the evaluation score from step 3, and the output is the action indicating whether the suspicious email will be moved to a quarantine folder or deleted. Specifically, it implements a function to move emails that exceed a certain score threshold to a quarantine folder.

[0650] Step 5:

[0651] The device displays a warning to the user in real time. The input is the result of the processing in step 4, and the output is the warning message displayed to the user. Specifically, the smartphone app uses push notifications to immediately inform the user that the email is suspicious.

[0652] Step 6:

[0653] The user reviews the warning and decides what to do about the suspicious email. The input is the warning message from step 5, and the output is selected as an action for the user to handle the email safely. The user receives a notification and takes action, such as not clicking on links in the suspicious email.

[0654] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0655] This invention enhances the user experience by combining a system that effectively detects and processes suspicious emails with an emotion engine that recognizes user emotions in real time. First, the server receives email data and analyzes its metadata and content. The analyzed data is scored by a machine learning model and classified according to its level of suspiciousness. While suspicious emails are quarantined or deleted through this process, this invention further adds a process in which the emotion engine analyzes the user's emotions.

[0656] The server uses an emotion engine to monitor the emotional impact that emails categorized into specific groups have on users. It captures in real time how users react when they receive certain email notifications on their devices, and if a negative reaction is detected, it can adjust the frequency and display method of notifications. For example, if a device notifies a user that "an important email has been identified as suspicious," the emotion engine will instruct the device to soften the notification to avoid causing the user emotional stress.

[0657] Furthermore, the emotion engine collects user feedback as training data, contributing to improved accuracy in emotion recognition. This allows the server to predict user emotions more accurately and optimize the methods and content of email notifications. The present invention not only processes suspicious emails but also aims to reduce the user's mental burden and provide a safer and more comfortable communication environment.

[0658] The following describes the processing flow.

[0659] Step 1:

[0660] The server receives new email data from the user's mail server. It retrieves the email's metadata (sender, date and time, subject, etc.) and content data, and converts it into a format that can be parsed within the program.

[0661] Step 2:

[0662] The server inputs the acquired email data into a machine learning model to evaluate its suspiciousness. The model uses past learning results to assign a score to the email and determine whether it is malicious or not.

[0663] Step 3:

[0664] The server classifies emails as "safe," "suspicious," or "dubious" based on a score. Emails deemed suspicious are moved to a separate folder or deleted.

[0665] Step 4:

[0666] The server receives the results of categorization and activates the sentiment engine. It evaluates in real time how emails categorized as "suspicious," in particular, affect the user's emotions.

[0667] Step 5:

[0668] The terminal receives instructions from the server and sends notifications to the user based on the content and status of the email. The notifications are adjusted to reduce the user's emotional response to the email content (e.g., stress or anxiety).

[0669] Step 6:

[0670] Based on the user's emotional response, the emotion engine sends feedback to the server to optimize future notifications and the user interface. This makes notification methods more effective and comfortable for the user.

[0671] Step 7:

[0672] The server incorporates feedback from the emotion engine into its training data, continuously improving the accuracy of email suspicion detection and availability notifications. This allows the system as a whole to provide a safer and less stressful environment for users.

[0673] (Example 2)

[0674] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0675] Conventional suspicious email detection systems typically focused on isolating or deleting suspicious emails, but they did not consider the emotional impact on users. Therefore, there was a need for measures to mitigate the stress and discomfort users experienced when receiving emails deemed suspicious. Furthermore, there was a need to provide a safe and comfortable communication environment that considered both the detection of suspicious emails and the user's emotional state.

[0676] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0677] In this invention, the server includes means for acquiring received data and analyzing attribute information and content data of the data; means for inputting the analyzed data into a machine learning model and evaluating it; means for classifying the data based on the evaluation and automatically processing the data classified into a specific category; means for using an emotion analysis engine to analyze user reactions; and means for adjusting the frequency and display method of notifications according to the user's emotional state. This makes it possible to reduce the mental burden on users when dealing with suspicious emails and to realize a safe and comfortable communication environment that further improves the user experience.

[0678] "Received data" refers to a collection of information and signals taken into the system from an external source, and is the digital data format used for its analysis.

[0679] "Attribute information" refers to metadata associated with data, including information such as the sender, recipient, and date and time of transmission.

[0680] "Content data" refers to the actual content other than metadata, and in the case of an email, it refers to the content of the body and attachments.

[0681] "Analysis" is a process that aims to break down and examine data in order to detect its meaning and patterns.

[0682] A "machine learning model" is an algorithmic framework used to make predictions and classifications based on input data, learning from large amounts of data and applying the results.

[0683] "Evaluation" is the process of quantifying the importance and characteristics of analyzed data and measuring its performance and value.

[0684] A "category" is a classification criterion that indicates a group of data that share specific characteristics, and is used to streamline data organization and processing.

[0685] "Automated processing" refers to a method of managing and processing data without human intervention, based on pre-defined rules or algorithms.

[0686] An "emotion analysis engine" refers to a technical tool that infers and analyzes a user's emotions from their voice, facial expressions, and other data.

[0687] A "notification" is a means of communication used to inform a user of specific information or circumstances, and can take the form of visual or auditory communication.

[0688] This invention provides a system for detecting suspicious communications, analyzing users' emotional responses, and improving the user experience. This system is implemented through the interaction of a server, a terminal, and a user.

[0689] The server acquires the received communication data and analyzes its attribute information and content. Natural language processing libraries such as NLTK and spaCy are used for this analysis. The analyzed data is then evaluated using machine learning platforms such as TensorFlow and Sci-kit Learn. This allows the data to be classified into specific categories based on its suspicious nature.

[0690] Next, the device receives a notification from the server and activates the sentiment analysis engine. The sentiment analysis engine uses a generative AI model to analyze the user's reactions in real time. Specifically, it utilizes the user's facial expressions and voice data acquired through the device's camera and microphone. For example, OpenAI's generative AI model is used for sentiment analysis.

[0691] If a user shows a negative reaction to data categorized into a specific group, the device adjusts the frequency and display method of notifications based on instructions from the sentiment analysis engine. This reduces user stress and ensures a more comfortable user experience.

[0692] For example, if a device sends a notification to a user stating that "certain information has been deemed suspicious," and the user finds this notification stressful, the notification content will be changed to softer language and the notification sound will be reduced to show consideration for the user's feelings.

[0693] A concrete example of a prompt message is: "Consider a system that recognizes user emotions in real time and optimizes the notification method for suspicious communications. Please come up with ideas for devising notification methods that will not cause stress to the user."

[0694] In summary, the present invention aims to provide a safer and more comfortable communication environment by detecting suspicious emails while also considering the user's emotional response.

[0695] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0696] Step 1:

[0697] The server retrieves received data from the mail server. This input includes various data such as email headers, sender, recipient, and message body. The server uses this input data to parse metadata and content using natural language processing libraries (such as NLTK or spaCy). This analysis tokenizes the data, converting each element into a format that is easier to parse. The output is the parsed structured data.

[0698] Step 2:

[0699] The server inputs the analyzed data into a machine learning model (e.g., TensorFlow, Sci-kit Learn) to calculate a suspiciousness score. Based on the input data, it evaluates the data's characteristics using a specific algorithm. This process classifies the data as either a normal email or a suspicious email. The output includes the score and the classification result.

[0700] Step 3:

[0701] The server automatically processes email data evaluated based on specific criteria. Specifically, highly suspicious emails are moved to a quarantine folder or permanently deleted. This process ensures email security and protects the user's communication environment. A list of emails classified as safe is generated as output.

[0702] Step 4:

[0703] The device receives notifications sent from the server and activates the emotion analysis engine. The input is the email notification information from the server. The device captures the user's real-time facial expressions and voice and analyzes this data using a generative AI model. The output is the analysis result indicating the user's emotional state.

[0704] Step 5:

[0705] The device adjusts the display method and frequency of notifications based on the sentiment analysis results. Specifically, if the user indicates a negative emotion, it will soften the tone of the notification message on the screen and lower the notification sound. This adjustment reduces user stress and provides a more comfortable experience. The output is the notification method optimized after the adjustments.

[0706] Step 6:

[0707] Users provide feedback on notifications. The device sends this feedback information to the server. The server uses this as training data for a machine learning model and performs continuous learning. The output is an improved model that reflects the feedback. This process improves the overall accuracy of the system and the user experience.

[0708] (Application Example 2)

[0709] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0710] One problem with email is that suspicious emails can cause psychological distress to users. Receiving suspicious emails can cause users to feel stressed or anxious. Furthermore, traditional email filtering systems may mistakenly classify important emails as suspicious, leading to the risk of missing crucial information. In addition, frequent email notifications can become a daily annoyance, detracting from the user experience. To solve these problems, there is a need for a system that incorporates a function to recognize user emotions in real time and adjust notification methods accordingly.

[0711] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0712] In this invention, the server includes means for acquiring received data and analyzing the indicator information and content information of the data; means for inputting the analyzed information into a machine learning model and evaluating it; means for classifying the information based on the evaluation and automatically processing the information classified into a specific area; and means for analyzing the user's emotional state in real time and adjusting the notification method according to the emotional state. This makes it possible to appropriately manage notifications of suspicious emails while taking into account the user's emotions, thereby reducing psychological burden and providing an environment in which important information is not missed.

[0713] "Received data" refers to the set of information that the system acquires from an external source and processes.

[0714] "Metric information" refers to metadata and attribute data included in received data, which serves as supplementary information for understanding the nature and characteristics of that data.

[0715] "Content information" refers to essential data such as text, numbers, and images contained within the received data.

[0716] A "machine learning model" is a mathematical model that uses algorithms to learn patterns in data and perform predictions and classifications.

[0717] "Real-time analysis" is a process that performs analysis immediately at the time intervals in which data is generated, and obtains results instantly.

[0718] A "specific domain" refers to a data category classified based on evaluation using machine learning.

[0719] "Automatic processing" means that the system manipulates or manages data according to established criteria without requiring human intervention.

[0720] "Emotional state" refers to the user's psychological and emotional condition, and this information influences their actions and responses.

[0721] "Notification methods" refer to the methods and formats used to convey information to users, including sound, visual effects, and text.

[0722] This invention provides a system that recognizes user emotions in real time and appropriately adjusts the notification method for suspicious emails. The system receives, analyzes, evaluates, and classifies data, and measures emotions and adjusts notifications accordingly.

[0723] The server first acquires the received data and then analyzes its metric and content information in detail. This process uses the data stream provided by the email client and employs analysis software to decompose important metadata and content information. The analyzed information is then input into a machine learning model. Here, machine learning frameworks such as TensorFlow are used to evaluate the suspiciousness of the data and classify it into specific areas.

[0724] Next, the system analyzes the user's emotional state in real time based on the classified data. This emotion recognition utilizes tools such as the Google Cloud Natural Language API, and sensors may be installed on the device to capture the user's facial expressions and reactions. If the system determines that the user's emotional state is negative, the server adjusts the notification method and softens the tone of the suspicious email notification. This aims to improve the user experience.

[0725] For example, when a user receives an email that has been flagged as suspicious, the server adjusts its system to reduce the user's psychological burden by sending a notification with gentle wording such as, "Please handle this email with care."

[0726] An example of a prompt might be, "Analyze the user's facial expression data regarding this email and suggest ways to adjust the notification tone if there is a stress response." Based on this prompt, the AI ​​model will customize the optimal notification method based on human emotions.

[0727] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0728] Step 1:

[0729] The server receives data from the email client. It obtains email metadata and content information as input data. Based on this information, the server analyzes the metadata (e.g., sender, subject, sending time, etc.) and extracts the content data. Initial spam filtering is performed through the analysis of the metadata.

[0730] Step 2:

[0731] The server inputs the analyzed metadata and content information into a machine learning model. The model is trained using TensorFlow and determines the suspiciousness of the data. The model outputs a suspiciousness score, and based on that score, the data is classified into different categories (e.g., spam, normal, caution).

[0732] Step 3:

[0733] The server automatically processes data based on its suspiciousness score. Suspicious data classified into specific areas is either isolated or deleted. Information deemed normal is sent to the user's terminal. This process is performed automatically by the system without any user intervention.

[0734] Step 4:

[0735] The user's terminal receives email data from the server and analyzes the user's emotional state in real time. Sensors and cameras are used to monitor the user's reactions (e.g., facial expressions and voice tone), and the Google Cloud Natural Language API is used to analyze emotions. The results are output as an emotional state.

[0736] Step 5:

[0737] The server adjusts notification methods based on the user's emotional state. If negative emotions are detected, the server generates a prompt to soften the notification and determines the appropriate notification method. For example, it might send a gentle notification message such as "Caution is advised, please proceed cautiously" to the user's terminal.

[0738] Step 6:

[0739] The server collects user feedback and uses it to continuously improve the system. The feedback data is used to retrain machine learning models and improve the accuracy of the sentiment engine. This allows the system to continuously improve the user experience.

[0740] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0741] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0742] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0743] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0744] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0745] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0746] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0747] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0748] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0749] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0750] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0751] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0752] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0753] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0754] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0755] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0756] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0757] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0758] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0759] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0760] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0761] The following is further disclosed regarding the embodiments described above.

[0762] (Claim 1)

[0763] A means for acquiring received data and analyzing the metadata and content data of said data,

[0764] A means for inputting the analyzed data into a machine learning model and evaluating it,

[0765] A means for classifying data based on the aforementioned evaluation and automatically processing data classified into a specific category,

[0766] A system that includes this.

[0767] (Claim 2)

[0768] The system according to claim 1, further comprising means for notifying a user terminal of data classified into a specific category.

[0769] (Claim 3)

[0770] The system according to claim 1, wherein the means for classifying into the aforementioned categories includes means for continuously learning by reflecting the input and evaluation of new data.

[0771] "Example 1"

[0772] (Claim 1)

[0773] A means for acquiring received data and analyzing the metadata and content data of said data,

[0774] A means for inputting the analyzed data into a generating AI model and evaluating it,

[0775] A means for classifying data into specific categories based on the aforementioned evaluation and for automatically processing the data based on those categories,

[0776] A means for generating dynamic prompt sentences to learn new features and updating the generating AI model,

[0777] A system that includes this.

[0778] (Claim 2)

[0779] The system according to claim 1, further comprising means for notifying a user device of data classified into a specific category.

[0780] (Claim 3)

[0781] The system according to claim 1, wherein the means for classifying into the aforementioned categories includes means for continuously learning by reflecting evaluations based on input of new data and past feedback.

[0782] "Application Example 1"

[0783] (Claim 1)

[0784] A means for acquiring received information and analyzing the metadata and content information of said information,

[0785] A means for inputting the analyzed information into a machine learning model and evaluating it,

[0786] A means for classifying information based on the aforementioned evaluation and automatically processing information classified into a specific category,

[0787] A means of displaying information categorized into a specific category on a smart device in real time,

[0788] A system that includes this.

[0789] (Claim 2)

[0790] The system according to claim 1, further comprising means for notifying a user device of information classified into a specific category, and for the user to receive an immediate warning.

[0791] (Claim 3)

[0792] The system according to claim 1, wherein the means for classifying into the aforementioned categories reflects the input and evaluation of new information, continuously learns, and improves the accuracy of the machine learning model by utilizing feedback based on user operations.

[0793] "Example 2 of combining an emotion engine"

[0794] (Claim 1)

[0795] A means for acquiring received data and analyzing the attribute information and content data of said data,

[0796] A means for inputting the analyzed data into a machine learning model and evaluating it,

[0797] A means for classifying data based on the aforementioned evaluation and automatically processing data classified into a specific category,

[0798] A means of using an emotion analysis engine to analyze user reactions,

[0799] A means of adjusting the frequency and display method of notifications according to the user's emotional state,

[0800] A system that includes this.

[0801] (Claim 2)

[0802] The system according to claim 1, further comprising means for notifying a user terminal of data classified into a specific category.

[0803] (Claim 3)

[0804] The system according to claim 1, wherein the means for classifying into the aforementioned categories includes means for continuously learning by reflecting the input and evaluation of new data.

[0805] "Application example 2 when combining with an emotional engine"

[0806] (Claim 1)

[0807] A means for acquiring received data and analyzing the indicator information and content information of said data,

[0808] A means for inputting the analyzed information into a machine learning model and evaluating it,

[0809] A means for classifying information based on the aforementioned evaluation and automatically processing information classified into a specific domain,

[0810] A means for analyzing the user's emotional state in real time and adjusting the notification method according to that emotional state,

[0811] A system that includes this.

[0812] (Claim 2)

[0813] The system according to claim 1, further comprising means for notifying a user device of information classified into a specific area, monitoring the emotional impact of the notification on the user, and changing the tone of the notification as necessary.

[0814] (Claim 3)

[0815] The system according to claim 1, wherein the means for classifying into the aforementioned domains includes means for continuously learning by reflecting the input and evaluation of new information, and means for improving the accuracy of emotion recognition by utilizing user feedback. [Explanation of symbols]

[0816] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring received data and analyzing the metadata and content data of said data, A means for inputting the analyzed data into a machine learning model and evaluating it, A means for classifying data based on the aforementioned evaluation and automatically processing data classified into a specific category, A system that includes this.

2. The system according to claim 1, further comprising means for notifying a user terminal of data classified into a specific category.

3. The system according to claim 1, wherein the means for classifying into the aforementioned categories includes means for continuously learning by reflecting the input and evaluation of new data.

Images