system

Abstract

Provide a system. [Solution] An information acquisition means; A means for inputting the acquired information into an application and transmitting it to a server; a means for the server to analyze the information using a neural network; A means for the server to organize the analysis results and return them to the terminal; A means for notifying the user of the results of the device organization; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The rapid growth of generative AI technology has made it easy for individuals to create fake information. As a result, a large amount of fake news, fabricated videos, and photographs are circulating online, causing social unrest and increasing criminal activity. For example, crimes such as telephone fraud and impersonation have become more sophisticated with the use of fake technology. In response to this situation, it is necessary to provide a safe and reliable information environment by quickly and accurately detecting fake information and notifying users. [Means for solving the problem]

[0005] This invention provides a system that analyzes fake information using multi-layered detection technology. Specifically, the user inputs the information they acquire into an application and sends it to a server. The server then uses a neural network to analyze the authenticity of the information, organizes the results, and returns them to the device. The device then immediately notifies the user of the results, thereby detecting fake information. This allows the user to check the reliability of the information in real time and use it safely. The system also includes means for multi-layered analysis of image, audio, and text elements, enabling detailed analysis depending on the type of fake information.

[0006] "Means of information acquisition" refers to the means by which a user captures information using a camera, screenshots, etc., and saves it on the device.

[0007] "Application" means a software program that runs on a User's Device and transmits captured information to a Server for analysis.

[0008] A "server" is a computer system on a network that receives, processes, and analyzes information sent from a client.

[0009] A "neural network" is a form of artificial intelligence, specifically an algorithm that uses deep learning to perform complex pattern recognition and analysis.

[0010] "Means of analysis" refers to the process of evaluating the authenticity of information using software or algorithms implemented on a server.

[0011] "Organization" is the process of compiling the processing results performed on the server into a meaningful form and converting it into a format for notifying the user.

[0012] A "terminal" is an information processing device used by a user, such as a smartphone or tablet.

[0013] "Means of notification" refers to functions such as pop-up notifications and message displays that the device uses to inform the user of the analysis results.

[0014] "Real-time" means that the entire process is carried out instantly, meaning that the time between when the user obtains the information and when they receive the analysis results is extremely short.

[0015] "Multi-layered detection technology" is a method of identifying fake information while increasing accuracy by using multiple analysis methods depending on the type of information (images, audio, text, etc.). [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[0038] System Overview

[0039] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to identify fake information. The server then sends back the organized analysis results to the device, which notifies the user in real time. This series of processes allows users to quickly and accurately verify the reliability of the information.

[0040] System Details

[0041] 1. Information acquisition means:

[0042] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0043] 2. Application:

[0044] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0045] 3. Server:

[0046] The server receives the data sent from the application. After receiving the data, it is decoded and input into a neural network model. Using deep learning technology, the server evaluates the authenticity of the information. The analysis results in a decision on whether the information is fake and its reliability.

[0047] 4. Organizational means:

[0048] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0049] 5. Means of notification:

[0050] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[0051] Specific examples

[0052] Example 1: Fake news detection

[0053] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[0054] Example 2: Deepfake video detection

[0055] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[0056] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] The user acquires information. Specifically, the user uses the camera function of their smartphone to take photos or videos, or to take screenshots.

[0060] Step 2:

[0061] The information acquired by the device is input into the application, which then accesses data stored on the device.

[0062] Step 3:

[0063] The device sends information to the server through the application, which encrypts the information for security purposes and sends it to the server using the SSL / TLS protocol.

[0064] Step 4:

[0065] The server receives the transmitted information. First, the server decrypts the received data.

[0066] Step 5:

[0067] The server analyzes the information. The decoded data is fed into a neural network model and deep learning techniques are used to assess the authenticity of the information. For example, image analysis algorithms are applied to images, and natural language processing techniques are used to assess text.

[0068] Step 6:

[0069] The server organizes the analysis results. Specifically, it summarizes the results of its judgment on whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[0070] Step 7:

[0071] The server then sends the compiled analysis results back to the device, which are also encrypted and transmitted using the SSL / TLS protocol.

[0072] Step 8:

[0073] The terminal receives and decodes the analysis results. The terminal receives the data from the server and converts it into a format that is easy for the user to understand.

[0074] Step 9:

[0075] The device will notify the user of the results. Specifically, it will display a pop-up notification or message, such as "This news article may be fake news (90% confidence level)."

[0076] The above is the specific processing flow of this system. This series of steps allows users to quickly and accurately confirm the reliability of the information they have obtained, enabling them to use the information safely.

[0077] Example 1

[0078] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0079] In recent years, the increase in fake information has led to social unrest and increased fraud. This problem is particularly serious in today's society, where information spreads instantly via the Internet. Current methods of verifying information often lack real-time capabilities and require users to determine the authenticity of information themselves, which requires a certain level of specialized knowledge. This creates a risk that ordinary users can easily spread unreliable information.

[0080] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0081] In this invention, the server includes means for inputting acquired information into an application, encrypting it, and sending it to the server, means for the server to receive the encrypted information, decrypt it, and analyze it using a neural network, and means for the server to organize the analysis results, re-encrypt the results, and send them back to the terminal. This makes it possible for information acquired by the user to be analyzed in real time, its reliability evaluated, and the results notified immediately.

[0082] "Means of information acquisition" refers to the means by which a user acquires data such as images, text, and audio using a device such as a smartphone or computer.

[0083] An "application" is software that runs on a terminal, encrypts acquired information, and transmits it to a server.

[0084] "Encryption" is the process of transforming transmitted data so that it cannot be deciphered by third parties.

[0085] A "server" is a computer system that receives information over a network, decodes it, and analyzes it using a neural network.

[0086] "Decryption" is the process of restoring encrypted data to its original state.

[0087] A "neural network" is an artificial intelligence model that analyzes and evaluates information by mimicking the function of nerve cells in the human brain.

[0088] "Analysis" is the process of examining information in detail to find specific features or patterns.

[0089] "Organization" is the process of converting the analysis results obtained within the server into a format that is easy for users to understand.

[0090] A "terminal" is a device that is directly operated by a user and has the role of obtaining and notifying information.

[0091] "Notification" refers to the means by which the device displays the analysis results to the user, and includes methods such as pop-up notifications and message displays.

[0092] "Real-time" refers to processes that are executed immediately, without delay.

[0093] "Reliability" is an indicator of the veracity and accuracy of information obtained through analysis.

[0094] "Multi-layered" means that the analysis is divided into multiple elements or layers, and refers to the simultaneous evaluation of different data formats such as images, audio, and text.

[0095] "Fake information" is data that contains intentionally misleading or false information.

[0096] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[0097] System Overview

[0098] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to determine whether it is fake or not. The server then sends back the organized analysis results to the device, which notifies the user in real time. This process allows users to quickly and accurately verify the reliability of the information.

[0099] Information acquisition means

[0100] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0101] application

[0102] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0103] server

[0104] The server receives the data sent from the application. After receiving the data, it is decrypted and input into a neural network model. Using deep learning technology, the server analyzes the authenticity of the information. The analysis results in a judgment of whether the information is fake and its reliability.

[0105] means of organization

[0106] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0107] Notification means

[0108] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[0109] Specific examples

[0110] Example 1: Fake news detection

[0111] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[0112] Example 2: Deepfake video detection

[0113] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[0114] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[0115] Prompt Sentence Examples

[0116] Please explain the process flow of the system program in detail using the following steps. The subject should be either the server, terminal, or user.

[0117] procedure:

[0118] 1. The user obtains the information.

[0119] 2. The device passes the information to the application.

[0120] 3. The application encrypts the information and sends it to the server.

[0121] 4. The server receives and decrypts the information.

[0122] 5. The server inputs the information into a neural network model for analysis.

[0123] 6. The server organizes the analysis results.

[0124] 7. The server encrypts the results and sends them to the device.

[0125] 8. The device receives the results and notifies the user.

[0126] Examples:

[0127] Users take screenshots of news articles and save them to their devices. The application encrypts them and sends them to a server. The server analyzes the screenshots and notifies the user if they are deemed to be fake news.

[0128] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0129] Step 1:

[0130] The user acquires information. The user acquires image or text information using the smartphone's camera or screenshot function. For example, the user views an article on a news site and takes a screenshot of it. The input data is the image or text. The output is the acquired information stored on the device.

[0131] Step 2:

[0132] The device passes information to the application. The device passes the captured image and text information to the application, which then retrieves it. During this process, screenshots stored on the device are imported into the application. Input data is information stored on the device, and output data is information imported into the application.

[0133] Step 3:

[0134] The application encrypts information and sends it to the server. The application encrypts information using the SSL / TLS protocol and sends it to the server. The input data here is the unencrypted information imported into the application, and the output data is the encrypted data sent to the server.

[0135] Step 4:

[0136] The server receives and decrypts the information. The server receives and decrypts the encrypted data. Inside the server, the data is decrypted using the AES algorithm. The input data is the encrypted information, and the output data is the decrypted information.

[0137] Step 5:

[0138] The server inputs the information into a neural network model for analysis. Using deep learning technology, the server inputs the decoded information into the neural network model for analysis. For example, the text of a news article is analyzed using the BERT model, and the image is analyzed using the ResNet model. The input data is the decoded information, and the output data is the analysis result.

[0139] Step 6:

[0140] The server organizes the analysis results. The server converts the analysis results into a user-friendly format and assigns a reliability rating. For example, the server might say, "This article may be fake news (90% reliability)." The input data is the analysis results, and the output data is the organized results.

[0141] Step 7:

[0142] The server encrypts the organized results and sends them to the terminal. The server re-encrypts the organized results and sends them to the terminal. The input data is the organized results, and the output data is the encrypted results.

[0143] Step 8:

[0144] The device receives the results and notifies the user. The device receives the encrypted data sent from the server and decrypts it. It then displays a pop-up notification or message to the user. For example, it displays a notification saying, "This article may be fake news (90% confidence level)." The input data is the encrypted result, and the output data is the result notified to the user.

[0145] (Application example 1)

[0146] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0147] The widespread use of the Internet has led to the rapid spread of harmful information, such as false information, fake news, and phishing links. This has led to many users becoming victims of fraud and experiencing social unrest due to misinformation. However, current systems lack a way to detect this fake information in real time and promptly notify users. In particular, there is a need for a means to instantly evaluate the reliability of links and attachments in electronic messages.

[0148] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0149] In this invention, the server includes means for acquiring information about links and attachments in electronic messages in real time, means for determining the authenticity of links and attachments using a neural network, and means for notifying the user of the authenticity information, which allows the user to evaluate the reliability of links and attachments in emails and messages in real time and prevent damage from fake information and phishing sites.

[0150] "Information acquisition means" refers to the function of a device or software that collects information (links, attachments, images, audio, text, etc.) that a user acquires.

[0151] An "application" is a program that sends acquired information to a server and analyzes and organizes the results.

[0152] A "server" is a computer system that receives information sent from an application and analyzes it using a neural network.

[0153] A "neural network" is a machine learning model that uses multiple layers of artificial neurons, and is a technology used to determine the authenticity of information.

[0154] The "organizing means" is a process in which the server converts the analysis results into a format that is easy for the user to understand and returns it to the terminal.

[0155] "Notification means" is a function that notifies the user of the organized results as a pop-up notification or message display.

[0156] "Links and attachments in electronic messages" are URL links and files contained in email or messaging applications.

[0157] "Real-time retrieval" refers to the ability to retrieve links and attachments in electronic messages immediately upon their arrival.

[0158] The "means for determining authenticity" is a function that uses a neural network to analyze acquired information and determine whether the information is fake.

[0159] "Means of notifying including reliability information" is a function that informs the user of the analysis results and their reliability (e.g., reliability 85%).

[0160] The present invention is a system that allows users to evaluate the reliability of links and attachments in electronic messages in real time, quickly determining whether the information is fake and notifying users. This system uses a smartphone, a server, and an application that links them.

[0161] System Overview

[0162] Users use an application installed on their smartphone to retrieve links and attachments from electronic messages and emails. This information is encrypted and sent to a server via the application. The server analyzes the information using a neural network model and notifies the user of the results in real time.

[0163] System Details

[0164] 1. Information acquisition means:

[0165] After users retrieve links and attachments in electronic messages and emails, the information is automatically collected by the application, which uses a module that retrieves information from the API of the smartphone's default email and messaging apps.

[0166] 2. Application:

[0167] The application receives the acquired information as input and encrypts the data, using the SSL / TLS protocol to ensure secure communication. The encrypted data is then sent to the server.

[0168] 3. Server:

[0169] The server is installed on AWS (registered trademark) EC2 or Google (registered trademark) Cloud Platform. The server decrypts the received data and analyzes it using a neural network model using PyTorch or TENSORFLOW (registered trademark). This analysis determines the authenticity of links and attachments and evaluates their reliability as a numerical value (e.g., 85% reliability).

[0170] 4. Organizational means:

[0171] The server then organizes the analysis results and converts them into a user-friendly format. For example, the result may be displayed as "This link may be a phishing site (confidence 85%)." The confidence level is also displayed, allowing users to trust the results.

[0172] 5. Means of notification:

[0173] The results are then encrypted and sent back to the device, where the smartphone application receives them and displays a pop-up notification or message to the user.

[0174] Specific examples

[0175] Example 1: Evaluating the trustworthiness of links in emails

[0176] When a user receives an email and tries to click on a link, the application automatically captures the link and sends it to the server, which analyzes the link's reliability and sends a notification to the user saying, "This link may be a phishing site (85% confidence)."

[0177] Example 2: Authentication of attached files

[0178] To verify the authenticity of an attachment received by a user via a messaging app, the user uses the application to send the file to a server, which analyzes the contents of the file and notifies the user that "This file may contain a virus (90% confidence level)."

[0179] Example prompts to input to the generative AI model

[0180] Generative AI model, please analyze whether the following link is fake: example_link. Print the result and confidence level.

[0181] This embodiment allows users to quickly evaluate the reliability of links and attachments in electronic messages and emails, and can prevent damage from fake information and phishing sites.

[0182] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0183] Step 1:

[0184] A user receives an email or message and retrieves a link or attachment from it. An application automatically detects the link or attachment information. The input here is the link or attachment from the electronic message, and the output is the raw data passed to the application.

[0185] Step 2:

[0186] The information obtained by the application is encrypted using the SSL / TLS protocol. The input is raw data and the output is encrypted data. Encryption ensures the protection of transmitted data.

[0187] Step 3:

[0188] The terminal sends encrypted data to the server. The input is the encrypted data, and the output is the received data that is recorded in the server's reception log. This transmission uses Internet communication.

[0189] Step 4:

[0190] The server receives the encrypted data and decrypts it using the SSL / TLS protocol. The input is the encrypted data and the output is the original, decrypted raw data.

[0191] Step 5:

[0192] The server uses PyTorch or TensorFlow to analyze the authenticity of links and attachments using a neural network model. The input is the decoded raw data, and the output is the analysis result (evaluation of whether the information is fake or not) and its reliability score. Specific operations include pattern recognition, feature extraction, and evaluation of links and files.

[0193] Step 6:

[0194] The server organizes the analysis results and converts them into a user-friendly format. The input is the analysis results and confidence score, and the output is a user notification such as "This link may be a phishing site (confidence 85%)."

[0195] Step 7:

[0196] The server re-encrypts the result and sends it to the terminal. The input is the result, and the output is the encrypted result data.

[0197] Step 8:

[0198] The terminal receives the encrypted result data and decrypts it again. The input is the encrypted result data, and the output is the decrypted notification data.

[0199] Step 9:

[0200] The application notifies the user of the notification data as a popup notification or message display. The input is the decoded notification data, and the output is the message or notification that is displayed on the user's device screen.

[0201] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0202] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users by combining it with an emotion engine to provide appropriate notifications that take user emotions into consideration. This system is realized by the user's device, a server, an application, and an emotion engine.

[0203] System Overview

[0204] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[0205] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[0206] System Details

[0207] 1. Information acquisition means:

[0208] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0209] 2. Application:

[0210] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0211] 3. Server:

[0212] The server receives data sent from the application. First, the server decrypts the received data. Next, it inputs the decrypted data into a neural network model and uses deep learning technology to evaluate the authenticity of the information. For example, in the case of images, it applies an image analysis algorithm, and in the case of text, it uses natural language processing technology.

[0213] 4. Organizational means:

[0214] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0215] 5. Emotion Engine Processing:

[0216] The emotion engine analyzes the user's facial expressions and voice data in real time, thereby determining the user's current emotional state and sending it to the server.

[0217] 6. Sentiment data analysis on the server:

[0218] The server then combines the received emotional data with the results of its fake information analysis to generate notification content that reflects the user's emotions. For example, if the server determines that the user is feeling extremely anxious, it will adjust the notification content to be gentler.

[0219] 7. Means of notification:

[0220] The sorted results are then encrypted again and sent back to the device. After receiving the results, the device displays a pop-up notification or message to the user, instantly informing them of the reliability of the information. Appropriate notification content is also displayed based on the data from the emotion engine.

[0221] Specific examples

[0222] Example 1: Fake news detection and emotional response

[0223] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, the emotion engine analyzes the user's facial expression. If the user looks anxious, the system displays a message such as, "This news article may be fake news (90% confidence). Don't worry, you can also find other reliable information."

[0224] Example 2: Deepfake video detection and emotion response

[0225] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server then performs multi-layered analysis of the video's frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)." Furthermore, if the emotion engine analyzes the user's tone of voice and determines that the user is surprised, a message similar to, "This video is suspicious, but it's okay. We recommend further verification." is displayed.

[0226] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[0227] The processing flow will be explained below.

[0228] Step 1:

[0229] A user acquires information. Specifically, the user uses the camera function of their smartphone to take a photo or video, or to take a screenshot of a news article.

[0230] Step 2:

[0231] The information acquired by the device is input to the application, which then accesses the data stored on the device and prepares it for analysis.

[0232] Step 3:

[0233] The device sends information to the server through the application, which is then encrypted for security purposes and sent to the server using the SSL / TLS protocol.

[0234] Step 4:

[0235] The server receives the transmitted information and first decrypts the received data.

[0236] Step 5:

[0237] The server analyzes the information. The received and decoded data is fed into a neural network model, which uses deep learning techniques to evaluate the authenticity of the information. For example, if it is an image, an image analysis algorithm is applied, and if it is text, natural language processing technology is used for analysis.

[0238] Step 6:

[0239] The server organizes the analysis results. It summarizes the judgment result regarding whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[0240] Step 7:

[0241] The device recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice in real time to understand their current emotional state.

[0242] Step 8:

[0243] The device sends emotional data to the server, which is also encrypted and sent to the server using the SSL / TLS protocol.

[0244] Step 9:

[0245] The server receives and analyzes the emotion data, decodes the received data, and evaluates the user's emotional state.

[0246] Step 10:

[0247] The server integrates the emotion data and the results of the information analysis. It adjusts the notification content based on the user's emotion and returns it to the device in the most appropriate format. For example, if the user is feeling anxious, the notification will be delivered in a softer tone.

[0248] Step 11:

[0249] The server encrypts the final notification and sends it to the device, again using the SSL / TLS protocol to send the data securely.

[0250] Step 12:

[0251] The device receives and decrypts the final notification. The device receives and decrypts the data sent from the server.

[0252] Step 13:

[0253] The device will notify the user of the results. Specifically, it will inform the user of the notification content through a pop-up notification or message display. For example, a tailored notification such as "This news article may be fake news (90% confidence level). Rest assured, there are other reliable sources available." will be displayed.

[0254] This completes the process flow of the system. This series of steps allows users to quickly and accurately check the reliability of the information they have acquired, while also receiving appropriate notifications based on their emotional state.

[0255] Example 2

[0256] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0257] In modern society, there is a need to quickly and accurately evaluate the reliability of information on the Internet. However, existing technologies lack the accuracy to detect fake information, which creates the risk that users will believe false information. Furthermore, there is a lack of technology that can adjust notification content taking into account the user's emotional state, which hinders efforts to improve the user experience.

[0258] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0259] In this invention, the server includes a means for using a neural network to evaluate the reliability of information, a means for using an emotion engine that analyzes the user's facial expressions and voice data, and a means for integrating the emotion data and the analysis results to adjust the notification content, thereby enabling highly accurate detection of fake information and flexible notification that takes the user's emotional state into consideration.

[0260] "Means of information acquisition" refers to the means by which users acquire information such as photos, videos, and news articles using smartphones or other devices.

[0261] "Application" means software that processes the acquired information on the user's device and transmits it to the server.

[0262] The "server" is a central component of the system that analyzes the received information using a neural network, organizes the results, and returns them to the terminal.

[0263] A "neural network" is a mathematical model that uses artificial intelligence techniques to evaluate the reliability of information.

[0264] The "emotion engine" is a component of the system that analyzes the user's facial expressions and voice data to determine their emotional state.

[0265] "Notification means" is a function for informing the user of the sorting results sent from the server, and includes pop-up notifications and messages displayed on the terminal.

[0266] MODE FOR CARRYING OUT THE INVENTION

[0267] The present invention provides a system for detecting and notifying fake information that takes into account user emotions. This system is realized by a user terminal, a server, an application, and an emotion engine. Detailed embodiments of the system are described below.

[0268] System configuration

[0269] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[0270] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[0271] Hardware and Software Used

[0272] The system uses the following hardware and software:

[0273] 1. Information acquisition means:

[0274] Users capture information using their smartphone's camera, screenshot function, or existing media files.

[0275] Information is stored on the device and accessed by applications.

[0276] 2. Application:

[0277] The acquired information is taken as input, and once the user initiates the analysis using the application, the information is encrypted and sent to the server.

[0278] The SSL / TLS protocol is used to ensure secure communications.

[0279] 3. Server:

[0280] The server decodes the received data and inputs it into a neural network model (e.g., PyTorch or TensorFlow) to evaluate the authenticity of the information.

[0281] For images, image analysis algorithms are applied, and for text, natural language processing techniques are used.

[0282] 4. Emotion Engine:

[0283] The device collects facial and voice data from the camera and microphone and analyzes it in real time.

[0284] An emotion engine (e.g., OpenFace or IBM Watson®) determines the user's emotional state.

[0285] 5. Organizational means:

[0286] The server organizes the analysis results in a format such as "High probability of fake news (95% confidence)" and integrates it with data from the emotion engine to adjust the content of the notification.

[0287] 6. Means of notification:

[0288] The server re-encrypts the results and sends them back to the terminal.

[0289] After the device receives the results, it will display a pop-up notification or message to the user to inform them of the reliability of the information.

[0290] Specific examples

[0291] Example 1: Fake news detection and emotional response

[0292] 1. Information acquisition

[0293] When a user reads an article on a news site, they take a screenshot of the article and send it to a server through the application.

[0294] 2. Data Analysis

[0295] The server analyzes the text and images in the screenshot and determines that the content is likely to be fake news.

[0296] 3. Sentiment analysis

[0297] The emotion engine analyzes the user's facial expressions, and if the user looks anxious, the system will notify them with a message such as, "This news article may be fake (90% confidence). Don't worry, there are other reliable sources available."

[0298] Example 2: Deepfake video detection and emotion response

[0299] 1. Information acquisition

[0300] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server.

[0301] 2. Data Analysis

[0302] The server performs multi-layered analysis of video frames and audio to detect deepfake characteristics.

[0303] 3. Sentiment analysis

[0304] If the emotion engine analyzes the user's tone of voice and determines that the user is surprised, it will display a message such as, "This video is suspicious, but it's okay. We recommend you check it out further."

[0305] Prompt Sentence Examples

[0306] To the generative AI model: Please provide an example prompt that uses a user-generated screenshot of a news article to detect fake news.

[0307] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[0308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0309] System program processing flow

[0310] Step 1: Information Acquisition

[0311] A user uses the camera function or screenshot function of their smartphone to obtain information (photos, videos, news articles, etc.).

[0312] Input: Information from social media, news sites, etc.

[0313] What happens: A user taps the camera icon on their smartphone to launch the camera app and take a screenshot of a suspicious news article.

[0314] Output: Information stored on the device (image files, etc.)

[0315] Step 2: Send data

[0316] The information acquired by the terminal is sent to the server via the application.

[0317] Input: Information stored on the device (image files, etc.)

[0318] What it does: When you tap the "Start Analysis" button in the app, the screenshot is encrypted and uploaded to the server, using the SSL / TLS protocol to ensure secure communication.

[0319] Output: Encrypted information (such as an image file) is sent to the server

[0320] Step 3: Data analysis

[0321] The server decodes the received information and analyzes it using a neural network model.

[0322] Input: Encrypted and transmitted information (such as image files)

[0323] What it does: The server decodes the received screenshot and evaluates its content using an image analysis algorithm (for example, a model using PyTorch or TensorFlow). If text is included, it extracts it using OCR (Optical Character Recognition) technology and analyzes the text using natural language processing technology.

[0324] Output: Evaluation result of whether the information is fake or not and the confidence level

[0325] Step 4: Sentiment analysis

[0326] The device uses a camera and microphone to collect the user's facial expressions and voice data, which is then analyzed in real time by an emotion engine.

[0327] Input: User's facial expression data and speech sound data

[0328] How it works: When a user looks into the smartphone camera, the camera captures their facial expressions and the microphone collects audio data. An emotion engine (e.g., OpenFace or IBM Watson) analyzes this data and estimates the user's emotional state.

[0329] Output: Evaluation of the user's emotional state

[0330] Step 5: Organize the results

[0331] The server organizes the analysis results and converts them into a format that is easy for the user to understand.

[0332] Input: Evaluation results of fake information and evaluation results of user's emotional state

[0333] How it works: The server generates results in the form of "Highly likely to be fake news (95% confidence)" and integrates information and emotional data. The tone and content of the notification message are also adjusted depending on the user's emotional state.

[0334] Output: Organized results (information message)

[0335] Step 6: Send results

[0336] The server re-encrypts the organized results and sends them to the terminal.

[0337] Input: Organized results (notification message)

[0338] How it works: The server encrypts the results again using SSL / TLS and sends them to the device. The notification contains the fake news analysis results and sentiment-based content.

[0339] Output: Encrypted notification message

[0340] Step 7: Notifications

[0341] The terminal receives the results and displays a pop-up notification or message to the user.

[0342] Input: Encrypted notification message

[0343] What it does: The device will display the results it receives from the server, with a pop-up notification on the screen. For example, it might say, "This news article may be fake (90% confidence level). Don't worry, there are other reliable sources available."

[0344] Output: A notification message that is displayed to the user

[0345] Through these steps, the system can analyze the reliability of the information obtained by the user with high accuracy and provide flexible notifications according to the user's emotional state.

[0346] (Application example 2)

[0347] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0348] Current fake information detection systems can instantly assess the authenticity of information, but lack the ability to adjust notification content based on the user's emotional state. This means that if a user feels uneasy or surprised about the reliability of the information, they are unable to respond appropriately, increasing the user's psychological burden. Furthermore, their multi-layered analysis of each type of information is insufficient, limiting the accuracy of their fake information detection.

[0349] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information using a neural network, means for organizing the analysis results and returning them to the terminal, and means having an emotion engine for analyzing the user's facial expressions and voice. This makes it possible to evaluate the reliability of information in real time and adjust the notification content according to the user's emotional state. In addition, multi-layered analysis according to the type of information, such as images, voice, and text, improves the accuracy of detecting fake information.

[0350] "Means of information acquisition" refers to the means by which a user acquires information such as images, audio, and text using a smartphone or other device.

[0351] "Means for inputting information into an application and transmitting it to a server" refers to means for inputting acquired information into an application, encrypting it, and transmitting it securely to a server.

[0352] "Means for analyzing information using neural networks" refers to means for analyzing information obtained on a server using deep learning technology and evaluating its authenticity.

[0353] The "means for organizing the analysis results and returning them to the terminal" refers to a means for converting the results analyzed by the server into a format that is easy for the user to understand and returning them to the terminal for notification.

[0354] "Means for notifying the user of the organized results" refers to means for informing the user of the analysis results using a pop-up notification or message display on the terminal.

[0355] "Means having an emotion engine that analyzes the user's facial expressions and voice" refers to means that uses the user's camera and microphone to analyze the user's facial expressions and voice in real time and recognize the user's emotional state.

[0356] The "means for adjusting the notification content based on the analysis results of the emotion engine" refers to a means for appropriately changing the notification content based on the analysis results in accordance with the emotional state of the user.

[0357] The present invention is a system that analyzes information about videos watched by users in real time, detects fake information, and notifies users appropriately based on their emotional state. This system includes the following elements:

[0358] System configuration and program processing

[0359] 1. Information acquisition means

[0360] Devices such as smartphones and tablets capture frames of the video a user is watching using their camera or screen recording capabilities.

[0361] 2. A means of inputting data into the application and sending it to the server

[0362] The captured video frames are input into a dedicated application, which then encrypts the frames and securely transmits them to the server. The communication between the client and server is secured using the SSL / TLS protocol.

[0363] 3. How to analyze information using neural networks

[0364] The server analyzes the received video frames using deep learning techniques, using OpenCV for image analysis, an NLP (natural language processing) library for text analysis, and a speech recognition library for audio analysis.

[0365] 4. A means of organizing the analysis results and returning them to the device

[0366] The server compiles the analysis results and returns them to the device in an easy-to-understand format, such as "Possibly fake information (confidence level XX%)." This result is also sent encrypted.

[0367] 5. A method for notifying users of the organized results

[0368] The analysis results received by the device are immediately communicated to the user via pop-up notifications or message displays, with the content of the notifications tailored based on the user's emotional state.

[0369] 6. Means for providing an emotion engine that analyzes the user's facial expressions and voice

[0370] The system uses the smartphone's camera and microphone to collect the user's facial expressions and voice in real time, using the emotion analysis library emotion_detection, and the engine analyzes the user's emotional state (anxiety, surprise, etc.).

[0371] 7. A way to tailor notifications based on the emotion engine analysis results

[0372] After the emotion engine determines the user's emotional state, the server will modify the notification accordingly. For example, if the user is anxious, the notification will say something like, "This information is suspicious, but it's okay. Please check for reliable information."

[0373] Specific examples

[0374] Example 1: Fake news detection and emotional response

[0375] When a user reads an article on a news site, they take a screenshot of the article and send it to the server. The server analyzes the text and images in the screenshot to detect suspicious content. Based on this result, if the sentiment analysis engine detects an anxious expression on the user's face, a notification will be displayed saying, "This news article may be fake (90% confidence). Don't worry, there is other reliable information available."

[0376] Example 2: Deepfake video detection and emotion response

[0377] To verify the authenticity of a video sent by a friend, the user uses the application to send the video to a server. The server analyzes the video frames and audio to detect deepfake characteristics. After analyzing the user's surprised tone of voice, a notification appears saying, "This video may be a deepfake (85% confidence). It's okay, but we recommend further verification."

[0378] Prompt Sentence Examples

[0379] Analyze the video frames being watched to detect fake information. Extract text and objects in the video frames, and comprehensively evaluate all information to determine whether it is fake. Analyze the user's facial expressions and voice, and generate appropriate notifications based on the results.

[0380] By detecting fake information and providing flexible notifications that adapt to the user's emotions, the present invention creates an environment where users can safely confirm the reliability of information.

[0381] The above configuration enables the rapid and accurate detection of fake information and notifications that reduce the psychological burden on users.

[0382] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0383] Step 1:

[0384] The video being watched by the user is captured using the camera or screen recording function of the smartphone or tablet device. When the user presses a specific button, the application captures and temporarily saves video frames. The input is the video being watched, and the output is the captured video frames.

[0385] Step 2:

[0386] The terminal application inputs the captured video frames, encrypts them, and sends them to the server. The application uses the SSL / TLS protocol to ensure secure data transmission. The input is the captured video frames, and the output is the encrypted video frame data.

[0387] Step 3:

[0388] The server decrypts the received encrypted data and analyzes the video frames. A generative AI model is used for the analysis, and image analysis is performed using OpenCV. The input is the encrypted video frame data, and the output is the analysis result (e.g., "Possible fake information").

[0389] Step 4:

[0390] The server organizes the analysis results and converts them into a user-friendly format. At this time, detailed information is added according to the reliability and type of the analysis results (image, audio, text). The input is the analysis results, and the output is the organized analysis results (e.g., "Possibly fake information (reliability 90%)").

[0391] Step 5:

[0392] The server again encrypts the compiled analysis results and sends them back to the terminal, again using the SSL / TLS protocol to ensure secure data transmission. The input is the compiled analysis results, and the output is the encrypted analysis result data.

[0393] Step 6:

[0394] The device decrypts the encrypted data received and notifies the user. Notifications are made using pop-ups or messages. The input is the encrypted analysis result data, and the output is a notification message for the user.

[0395] Step 7:

[0396] The emotion engine collects the user's facial expression and voice data and performs real-time emotion analysis. The emotion_detection library is used for emotion analysis. The input is the user's facial expression and voice data, and the output is the emotion analysis result (e.g., "anxious state").

[0397] Step 8:

[0398] The server adjusts the notification content based on the analysis results of the emotion engine. For example, if the user is in an anxious state, it generates a gentle notification message. The input is the emotion analysis result and the analyzed fake information, and the output is the adjusted notification message.

[0399] Step 9:

[0400] The device displays the final notification message to the user, allowing the user to confirm the reliability of the information in a way that takes into account their emotional state. The input is the adjusted notification message, and the output is the final notification to the user.

[0401] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0402] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0403] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0404] [Second embodiment]

[0405] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0406] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0407] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0408] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0409] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0410] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0411] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0412] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0413] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0414] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0415] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0416] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0417] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[0418] System Overview

[0419] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to identify fake information. The server then sends back the organized analysis results to the device, which notifies the user in real time. This series of processes allows users to quickly and accurately verify the reliability of the information.

[0420] System Details

[0421] 1. Information acquisition means:

[0422] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0423] 2. Application:

[0424] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0425] 3. Server:

[0426] The server receives the data sent from the application. After receiving the data, it is decoded and input into a neural network model. Using deep learning technology, the server evaluates the authenticity of the information. The analysis results in a decision on whether the information is fake and its reliability.

[0427] 4. Organizational means:

[0428] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0429] 5. Means of notification:

[0430] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[0431] Specific examples

[0432] Example 1: Fake news detection

[0433] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[0434] Example 2: Deepfake video detection

[0435] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[0436] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[0437] The processing flow will be explained below.

[0438] Step 1:

[0439] The user acquires information. Specifically, the user uses the camera function of their smartphone to take photos or videos, or to take screenshots.

[0440] Step 2:

[0441] The information acquired by the device is input into the application, which then accesses data stored on the device.

[0442] Step 3:

[0443] The device sends information to the server through the application, which encrypts the information for security purposes and sends it to the server using the SSL / TLS protocol.

[0444] Step 4:

[0445] The server receives the transmitted information. First, the server decrypts the received data.

[0446] Step 5:

[0447] The server analyzes the information. The decoded data is fed into a neural network model and deep learning techniques are used to assess the authenticity of the information. For example, image analysis algorithms are applied to images, and natural language processing techniques are used to assess text.

[0448] Step 6:

[0449] The server organizes the analysis results. Specifically, it summarizes the results of its judgment on whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[0450] Step 7:

[0451] The server then sends the compiled analysis results back to the device, which are also encrypted and transmitted using the SSL / TLS protocol.

[0452] Step 8:

[0453] The terminal receives and decodes the analysis results. The terminal receives the data from the server and converts it into a format that is easy for the user to understand.

[0454] Step 9:

[0455] The device will notify the user of the results. Specifically, it will display a pop-up notification or message, such as "This news article may be fake news (90% confidence level)."

[0456] The above is the specific processing flow of this system. This series of steps allows users to quickly and accurately confirm the reliability of the information they have obtained, enabling them to use the information safely.

[0457] Example 1

[0458] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0459] In recent years, the increase in fake information has led to social unrest and increased fraud. This problem is particularly serious in today's society, where information spreads instantly via the Internet. Current methods of verifying information often lack real-time capabilities and require users to determine the authenticity of information themselves, which requires a certain level of specialized knowledge. This creates a risk that ordinary users can easily spread unreliable information.

[0460] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0461] In this invention, the server includes means for inputting acquired information into an application, encrypting it, and sending it to the server, means for the server to receive the encrypted information, decrypt it, and analyze it using a neural network, and means for the server to organize the analysis results, re-encrypt the results, and send them back to the terminal. This makes it possible for information acquired by the user to be analyzed in real time, its reliability evaluated, and the results notified immediately.

[0462] "Means of information acquisition" refers to the means by which a user acquires data such as images, text, and audio using a device such as a smartphone or computer.

[0463] An "application" is software that runs on a terminal, encrypts acquired information, and transmits it to a server.

[0464] "Encryption" is the process of transforming transmitted data so that it cannot be deciphered by third parties.

[0465] A "server" is a computer system that receives information over a network, decodes it, and analyzes it using a neural network.

[0466] "Decryption" is the process of restoring encrypted data to its original state.

[0467] A "neural network" is an artificial intelligence model that analyzes and evaluates information by mimicking the function of nerve cells in the human brain.

[0468] "Analysis" is the process of examining information in detail to find specific features or patterns.

[0469] "Organization" is the process of converting the analysis results obtained within the server into a format that is easy for users to understand.

[0470] A "terminal" is a device that is directly operated by a user and has the role of obtaining and notifying information.

[0471] "Notification" refers to the means by which the device displays the analysis results to the user, and includes methods such as pop-up notifications and message displays.

[0472] "Real-time" refers to processes that are executed immediately, without delay.

[0473] "Reliability" is an indicator of the veracity and accuracy of information obtained through analysis.

[0474] "Multi-layered" means that the analysis is divided into multiple elements or layers, and refers to the simultaneous evaluation of different data formats such as images, audio, and text.

[0475] "Fake information" is data that contains intentionally misleading or false information.

[0476] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[0477] System Overview

[0478] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to determine whether it is fake or not. The server then sends back the organized analysis results to the device, which notifies the user in real time. This process allows users to quickly and accurately verify the reliability of the information.

[0479] Information acquisition means

[0480] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0481] application

[0482] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0483] server

[0484] The server receives the data sent from the application. After receiving the data, it is decrypted and input into a neural network model. Using deep learning technology, the server analyzes the authenticity of the information. The analysis results in a judgment of whether the information is fake and its reliability.

[0485] means of organization

[0486] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0487] Notification means

[0488] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[0489] Specific examples

[0490] Example 1: Fake news detection

[0491] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[0492] Example 2: Deepfake video detection

[0493] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[0494] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[0495] Prompt Sentence Examples

[0496] Please explain the process flow of the system program in detail using the following steps. The subject should be either the server, terminal, or user.

[0497] procedure:

[0498] 1. The user obtains the information.

[0499] 2. The device passes the information to the application.

[0500] 3. The application encrypts the information and sends it to the server.

[0501] 4. The server receives and decrypts the information.

[0502] 5. The server inputs the information into a neural network model for analysis.

[0503] 6. The server organizes the analysis results.

[0504] 7. The server encrypts the results and sends them to the device.

[0505] 8. The device receives the results and notifies the user.

[0506] Examples:

[0507] Users take screenshots of news articles and save them to their devices. The application encrypts them and sends them to a server. The server analyzes the screenshots and notifies the user if they are deemed to be fake news.

[0508] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0509] Step 1:

[0510] The user acquires information. The user acquires image or text information using the smartphone's camera or screenshot function. For example, the user views an article on a news site and takes a screenshot of it. The input data is the image or text. The output is the acquired information stored on the device.

[0511] Step 2:

[0512] The device passes information to the application. The device passes the captured image and text information to the application, which then retrieves it. During this process, screenshots stored on the device are imported into the application. Input data is information stored on the device, and output data is information imported into the application.

[0513] Step 3:

[0514] The application encrypts information and sends it to the server. The application encrypts information using the SSL / TLS protocol and sends it to the server. The input data here is the unencrypted information imported into the application, and the output data is the encrypted data sent to the server.

[0515] Step 4:

[0516] The server receives and decrypts the information. The server receives and decrypts the encrypted data. Inside the server, the data is decrypted using the AES algorithm. The input data is the encrypted information, and the output data is the decrypted information.

[0517] Step 5:

[0518] The server inputs the information into a neural network model for analysis. Using deep learning technology, the server inputs the decoded information into the neural network model for analysis. For example, the text of a news article is analyzed using the BERT model, and the image is analyzed using the ResNet model. The input data is the decoded information, and the output data is the analysis result.

[0519] Step 6:

[0520] The server organizes the analysis results. The server converts the analysis results into a user-friendly format and assigns a reliability rating. For example, the server might say, "This article may be fake news (90% reliability)." The input data is the analysis results, and the output data is the organized results.

[0521] Step 7:

[0522] The server encrypts the organized results and sends them to the terminal. The server re-encrypts the organized results and sends them to the terminal. The input data is the organized results, and the output data is the encrypted results.

[0523] Step 8:

[0524] The device receives the results and notifies the user. The device receives the encrypted data sent from the server and decrypts it. It then displays a pop-up notification or message to the user. For example, it displays a notification saying, "This article may be fake news (90% confidence level)." The input data is the encrypted result, and the output data is the result notified to the user.

[0525] (Application example 1)

[0526] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0527] The widespread use of the Internet has led to the rapid spread of harmful information, such as false information, fake news, and phishing links. This has led to many users becoming victims of fraud and experiencing social unrest due to misinformation. However, current systems lack a way to detect this fake information in real time and promptly notify users. In particular, there is a need for a means to instantly evaluate the reliability of links and attachments in electronic messages.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0529] In this invention, the server includes means for acquiring information about links and attachments in electronic messages in real time, means for determining the authenticity of links and attachments using a neural network, and means for notifying the user of the authenticity information, which allows the user to evaluate the reliability of links and attachments in emails and messages in real time and prevent damage from fake information and phishing sites.

[0530] "Information acquisition means" refers to the function of a device or software that collects information (links, attachments, images, audio, text, etc.) that a user acquires.

[0531] An "application" is a program that sends acquired information to a server and analyzes and organizes the results.

[0532] A "server" is a computer system that receives information sent from an application and analyzes it using a neural network.

[0533] A "neural network" is a machine learning model that uses multiple layers of artificial neurons, and is a technology used to determine the authenticity of information.

[0534] The "organizing means" is a process in which the server converts the analysis results into a format that is easy for the user to understand and returns it to the terminal.

[0535] "Notification means" is a function that notifies the user of the organized results as a pop-up notification or message display.

[0536] "Links and attachments in electronic messages" are URL links and files contained in email or messaging applications.

[0537] "Real-time retrieval" refers to the ability to retrieve links and attachments in electronic messages immediately upon their arrival.

[0538] The "means for determining authenticity" is a function that uses a neural network to analyze acquired information and determine whether the information is fake.

[0539] "Means of notifying including reliability information" is a function that informs the user of the analysis results and their reliability (e.g., reliability 85%).

[0540] The present invention is a system that allows users to evaluate the reliability of links and attachments in electronic messages in real time, quickly determining whether the information is fake and notifying users. This system uses a smartphone, a server, and an application that links them.

[0541] System Overview

[0542] Users use an application installed on their smartphone to retrieve links and attachments from electronic messages and emails. This information is encrypted and sent to a server via the application. The server analyzes the information using a neural network model and notifies the user of the results in real time.

[0543] System Details

[0544] 1. Information acquisition means:

[0545] After users retrieve links and attachments in electronic messages and emails, the information is automatically collected by the application, which uses a module that retrieves information from the API of the smartphone's default email and messaging apps.

[0546] 2. Application:

[0547] The application receives the acquired information as input and encrypts the data, using the SSL / TLS protocol to ensure secure communication. The encrypted data is then sent to the server.

[0548] 3. Server:

[0549] The server is installed on AWS EC2 or Google Cloud Platform. The server decrypts the received data and analyzes it using a neural network model using PyTorch or TensorFlow. This analysis determines the authenticity of links and attachments and evaluates their reliability as a numerical value (e.g., 85% reliability).

[0550] 4. Organizational means:

[0551] The server then organizes the analysis results and converts them into a user-friendly format. For example, the result may be displayed as "This link may be a phishing site (confidence 85%)." The confidence level is also displayed, allowing users to trust the results.

[0552] 5. Means of notification:

[0553] The results are then encrypted and sent back to the device, where the smartphone application receives them and displays a pop-up notification or message to the user.

[0554] Specific examples

[0555] Example 1: Evaluating the trustworthiness of links in emails

[0556] When a user receives an email and tries to click on a link, the application automatically captures the link and sends it to the server, which analyzes the link's reliability and sends a notification to the user saying, "This link may be a phishing site (85% confidence)."

[0557] Example 2: Authentication of attached files

[0558] To verify the authenticity of an attachment received by a user via a messaging app, the user uses the application to send the file to a server, which analyzes the contents of the file and notifies the user that "This file may contain a virus (90% confidence level)."

[0559] Example prompts to input to the generative AI model

[0560] Generative AI model, please analyze whether the following link is fake: example_link. Print the result and confidence level.

[0561] This embodiment allows users to quickly evaluate the reliability of links and attachments in electronic messages and emails, and can prevent damage from fake information and phishing sites.

[0562] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0563] Step 1:

[0564] A user receives an email or message and retrieves a link or attachment from it. An application automatically detects the link or attachment information. The input here is the link or attachment from the electronic message, and the output is the raw data passed to the application.

[0565] Step 2:

[0566] The information obtained by the application is encrypted using the SSL / TLS protocol. The input is raw data and the output is encrypted data. Encryption ensures the protection of transmitted data.

[0567] Step 3:

[0568] The terminal sends encrypted data to the server. The input is the encrypted data, and the output is the received data that is recorded in the server's reception log. This transmission uses Internet communication.

[0569] Step 4:

[0570] The server receives the encrypted data and decrypts it using the SSL / TLS protocol. The input is the encrypted data and the output is the original, decrypted raw data.

[0571] Step 5:

[0572] The server uses PyTorch or TensorFlow to analyze the authenticity of links and attachments using a neural network model. The input is the decoded raw data, and the output is the analysis result (evaluation of whether the information is fake or not) and its reliability score. Specific operations include pattern recognition, feature extraction, and evaluation of links and files.

[0573] Step 6:

[0574] The server organizes the analysis results and converts them into a user-friendly format. The input is the analysis results and confidence score, and the output is a user notification such as "This link may be a phishing site (confidence 85%)."

[0575] Step 7:

[0576] The server re-encrypts the result and sends it to the terminal. The input is the result, and the output is the encrypted result data.

[0577] Step 8:

[0578] The terminal receives the encrypted result data and decrypts it again. The input is the encrypted result data, and the output is the decrypted notification data.

[0579] Step 9:

[0580] The application notifies the user of the notification data as a popup notification or message display. The input is the decoded notification data, and the output is the message or notification that is displayed on the user's device screen.

[0581] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0582] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users by combining it with an emotion engine to provide appropriate notifications that take user emotions into consideration. This system is realized by the user's device, a server, an application, and an emotion engine.

[0583] System Overview

[0584] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[0585] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[0586] System Details

[0587] 1. Information acquisition means:

[0588] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0589] 2. Application:

[0590] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0591] 3. Server:

[0592] The server receives data sent from the application. First, the server decrypts the received data. Next, it inputs the decrypted data into a neural network model and uses deep learning technology to evaluate the authenticity of the information. For example, in the case of images, it applies an image analysis algorithm, and in the case of text, it uses natural language processing technology.

[0593] 4. Organizational means:

[0594] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0595] 5. Emotion Engine Processing:

[0596] The emotion engine analyzes the user's facial expressions and voice data in real time, thereby determining the user's current emotional state and sending it to the server.

[0597] 6. Sentiment data analysis on the server:

[0598] The server then combines the received emotional data with the results of its fake information analysis to generate notification content that reflects the user's emotions. For example, if the server determines that the user is feeling extremely anxious, it will adjust the notification content to be gentler.

[0599] 7. Means of notification:

[0600] The sorted results are then encrypted again and sent back to the device. After receiving the results, the device displays a pop-up notification or message to the user, instantly informing them of the reliability of the information. Appropriate notification content is also displayed based on the data from the emotion engine.

[0601] Specific examples

[0602] Example 1: Fake news detection and emotional response

[0603] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, the emotion engine analyzes the user's facial expression. If the user looks anxious, the system displays a message such as, "This news article may be fake news (90% confidence). Don't worry, you can also find other reliable information."

[0604] Example 2: Deepfake video detection and emotion response

[0605] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server then performs multi-layered analysis of the video's frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)." Furthermore, if the emotion engine analyzes the user's tone of voice and determines that the user is surprised, a message similar to, "This video is suspicious, but it's okay. We recommend further verification." is displayed.

[0606] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[0607] The processing flow will be explained below.

[0608] Step 1:

[0609] A user acquires information. Specifically, the user uses the camera function of their smartphone to take a photo or video, or to take a screenshot of a news article.

[0610] Step 2:

[0611] The information acquired by the device is input to the application, which then accesses the data stored on the device and prepares it for analysis.

[0612] Step 3:

[0613] The device sends information to the server through the application, which is then encrypted for security purposes and sent to the server using the SSL / TLS protocol.

[0614] Step 4:

[0615] The server receives the transmitted information and first decrypts the received data.

[0616] Step 5:

[0617] The server analyzes the information. The received and decoded data is fed into a neural network model, which uses deep learning techniques to evaluate the authenticity of the information. For example, if it is an image, an image analysis algorithm is applied, and if it is text, natural language processing technology is used for analysis.

[0618] Step 6:

[0619] The server organizes the analysis results. It summarizes the judgment result regarding whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[0620] Step 7:

[0621] The device recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice in real time to understand their current emotional state.

[0622] Step 8:

[0623] The device sends emotional data to the server, which is also encrypted and sent to the server using the SSL / TLS protocol.

[0624] Step 9:

[0625] The server receives and analyzes the emotion data, decodes the received data, and evaluates the user's emotional state.

[0626] Step 10:

[0627] The server integrates the emotion data and the results of the information analysis. It adjusts the notification content based on the user's emotion and returns it to the device in the most appropriate format. For example, if the user is feeling anxious, the notification will be delivered in a softer tone.

[0628] Step 11:

[0629] The server encrypts the final notification and sends it to the device, again using the SSL / TLS protocol to send the data securely.

[0630] Step 12:

[0631] The device receives and decrypts the final notification. The device receives and decrypts the data sent from the server.

[0632] Step 13:

[0633] The device will notify the user of the results. Specifically, it will inform the user of the notification content through a pop-up notification or message display. For example, a tailored notification such as "This news article may be fake news (90% confidence level). Rest assured, there are other reliable sources available." will be displayed.

[0634] This completes the process flow of the system. This series of steps allows users to quickly and accurately check the reliability of the information they have acquired, while also receiving appropriate notifications based on their emotional state.

[0635] Example 2

[0636] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] In modern society, there is a need to quickly and accurately evaluate the reliability of information on the Internet. However, existing technologies lack the accuracy to detect fake information, which creates the risk that users will believe false information. Furthermore, there is a lack of technology that can adjust notification content taking into account the user's emotional state, which hinders efforts to improve the user experience.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0639] In this invention, the server includes a means for using a neural network to evaluate the reliability of information, a means for using an emotion engine that analyzes the user's facial expressions and voice data, and a means for integrating the emotion data and the analysis results to adjust the notification content, thereby enabling highly accurate detection of fake information and flexible notification that takes the user's emotional state into consideration.

[0640] "Means of information acquisition" refers to the means by which users acquire information such as photos, videos, and news articles using smartphones or other devices.

[0641] "Application" means software that processes the acquired information on the user's device and transmits it to the server.

[0642] The "server" is a central component of the system that analyzes the received information using a neural network, organizes the results, and returns them to the terminal.

[0643] A "neural network" is a mathematical model that uses artificial intelligence techniques to evaluate the reliability of information.

[0644] The "emotion engine" is a component of the system that analyzes the user's facial expressions and voice data to determine their emotional state.

[0645] "Notification means" is a function for informing the user of the sorting results sent from the server, and includes pop-up notifications and messages displayed on the terminal.

[0646] MODE FOR CARRYING OUT THE INVENTION

[0647] The present invention provides a system for detecting and notifying fake information that takes into account user emotions. This system is realized by a user terminal, a server, an application, and an emotion engine. Detailed embodiments of the system are described below.

[0648] System configuration

[0649] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[0650] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[0651] Hardware and Software Used

[0652] The system uses the following hardware and software:

[0653] 1. Information acquisition means:

[0654] Users capture information using their smartphone's camera, screenshot function, or existing media files.

[0655] Information is stored on the device and accessed by applications.

[0656] 2. Application:

[0657] The acquired information is taken as input, and once the user initiates the analysis using the application, the information is encrypted and sent to the server.

[0658] The SSL / TLS protocol is used to ensure secure communications.

[0659] 3. Server:

[0660] The server decodes the received data and inputs it into a neural network model (e.g., PyTorch or TensorFlow) to evaluate the authenticity of the information.

[0661] For images, image analysis algorithms are applied, and for text, natural language processing techniques are used.

[0662] 4. Emotion Engine:

[0663] The device collects facial and voice data from the camera and microphone and analyzes it in real time.

[0664] An emotion engine (e.g., OpenFace or IBM Watson) determines the user's emotional state.

[0665] 5. Organizational means:

[0666] The server organizes the analysis results in a format such as "High probability of fake news (95% confidence)" and integrates it with data from the emotion engine to adjust the content of the notification.

[0667] 6. Means of notification:

[0668] The server re-encrypts the results and sends them back to the terminal.

[0669] After the device receives the results, it will display a pop-up notification or message to the user to inform them of the reliability of the information.

[0670] Specific examples

[0671] Example 1: Fake news detection and emotional response

[0672] 1. Information acquisition

[0673] When a user reads an article on a news site, they take a screenshot of the article and send it to a server through the application.

[0674] 2. Data Analysis

[0675] The server analyzes the text and images in the screenshot and determines that the content is likely to be fake news.

[0676] 3. Sentiment analysis

[0677] The emotion engine analyzes the user's facial expressions, and if the user looks anxious, the system will notify them with a message such as, "This news article may be fake (90% confidence). Don't worry, there are other reliable sources available."

[0678] Example 2: Deepfake video detection and emotion response

[0679] 1. Information acquisition

[0680] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server.

[0681] 2. Data Analysis

[0682] The server performs multi-layered analysis of video frames and audio to detect deepfake characteristics.

[0683] 3. Sentiment analysis

[0684] If the emotion engine analyzes the user's tone of voice and determines that the user is surprised, it will display a message such as, "This video is suspicious, but it's okay. We recommend you check it out further."

[0685] Prompt Sentence Examples

[0686] To the generative AI model: Please provide an example prompt that uses a user-generated screenshot of a news article to detect fake news.

[0687] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[0688] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0689] System program processing flow

[0690] Step 1: Information Acquisition

[0691] A user uses the camera function or screenshot function of their smartphone to obtain information (photos, videos, news articles, etc.).

[0692] Input: Information from social media, news sites, etc.

[0693] What happens: A user taps the camera icon on their smartphone to launch the camera app and take a screenshot of a suspicious news article.

[0694] Output: Information stored on the device (image files, etc.)

[0695] Step 2: Send data

[0696] The information acquired by the terminal is sent to the server via the application.

[0697] Input: Information stored on the device (image files, etc.)

[0698] What it does: When you tap the "Start Analysis" button in the app, the screenshot is encrypted and uploaded to the server, using the SSL / TLS protocol to ensure secure communication.

[0699] Output: Encrypted information (such as an image file) is sent to the server

[0700] Step 3: Data analysis

[0701] The server decodes the received information and analyzes it using a neural network model.

[0702] Input: Encrypted and transmitted information (such as image files)

[0703] What it does: The server decodes the received screenshot and evaluates its content using an image analysis algorithm (for example, a model using PyTorch or TensorFlow). If text is included, it extracts it using OCR (Optical Character Recognition) technology and analyzes the text using natural language processing technology.

[0704] Output: Evaluation result of whether the information is fake or not and the confidence level

[0705] Step 4: Sentiment analysis

[0706] The device uses a camera and microphone to collect the user's facial expressions and voice data, which is then analyzed in real time by an emotion engine.

[0707] Input: User's facial expression data and speech sound data

[0708] How it works: When a user looks into the smartphone camera, the camera captures their facial expressions and the microphone collects audio data. An emotion engine (e.g., OpenFace or IBM Watson) analyzes this data and estimates the user's emotional state.

[0709] Output: Evaluation of the user's emotional state

[0710] Step 5: Organize the results

[0711] The server organizes the analysis results and converts them into a format that is easy for the user to understand.

[0712] Input: Evaluation results of fake information and evaluation results of user's emotional state

[0713] How it works: The server generates results in the form of "Highly likely to be fake news (95% confidence)" and integrates information and emotional data. The tone and content of the notification message are also adjusted depending on the user's emotional state.

[0714] Output: Organized results (information message)

[0715] Step 6: Send results

[0716] The server re-encrypts the organized results and sends them to the terminal.

[0717] Input: Organized results (notification message)

[0718] How it works: The server encrypts the results again using SSL / TLS and sends them to the device. The notification contains the fake news analysis results and sentiment-based content.

[0719] Output: Encrypted notification message

[0720] Step 7: Notifications

[0721] The terminal receives the results and displays a pop-up notification or message to the user.

[0722] Input: Encrypted notification message

[0723] What it does: The device will display the results it receives from the server, with a pop-up notification on the screen. For example, it might say, "This news article may be fake (90% confidence level). Don't worry, there are other reliable sources available."

[0724] Output: A notification message that is displayed to the user

[0725] Through these steps, the system can analyze the reliability of the information obtained by the user with high accuracy and provide flexible notifications according to the user's emotional state.

[0726] (Application example 2)

[0727] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] Current fake information detection systems can instantly assess the authenticity of information, but lack the ability to adjust notification content based on the user's emotional state. This means that if a user feels uneasy or surprised about the reliability of the information, they are unable to respond appropriately, increasing the user's psychological burden. Furthermore, their multi-layered analysis of each type of information is insufficient, limiting the accuracy of their fake information detection.

[0729] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information using a neural network, means for organizing the analysis results and returning them to the terminal, and means having an emotion engine for analyzing the user's facial expressions and voice. This makes it possible to evaluate the reliability of information in real time and adjust the notification content according to the user's emotional state. In addition, multi-layered analysis according to the type of information, such as images, voice, and text, improves the accuracy of detecting fake information.

[0730] "Means of information acquisition" refers to the means by which a user acquires information such as images, audio, and text using a smartphone or other device.

[0731] "Means for inputting information into an application and transmitting it to a server" refers to means for inputting acquired information into an application, encrypting it, and transmitting it securely to a server.

[0732] "Means for analyzing information using neural networks" refers to means for analyzing information obtained on a server using deep learning technology and evaluating its authenticity.

[0733] The "means for organizing the analysis results and returning them to the terminal" refers to a means for converting the results analyzed by the server into a format that is easy for the user to understand and returning them to the terminal for notification.

[0734] "Means for notifying the user of the organized results" refers to means for informing the user of the analysis results using a pop-up notification or message display on the terminal.

[0735] "Means having an emotion engine that analyzes the user's facial expressions and voice" refers to means that uses the user's camera and microphone to analyze the user's facial expressions and voice in real time and recognize the user's emotional state.

[0736] The "means for adjusting the notification content based on the analysis results of the emotion engine" refers to a means for appropriately changing the notification content based on the analysis results in accordance with the emotional state of the user.

[0737] The present invention is a system that analyzes information about videos watched by users in real time, detects fake information, and notifies users appropriately based on their emotional state. This system includes the following elements:

[0738] System configuration and program processing

[0739] 1. Information acquisition means

[0740] Devices such as smartphones and tablets capture frames of the video a user is watching using their camera or screen recording capabilities.

[0741] 2. A means of inputting data into the application and sending it to the server

[0742] The captured video frames are input into a dedicated application, which then encrypts the frames and securely transmits them to the server. The communication between the client and server is secured using the SSL / TLS protocol.

[0743] 3. How to analyze information using neural networks

[0744] The server analyzes the received video frames using deep learning techniques, using OpenCV for image analysis, an NLP (natural language processing) library for text analysis, and a speech recognition library for audio analysis.

[0745] 4. A means of organizing the analysis results and returning them to the device

[0746] The server compiles the analysis results and returns them to the device in an easy-to-understand format, such as "Possibly fake information (confidence level XX%)." This result is also sent encrypted.

[0747] 5. A method for notifying users of the organized results

[0748] The analysis results received by the device are immediately communicated to the user via pop-up notifications or message displays, with the content of the notifications tailored based on the user's emotional state.

[0749] 6. Means for providing an emotion engine that analyzes the user's facial expressions and voice

[0750] The system uses the smartphone's camera and microphone to collect the user's facial expressions and voice in real time, using the emotion analysis library emotion_detection, and the engine analyzes the user's emotional state (anxiety, surprise, etc.).

[0751] 7. A way to tailor notifications based on the emotion engine analysis results

[0752] After the emotion engine determines the user's emotional state, the server will modify the notification accordingly. For example, if the user is anxious, the notification will say something like, "This information is suspicious, but it's okay. Please check for reliable information."

[0753] Specific examples

[0754] Example 1: Fake news detection and emotional response

[0755] When a user reads an article on a news site, they take a screenshot of the article and send it to the server. The server analyzes the text and images in the screenshot to detect suspicious content. Based on this result, if the sentiment analysis engine detects an anxious expression on the user's face, a notification will be displayed saying, "This news article may be fake (90% confidence). Don't worry, there is other reliable information available."

[0756] Example 2: Deepfake video detection and emotion response

[0757] To verify the authenticity of a video sent by a friend, the user uses the application to send the video to a server. The server analyzes the video frames and audio to detect deepfake characteristics. After analyzing the user's surprised tone of voice, a notification appears saying, "This video may be a deepfake (85% confidence). It's okay, but we recommend further verification."

[0758] Prompt Sentence Examples

[0759] Analyze the video frames being watched to detect fake information. Extract text and objects in the video frames, and comprehensively evaluate all information to determine whether it is fake. Analyze the user's facial expressions and voice, and generate appropriate notifications based on the results.

[0760] By detecting fake information and providing flexible notifications that adapt to the user's emotions, the present invention creates an environment where users can safely confirm the reliability of information.

[0761] The above configuration enables the rapid and accurate detection of fake information and notifications that reduce the psychological burden on users.

[0762] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0763] Step 1:

[0764] The video being watched by the user is captured using the camera or screen recording function of the smartphone or tablet device. When the user presses a specific button, the application captures and temporarily saves video frames. The input is the video being watched, and the output is the captured video frames.

[0765] Step 2:

[0766] The terminal application inputs the captured video frames, encrypts them, and sends them to the server. The application uses the SSL / TLS protocol to ensure secure data transmission. The input is the captured video frames, and the output is the encrypted video frame data.

[0767] Step 3:

[0768] The server decrypts the received encrypted data and analyzes the video frames. A generative AI model is used for the analysis, and image analysis is performed using OpenCV. The input is the encrypted video frame data, and the output is the analysis result (e.g., "Possible fake information").

[0769] Step 4:

[0770] The server organizes the analysis results and converts them into a user-friendly format. At this time, detailed information is added according to the reliability and type of the analysis results (image, audio, text). The input is the analysis results, and the output is the organized analysis results (e.g., "Possibly fake information (reliability 90%)").

[0771] Step 5:

[0772] The server again encrypts the compiled analysis results and sends them back to the terminal, again using the SSL / TLS protocol to ensure secure data transmission. The input is the compiled analysis results, and the output is the encrypted analysis result data.

[0773] Step 6:

[0774] The device decrypts the encrypted data received and notifies the user. Notifications are made using pop-ups or messages. The input is the encrypted analysis result data, and the output is a notification message for the user.

[0775] Step 7:

[0776] The emotion engine collects the user's facial expression and voice data and performs real-time emotion analysis. The emotion_detection library is used for emotion analysis. The input is the user's facial expression and voice data, and the output is the emotion analysis result (e.g., "anxious state").

[0777] Step 8:

[0778] The server adjusts the notification content based on the analysis results of the emotion engine. For example, if the user is in an anxious state, it generates a gentle notification message. The input is the emotion analysis result and the analyzed fake information, and the output is the adjusted notification message.

[0779] Step 9:

[0780] The device displays the final notification message to the user, allowing the user to confirm the reliability of the information in a way that takes into account their emotional state. The input is the adjusted notification message, and the output is the final notification to the user.

[0781] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0782] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0783] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0784] [Third embodiment]

[0785] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0786] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0787] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0788] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0789] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0790] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0791] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0792] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0793] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0794] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0795] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0796] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0797] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[0798] System Overview

[0799] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to identify fake information. The server then sends back the organized analysis results to the device, which notifies the user in real time. This series of processes allows users to quickly and accurately verify the reliability of the information.

[0800] System Details

[0801] 1. Information acquisition means:

[0802] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0803] 2. Application:

[0804] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0805] 3. Server:

[0806] The server receives the data sent from the application. After receiving the data, it is decoded and input into a neural network model. Using deep learning technology, the server evaluates the authenticity of the information. The analysis results in a decision on whether the information is fake and its reliability.

[0807] 4. Organizational means:

[0808] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0809] 5. Means of notification:

[0810] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[0811] Specific examples

[0812] Example 1: Fake news detection

[0813] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[0814] Example 2: Deepfake video detection

[0815] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[0816] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[0817] The processing flow will be explained below.

[0818] Step 1:

[0819] The user acquires information. Specifically, the user uses the camera function of their smartphone to take photos or videos, or to take screenshots.

[0820] Step 2:

[0821] The information acquired by the device is input into the application, which then accesses data stored on the device.

[0822] Step 3:

[0823] The device sends information to the server through the application, which encrypts the information for security purposes and sends it to the server using the SSL / TLS protocol.

[0824] Step 4:

[0825] The server receives the transmitted information. First, the server decrypts the received data.

[0826] Step 5:

[0827] The server analyzes the information. The decoded data is fed into a neural network model and deep learning techniques are used to assess the authenticity of the information. For example, image analysis algorithms are applied to images, and natural language processing techniques are used to assess text.

[0828] Step 6:

[0829] The server organizes the analysis results. Specifically, it summarizes the results of its judgment on whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[0830] Step 7:

[0831] The server then sends the compiled analysis results back to the device, which are also encrypted and transmitted using the SSL / TLS protocol.

[0832] Step 8:

[0833] The terminal receives and decodes the analysis results. The terminal receives the data from the server and converts it into a format that is easy for the user to understand.

[0834] Step 9:

[0835] The device will notify the user of the results. Specifically, it will display a pop-up notification or message, such as "This news article may be fake news (90% confidence level)."

[0836] The above is the specific processing flow of this system. This series of steps allows users to quickly and accurately confirm the reliability of the information they have obtained, enabling them to use the information safely.

[0837] Example 1

[0838] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0839] In recent years, the increase in fake information has led to social unrest and increased fraud. This problem is particularly serious in today's society, where information spreads instantly via the Internet. Current methods of verifying information often lack real-time capabilities and require users to determine the authenticity of information themselves, which requires a certain level of specialized knowledge. This creates a risk that ordinary users can easily spread unreliable information.

[0840] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0841] In this invention, the server includes means for inputting acquired information into an application, encrypting it, and sending it to the server, means for the server to receive the encrypted information, decrypt it, and analyze it using a neural network, and means for the server to organize the analysis results, re-encrypt the results, and send them back to the terminal. This makes it possible for information acquired by the user to be analyzed in real time, its reliability evaluated, and the results notified immediately.

[0842] "Means of information acquisition" refers to the means by which a user acquires data such as images, text, and audio using a device such as a smartphone or computer.

[0843] An "application" is software that runs on a terminal, encrypts acquired information, and transmits it to a server.

[0844] "Encryption" is the process of transforming transmitted data so that it cannot be deciphered by third parties.

[0845] A "server" is a computer system that receives information over a network, decodes it, and analyzes it using a neural network.

[0846] "Decryption" is the process of restoring encrypted data to its original state.

[0847] A "neural network" is an artificial intelligence model that analyzes and evaluates information by mimicking the function of nerve cells in the human brain.

[0848] "Analysis" is the process of examining information in detail to find specific features or patterns.

[0849] "Organization" is the process of converting the analysis results obtained within the server into a format that is easy for users to understand.

[0850] A "terminal" is a device that is directly operated by a user and has the role of obtaining and notifying information.

[0851] "Notification" refers to the means by which the device displays the analysis results to the user, and includes methods such as pop-up notifications and message displays.

[0852] "Real-time" refers to processes that are executed immediately, without delay.

[0853] "Reliability" is an indicator of the veracity and accuracy of information obtained through analysis.

[0854] "Multi-layered" means that the analysis is divided into multiple elements or layers, and refers to the simultaneous evaluation of different data formats such as images, audio, and text.

[0855] "Fake information" is data that contains intentionally misleading or false information.

[0856] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[0857] System Overview

[0858] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to determine whether it is fake or not. The server then sends back the organized analysis results to the device, which notifies the user in real time. This process allows users to quickly and accurately verify the reliability of the information.

[0859] Information acquisition means

[0860] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0861] application

[0862] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0863] server

[0864] The server receives the data sent from the application. After receiving the data, it is decrypted and input into a neural network model. Using deep learning technology, the server analyzes the authenticity of the information. The analysis results in a judgment of whether the information is fake and its reliability.

[0865] means of organization

[0866] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0867] Notification means

[0868] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[0869] Specific examples

[0870] Example 1: Fake news detection

[0871] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[0872] Example 2: Deepfake video detection

[0873] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[0874] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[0875] Prompt Sentence Examples

[0876] Please explain the process flow of the system program in detail using the following steps. The subject should be either the server, terminal, or user.

[0877] procedure:

[0878] 1. The user obtains the information.

[0879] 2. The device passes the information to the application.

[0880] 3. The application encrypts the information and sends it to the server.

[0881] 4. The server receives and decrypts the information.

[0882] 5. The server inputs the information into a neural network model for analysis.

[0883] 6. The server organizes the analysis results.

[0884] 7. The server encrypts the results and sends them to the device.

[0885] 8. The device receives the results and notifies the user.

[0886] Examples:

[0887] Users take screenshots of news articles and save them to their devices. The application encrypts them and sends them to a server. The server analyzes the screenshots and notifies the user if they are deemed to be fake news.

[0888] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0889] Step 1:

[0890] The user acquires information. The user acquires image or text information using the smartphone's camera or screenshot function. For example, the user views an article on a news site and takes a screenshot of it. The input data is the image or text. The output is the acquired information stored on the device.

[0891] Step 2:

[0892] The device passes information to the application. The device passes the captured image and text information to the application, which then retrieves it. During this process, screenshots stored on the device are imported into the application. Input data is information stored on the device, and output data is information imported into the application.

[0893] Step 3:

[0894] The application encrypts information and sends it to the server. The application encrypts information using the SSL / TLS protocol and sends it to the server. The input data here is the unencrypted information imported into the application, and the output data is the encrypted data sent to the server.

[0895] Step 4:

[0896] The server receives and decrypts the information. The server receives and decrypts the encrypted data. Inside the server, the data is decrypted using the AES algorithm. The input data is the encrypted information, and the output data is the decrypted information.

[0897] Step 5:

[0898] The server inputs the information into a neural network model for analysis. Using deep learning technology, the server inputs the decoded information into the neural network model for analysis. For example, the text of a news article is analyzed using the BERT model, and the image is analyzed using the ResNet model. The input data is the decoded information, and the output data is the analysis result.

[0899] Step 6:

[0900] The server organizes the analysis results. The server converts the analysis results into a user-friendly format and assigns a reliability rating. For example, the server might say, "This article may be fake news (90% reliability)." The input data is the analysis results, and the output data is the organized results.

[0901] Step 7:

[0902] The server encrypts the organized results and sends them to the terminal. The server re-encrypts the organized results and sends them to the terminal. The input data is the organized results, and the output data is the encrypted results.

[0903] Step 8:

[0904] The device receives the results and notifies the user. The device receives the encrypted data sent from the server and decrypts it. It then displays a pop-up notification or message to the user. For example, it displays a notification saying, "This article may be fake news (90% confidence level)." The input data is the encrypted result, and the output data is the result notified to the user.

[0905] (Application example 1)

[0906] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0907] The widespread use of the Internet has led to the rapid spread of harmful information, such as false information, fake news, and phishing links. This has led to many users becoming victims of fraud and experiencing social unrest due to misinformation. However, current systems lack a way to detect this fake information in real time and promptly notify users. In particular, there is a need for a means to instantly evaluate the reliability of links and attachments in electronic messages.

[0908] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0909] In this invention, the server includes means for acquiring information about links and attachments in electronic messages in real time, means for determining the authenticity of links and attachments using a neural network, and means for notifying the user of the authenticity information, which allows the user to evaluate the reliability of links and attachments in emails and messages in real time and prevent damage from fake information and phishing sites.

[0910] "Information acquisition means" refers to the function of a device or software that collects information (links, attachments, images, audio, text, etc.) that a user acquires.

[0911] An "application" is a program that sends acquired information to a server and analyzes and organizes the results.

[0912] A "server" is a computer system that receives information sent from an application and analyzes it using a neural network.

[0913] A "neural network" is a machine learning model that uses multiple layers of artificial neurons, and is a technology used to determine the authenticity of information.

[0914] The "organizing means" is a process in which the server converts the analysis results into a format that is easy for the user to understand and returns it to the terminal.

[0915] "Notification means" is a function that notifies the user of the organized results as a pop-up notification or message display.

[0916] "Links and attachments in electronic messages" are URL links and files contained in email or messaging applications.

[0917] "Real-time retrieval" refers to the ability to retrieve links and attachments in electronic messages immediately upon their arrival.

[0918] The "means for determining authenticity" is a function that uses a neural network to analyze acquired information and determine whether the information is fake.

[0919] "Means of notifying including reliability information" is a function that informs the user of the analysis results and their reliability (e.g., reliability 85%).

[0920] The present invention is a system that allows users to evaluate the reliability of links and attachments in electronic messages in real time, quickly determining whether the information is fake and notifying users. This system uses a smartphone, a server, and an application that links them.

[0921] System Overview

[0922] Users use an application installed on their smartphone to retrieve links and attachments from electronic messages and emails. This information is encrypted and sent to a server via the application. The server analyzes the information using a neural network model and notifies the user of the results in real time.

[0923] System Details

[0924] 1. Information acquisition means:

[0925] After users retrieve links and attachments in electronic messages and emails, the information is automatically collected by the application, which uses a module that retrieves information from the API of the smartphone's default email and messaging apps.

[0926] 2. Application:

[0927] The application receives the acquired information as input and encrypts the data, using the SSL / TLS protocol to ensure secure communication. The encrypted data is then sent to the server.

[0928] 3. Server:

[0929] The server is installed on AWS EC2 or Google Cloud Platform. The server decrypts the received data and analyzes it using a neural network model using PyTorch or TensorFlow. This analysis determines the authenticity of links and attachments and evaluates their reliability as a numerical value (e.g., 85% reliability).

[0930] 4. Organizational means:

[0931] The server then organizes the analysis results and converts them into a user-friendly format. For example, the result may be displayed as "This link may be a phishing site (confidence 85%)." The confidence level is also displayed, allowing users to trust the results.

[0932] 5. Means of notification:

[0933] The results are then encrypted and sent back to the device, where the smartphone application receives them and displays a pop-up notification or message to the user.

[0934] Specific examples

[0935] Example 1: Evaluating the trustworthiness of links in emails

[0936] When a user receives an email and tries to click on a link, the application automatically captures the link and sends it to the server, which analyzes the link's reliability and sends a notification to the user saying, "This link may be a phishing site (85% confidence)."

[0937] Example 2: Authentication of attached files

[0938] To verify the authenticity of an attachment received by a user via a messaging app, the user uses the application to send the file to a server, which analyzes the contents of the file and notifies the user that "This file may contain a virus (90% confidence level)."

[0939] Example prompts to input to the generative AI model

[0940] Generative AI model, please analyze whether the following link is fake: example_link. Print the result and confidence level.

[0941] This embodiment allows users to quickly evaluate the reliability of links and attachments in electronic messages and emails, and can prevent damage from fake information and phishing sites.

[0942] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0943] Step 1:

[0944] A user receives an email or message and retrieves a link or attachment from it. An application automatically detects the link or attachment information. The input here is the link or attachment from the electronic message, and the output is the raw data passed to the application.

[0945] Step 2:

[0946] The information obtained by the application is encrypted using the SSL / TLS protocol. The input is raw data and the output is encrypted data. Encryption ensures the protection of transmitted data.

[0947] Step 3:

[0948] The terminal sends encrypted data to the server. The input is the encrypted data, and the output is the received data that is recorded in the server's reception log. This transmission uses Internet communication.

[0949] Step 4:

[0950] The server receives the encrypted data and decrypts it using the SSL / TLS protocol. The input is the encrypted data and the output is the original, decrypted raw data.

[0951] Step 5:

[0952] The server uses PyTorch or TensorFlow to analyze the authenticity of links and attachments using a neural network model. The input is the decoded raw data, and the output is the analysis result (evaluation of whether the information is fake or not) and its reliability score. Specific operations include pattern recognition, feature extraction, and evaluation of links and files.

[0953] Step 6:

[0954] The server organizes the analysis results and converts them into a user-friendly format. The input is the analysis results and confidence score, and the output is a user notification such as "This link may be a phishing site (confidence 85%)."

[0955] Step 7:

[0956] The server re-encrypts the result and sends it to the terminal. The input is the result, and the output is the encrypted result data.

[0957] Step 8:

[0958] The terminal receives the encrypted result data and decrypts it again. The input is the encrypted result data, and the output is the decrypted notification data.

[0959] Step 9:

[0960] The application notifies the user of the notification data as a popup notification or message display. The input is the decoded notification data, and the output is the message or notification that is displayed on the user's device screen.

[0961] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0962] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users by combining it with an emotion engine to provide appropriate notifications that take user emotions into consideration. This system is realized by the user's device, a server, an application, and an emotion engine.

[0963] System Overview

[0964] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[0965] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[0966] System Details

[0967] 1. Information acquisition means:

[0968] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[0969] 2. Application:

[0970] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[0971] 3. Server:

[0972] The server receives data sent from the application. First, the server decrypts the received data. Next, it inputs the decrypted data into a neural network model and uses deep learning technology to evaluate the authenticity of the information. For example, in the case of images, it applies an image analysis algorithm, and in the case of text, it uses natural language processing technology.

[0973] 4. Organizational means:

[0974] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[0975] 5. Emotion Engine Processing:

[0976] The emotion engine analyzes the user's facial expressions and voice data in real time, thereby determining the user's current emotional state and sending it to the server.

[0977] 6. Sentiment data analysis on the server:

[0978] The server then combines the received emotional data with the results of its fake information analysis to generate notification content that reflects the user's emotions. For example, if the server determines that the user is feeling extremely anxious, it will adjust the notification content to be gentler.

[0979] 7. Means of notification:

[0980] The sorted results are then encrypted again and sent back to the device. After receiving the results, the device displays a pop-up notification or message to the user, instantly informing them of the reliability of the information. Appropriate notification content is also displayed based on the data from the emotion engine.

[0981] Specific examples

[0982] Example 1: Fake news detection and emotional response

[0983] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, the emotion engine analyzes the user's facial expression. If the user looks anxious, the system displays a message such as, "This news article may be fake news (90% confidence). Don't worry, you can also find other reliable information."

[0984] Example 2: Deepfake video detection and emotion response

[0985] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server then performs multi-layered analysis of the video's frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)." Furthermore, if the emotion engine analyzes the user's tone of voice and determines that the user is surprised, a message similar to, "This video is suspicious, but it's okay. We recommend further verification." is displayed.

[0986] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[0987] The processing flow will be explained below.

[0988] Step 1:

[0989] A user acquires information. Specifically, the user uses the camera function of their smartphone to take a photo or video, or to take a screenshot of a news article.

[0990] Step 2:

[0991] The information acquired by the device is input to the application, which then accesses the data stored on the device and prepares it for analysis.

[0992] Step 3:

[0993] The device sends information to the server through the application, which is then encrypted for security purposes and sent to the server using the SSL / TLS protocol.

[0994] Step 4:

[0995] The server receives the transmitted information and first decrypts the received data.

[0996] Step 5:

[0997] The server analyzes the information. The received and decoded data is fed into a neural network model, which uses deep learning techniques to evaluate the authenticity of the information. For example, if it is an image, an image analysis algorithm is applied, and if it is text, natural language processing technology is used for analysis.

[0998] Step 6:

[0999] The server organizes the analysis results. It summarizes the judgment result regarding whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[1000] Step 7:

[1001] The device recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice in real time to understand their current emotional state.

[1002] Step 8:

[1003] The device sends emotional data to the server, which is also encrypted and sent to the server using the SSL / TLS protocol.

[1004] Step 9:

[1005] The server receives and analyzes the emotion data, decodes the received data, and evaluates the user's emotional state.

[1006] Step 10:

[1007] The server integrates the emotion data and the results of the information analysis. It adjusts the notification content based on the user's emotion and returns it to the device in the most appropriate format. For example, if the user is feeling anxious, the notification will be delivered in a softer tone.

[1008] Step 11:

[1009] The server encrypts the final notification and sends it to the device, again using the SSL / TLS protocol to send the data securely.

[1010] Step 12:

[1011] The device receives and decrypts the final notification. The device receives and decrypts the data sent from the server.

[1012] Step 13:

[1013] The device will notify the user of the results. Specifically, it will inform the user of the notification content through a pop-up notification or message display. For example, a tailored notification such as "This news article may be fake news (90% confidence level). Rest assured, there are other reliable sources available." will be displayed.

[1014] This completes the process flow of the system. This series of steps allows users to quickly and accurately check the reliability of the information they have acquired, while also receiving appropriate notifications based on their emotional state.

[1015] Example 2

[1016] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1017] In modern society, there is a need to quickly and accurately evaluate the reliability of information on the Internet. However, existing technologies lack the accuracy to detect fake information, which creates the risk that users will believe false information. Furthermore, there is a lack of technology that can adjust notification content taking into account the user's emotional state, which hinders efforts to improve the user experience.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1019] In this invention, the server includes a means for using a neural network to evaluate the reliability of information, a means for using an emotion engine that analyzes the user's facial expressions and voice data, and a means for integrating the emotion data and the analysis results to adjust the notification content, thereby enabling highly accurate detection of fake information and flexible notification that takes the user's emotional state into consideration.

[1020] "Means of information acquisition" refers to the means by which users acquire information such as photos, videos, and news articles using smartphones or other devices.

[1021] "Application" means software that processes the acquired information on the user's device and transmits it to the server.

[1022] The "server" is a central component of the system that analyzes the received information using a neural network, organizes the results, and returns them to the terminal.

[1023] A "neural network" is a mathematical model that uses artificial intelligence techniques to evaluate the reliability of information.

[1024] The "emotion engine" is a component of the system that analyzes the user's facial expressions and voice data to determine their emotional state.

[1025] "Notification means" is a function for informing the user of the sorting results sent from the server, and includes pop-up notifications and messages displayed on the terminal.

[1026] MODE FOR CARRYING OUT THE INVENTION

[1027] The present invention provides a system for detecting and notifying fake information that takes into account user emotions. This system is realized by a user terminal, a server, an application, and an emotion engine. Detailed embodiments of the system are described below.

[1028] System configuration

[1029] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[1030] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[1031] Hardware and Software Used

[1032] The system uses the following hardware and software:

[1033] 1. Information acquisition means:

[1034] Users capture information using their smartphone's camera, screenshot function, or existing media files.

[1035] Information is stored on the device and accessed by applications.

[1036] 2. Application:

[1037] The acquired information is taken as input, and once the user initiates the analysis using the application, the information is encrypted and sent to the server.

[1038] The SSL / TLS protocol is used to ensure secure communications.

[1039] 3. Server:

[1040] The server decodes the received data and inputs it into a neural network model (e.g., PyTorch or TensorFlow) to evaluate the authenticity of the information.

[1041] For images, image analysis algorithms are applied, and for text, natural language processing techniques are used.

[1042] 4. Emotion Engine:

[1043] The device collects facial and voice data from the camera and microphone and analyzes it in real time.

[1044] An emotion engine (e.g., OpenFace or IBM Watson) determines the user's emotional state.

[1045] 5. Organizational means:

[1046] The server organizes the analysis results in a format such as "High probability of fake news (95% confidence)" and integrates it with data from the emotion engine to adjust the content of the notification.

[1047] 6. Means of notification:

[1048] The server re-encrypts the results and sends them back to the terminal.

[1049] After the device receives the results, it will display a pop-up notification or message to the user to inform them of the reliability of the information.

[1050] Specific examples

[1051] Example 1: Fake news detection and emotional response

[1052] 1. Information acquisition

[1053] When a user reads an article on a news site, they take a screenshot of the article and send it to a server through the application.

[1054] 2. Data Analysis

[1055] The server analyzes the text and images in the screenshot and determines that the content is likely to be fake news.

[1056] 3. Sentiment analysis

[1057] The emotion engine analyzes the user's facial expressions, and if the user looks anxious, the system will notify them with a message such as, "This news article may be fake (90% confidence). Don't worry, there are other reliable sources available."

[1058] Example 2: Deepfake video detection and emotion response

[1059] 1. Information acquisition

[1060] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server.

[1061] 2. Data Analysis

[1062] The server performs multi-layered analysis of video frames and audio to detect deepfake characteristics.

[1063] 3. Sentiment analysis

[1064] If the emotion engine analyzes the user's tone of voice and determines that the user is surprised, it will display a message such as, "This video is suspicious, but it's okay. We recommend you check it out further."

[1065] Prompt Sentence Examples

[1066] To the generative AI model: Please provide an example prompt that uses a user-generated screenshot of a news article to detect fake news.

[1067] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[1068] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1069] System program processing flow

[1070] Step 1: Information Acquisition

[1071] A user uses the camera function or screenshot function of their smartphone to obtain information (photos, videos, news articles, etc.).

[1072] Input: Information from social media, news sites, etc.

[1073] What happens: A user taps the camera icon on their smartphone to launch the camera app and take a screenshot of a suspicious news article.

[1074] Output: Information stored on the device (image files, etc.)

[1075] Step 2: Send data

[1076] The information acquired by the terminal is sent to the server via the application.

[1077] Input: Information stored on the device (image files, etc.)

[1078] What it does: When you tap the "Start Analysis" button in the app, the screenshot is encrypted and uploaded to the server, using the SSL / TLS protocol to ensure secure communication.

[1079] Output: Encrypted information (such as an image file) is sent to the server

[1080] Step 3: Data analysis

[1081] The server decodes the received information and analyzes it using a neural network model.

[1082] Input: Encrypted and transmitted information (such as image files)

[1083] What it does: The server decodes the received screenshot and evaluates its content using an image analysis algorithm (for example, a model using PyTorch or TensorFlow). If text is included, it extracts it using OCR (Optical Character Recognition) technology and analyzes the text using natural language processing technology.

[1084] Output: Evaluation result of whether the information is fake or not and the confidence level

[1085] Step 4: Sentiment analysis

[1086] The device uses a camera and microphone to collect the user's facial expressions and voice data, which is then analyzed in real time by an emotion engine.

[1087] Input: User's facial expression data and speech sound data

[1088] How it works: When a user looks into the smartphone camera, the camera captures their facial expressions and the microphone collects audio data. An emotion engine (e.g., OpenFace or IBM Watson) analyzes this data and estimates the user's emotional state.

[1089] Output: Evaluation of the user's emotional state

[1090] Step 5: Organize the results

[1091] The server organizes the analysis results and converts them into a format that is easy for the user to understand.

[1092] Input: Evaluation results of fake information and evaluation results of user's emotional state

[1093] How it works: The server generates results in the form of "Highly likely to be fake news (95% confidence)" and integrates information and emotional data. The tone and content of the notification message are also adjusted depending on the user's emotional state.

[1094] Output: Organized results (information message)

[1095] Step 6: Send results

[1096] The server re-encrypts the organized results and sends them to the terminal.

[1097] Input: Organized results (notification message)

[1098] How it works: The server encrypts the results again using SSL / TLS and sends them to the device. The notification contains the fake news analysis results and sentiment-based content.

[1099] Output: Encrypted notification message

[1100] Step 7: Notifications

[1101] The terminal receives the results and displays a pop-up notification or message to the user.

[1102] Input: Encrypted notification message

[1103] What it does: The device will display the results it receives from the server, with a pop-up notification on the screen. For example, it might say, "This news article may be fake (90% confidence level). Don't worry, there are other reliable sources available."

[1104] Output: A notification message that is displayed to the user

[1105] Through these steps, the system can analyze the reliability of the information obtained by the user with high accuracy and provide flexible notifications according to the user's emotional state.

[1106] (Application example 2)

[1107] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1108] Current fake information detection systems can instantly assess the authenticity of information, but lack the ability to adjust notification content based on the user's emotional state. This means that if a user feels uneasy or surprised about the reliability of the information, they are unable to respond appropriately, increasing the user's psychological burden. Furthermore, their multi-layered analysis of each type of information is insufficient, limiting the accuracy of their fake information detection.

[1109] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information using a neural network, means for organizing the analysis results and returning them to the terminal, and means having an emotion engine for analyzing the user's facial expressions and voice. This makes it possible to evaluate the reliability of information in real time and adjust the notification content according to the user's emotional state. In addition, multi-layered analysis according to the type of information, such as images, voice, and text, improves the accuracy of detecting fake information.

[1110] "Means of information acquisition" refers to the means by which a user acquires information such as images, audio, and text using a smartphone or other device.

[1111] "Means for inputting information into an application and transmitting it to a server" refers to means for inputting acquired information into an application, encrypting it, and transmitting it securely to a server.

[1112] "Means for analyzing information using neural networks" refers to means for analyzing information obtained on a server using deep learning technology and evaluating its authenticity.

[1113] The "means for organizing the analysis results and returning them to the terminal" refers to a means for converting the results analyzed by the server into a format that is easy for the user to understand and returning them to the terminal for notification.

[1114] "Means for notifying the user of the organized results" refers to means for informing the user of the analysis results using a pop-up notification or message display on the terminal.

[1115] "Means having an emotion engine that analyzes the user's facial expressions and voice" refers to means that uses the user's camera and microphone to analyze the user's facial expressions and voice in real time and recognize the user's emotional state.

[1116] The "means for adjusting the notification content based on the analysis results of the emotion engine" refers to a means for appropriately changing the notification content based on the analysis results in accordance with the emotional state of the user.

[1117] The present invention is a system that analyzes information about videos watched by users in real time, detects fake information, and notifies users appropriately based on their emotional state. This system includes the following elements:

[1118] System configuration and program processing

[1119] 1. Information acquisition means

[1120] Devices such as smartphones and tablets capture frames of the video a user is watching using their camera or screen recording capabilities.

[1121] 2. A means of inputting data into the application and sending it to the server

[1122] The captured video frames are input into a dedicated application, which then encrypts the frames and securely transmits them to the server. The communication between the client and server is secured using the SSL / TLS protocol.

[1123] 3. How to analyze information using neural networks

[1124] The server analyzes the received video frames using deep learning techniques, using OpenCV for image analysis, an NLP (natural language processing) library for text analysis, and a speech recognition library for audio analysis.

[1125] 4. A means of organizing the analysis results and returning them to the device

[1126] The server compiles the analysis results and returns them to the device in an easy-to-understand format, such as "Possibly fake information (confidence level XX%)." This result is also sent encrypted.

[1127] 5. A method for notifying users of the organized results

[1128] The analysis results received by the device are immediately communicated to the user via pop-up notifications or message displays, with the content of the notifications tailored based on the user's emotional state.

[1129] 6. Means for providing an emotion engine that analyzes the user's facial expressions and voice

[1130] The system uses the smartphone's camera and microphone to collect the user's facial expressions and voice in real time, using the emotion analysis library emotion_detection, and the engine analyzes the user's emotional state (anxiety, surprise, etc.).

[1131] 7. A way to tailor notifications based on the emotion engine analysis results

[1132] After the emotion engine determines the user's emotional state, the server will modify the notification accordingly. For example, if the user is anxious, the notification will say something like, "This information is suspicious, but it's okay. Please check for reliable information."

[1133] Specific examples

[1134] Example 1: Fake news detection and emotional response

[1135] When a user reads an article on a news site, they take a screenshot of the article and send it to the server. The server analyzes the text and images in the screenshot to detect suspicious content. Based on this result, if the sentiment analysis engine detects an anxious expression on the user's face, a notification will be displayed saying, "This news article may be fake (90% confidence). Don't worry, there is other reliable information available."

[1136] Example 2: Deepfake video detection and emotion response

[1137] To verify the authenticity of a video sent by a friend, the user uses the application to send the video to a server. The server analyzes the video frames and audio to detect deepfake characteristics. After analyzing the user's surprised tone of voice, a notification appears saying, "This video may be a deepfake (85% confidence). It's okay, but we recommend further verification."

[1138] Prompt Sentence Examples

[1139] Analyze the video frames being watched to detect fake information. Extract text and objects in the video frames, and comprehensively evaluate all information to determine whether it is fake. Analyze the user's facial expressions and voice, and generate appropriate notifications based on the results.

[1140] By detecting fake information and providing flexible notifications that adapt to the user's emotions, the present invention creates an environment where users can safely confirm the reliability of information.

[1141] The above configuration enables the rapid and accurate detection of fake information and notifications that reduce the psychological burden on users.

[1142] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1143] Step 1:

[1144] The video being watched by the user is captured using the camera or screen recording function of the smartphone or tablet device. When the user presses a specific button, the application captures and temporarily saves video frames. The input is the video being watched, and the output is the captured video frames.

[1145] Step 2:

[1146] The terminal application inputs the captured video frames, encrypts them, and sends them to the server. The application uses the SSL / TLS protocol to ensure secure data transmission. The input is the captured video frames, and the output is the encrypted video frame data.

[1147] Step 3:

[1148] The server decrypts the received encrypted data and analyzes the video frames. A generative AI model is used for the analysis, and image analysis is performed using OpenCV. The input is the encrypted video frame data, and the output is the analysis result (e.g., "Possible fake information").

[1149] Step 4:

[1150] The server organizes the analysis results and converts them into a user-friendly format. At this time, detailed information is added according to the reliability and type of the analysis results (image, audio, text). The input is the analysis results, and the output is the organized analysis results (e.g., "Possibly fake information (reliability 90%)").

[1151] Step 5:

[1152] The server again encrypts the compiled analysis results and sends them back to the terminal, again using the SSL / TLS protocol to ensure secure data transmission. The input is the compiled analysis results, and the output is the encrypted analysis result data.

[1153] Step 6:

[1154] The device decrypts the encrypted data received and notifies the user. Notifications are made using pop-ups or messages. The input is the encrypted analysis result data, and the output is a notification message for the user.

[1155] Step 7:

[1156] The emotion engine collects the user's facial expression and voice data and performs real-time emotion analysis. The emotion_detection library is used for emotion analysis. The input is the user's facial expression and voice data, and the output is the emotion analysis result (e.g., "anxious state").

[1157] Step 8:

[1158] The server adjusts the notification content based on the analysis results of the emotion engine. For example, if the user is in an anxious state, it generates a gentle notification message. The input is the emotion analysis result and the analyzed fake information, and the output is the adjusted notification message.

[1159] Step 9:

[1160] The device displays the final notification message to the user, allowing the user to confirm the reliability of the information in a way that takes into account their emotional state. The input is the adjusted notification message, and the output is the final notification to the user.

[1161] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1162] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1163] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1164] [Fourth embodiment]

[1165] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1166] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1167] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1168] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1169] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1170] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1171] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1172] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1173] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1174] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1175] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1176] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1177] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1178] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[1179] System Overview

[1180] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to identify fake information. The server then sends back the organized analysis results to the device, which notifies the user in real time. This series of processes allows users to quickly and accurately verify the reliability of the information.

[1181] System Details

[1182] 1. Information acquisition means:

[1183] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[1184] 2. Application:

[1185] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[1186] 3. Server:

[1187] The server receives the data sent from the application. After receiving the data, it is decoded and input into a neural network model. Using deep learning technology, the server evaluates the authenticity of the information. The analysis results in a decision on whether the information is fake and its reliability.

[1188] 4. Organizational means:

[1189] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[1190] 5. Means of notification:

[1191] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[1192] Specific examples

[1193] Example 1: Fake news detection

[1194] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[1195] Example 2: Deepfake video detection

[1196] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[1197] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[1198] The processing flow will be explained below.

[1199] Step 1:

[1200] The user acquires information. Specifically, the user uses the camera function of their smartphone to take photos or videos, or to take screenshots.

[1201] Step 2:

[1202] The information acquired by the device is input into the application, which then accesses data stored on the device.

[1203] Step 3:

[1204] The device sends information to the server through the application, which encrypts the information for security purposes and sends it to the server using the SSL / TLS protocol.

[1205] Step 4:

[1206] The server receives the transmitted information. First, the server decrypts the received data.

[1207] Step 5:

[1208] The server analyzes the information. The decoded data is fed into a neural network model and deep learning techniques are used to assess the authenticity of the information. For example, image analysis algorithms are applied to images, and natural language processing techniques are used to assess text.

[1209] Step 6:

[1210] The server organizes the analysis results. Specifically, it summarizes the results of its judgment on whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[1211] Step 7:

[1212] The server then sends the compiled analysis results back to the device, which are also encrypted and transmitted using the SSL / TLS protocol.

[1213] Step 8:

[1214] The terminal receives and decodes the analysis results. The terminal receives the data from the server and converts it into a format that is easy for the user to understand.

[1215] Step 9:

[1216] The device will notify the user of the results. Specifically, it will display a pop-up notification or message, such as "This news article may be fake news (90% confidence level)."

[1217] The above is the specific processing flow of this system. This series of steps allows users to quickly and accurately confirm the reliability of the information they have obtained, enabling them to use the information safely.

[1218] Example 1

[1219] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1220] In recent years, the increase in fake information has led to social unrest and increased fraud. This problem is particularly serious in today's society, where information spreads instantly via the Internet. Current methods of verifying information often lack real-time capabilities and require users to determine the authenticity of information themselves, which requires a certain level of specialized knowledge. This creates a risk that ordinary users can easily spread unreliable information.

[1221] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1222] In this invention, the server includes means for inputting acquired information into an application, encrypting it, and sending it to the server, means for the server to receive the encrypted information, decrypt it, and analyze it using a neural network, and means for the server to organize the analysis results, re-encrypt the results, and send them back to the terminal. This makes it possible for information acquired by the user to be analyzed in real time, its reliability evaluated, and the results notified immediately.

[1223] "Means of information acquisition" refers to the means by which a user acquires data such as images, text, and audio using a device such as a smartphone or computer.

[1224] An "application" is software that runs on a terminal, encrypts acquired information, and transmits it to a server.

[1225] "Encryption" is the process of transforming transmitted data so that it cannot be deciphered by third parties.

[1226] A "server" is a computer system that receives information over a network, decodes it, and analyzes it using a neural network.

[1227] "Decryption" is the process of restoring encrypted data to its original state.

[1228] A "neural network" is an artificial intelligence model that analyzes and evaluates information by mimicking the function of nerve cells in the human brain.

[1229] "Analysis" is the process of examining information in detail to find specific features or patterns.

[1230] "Organization" is the process of converting the analysis results obtained within the server into a format that is easy for users to understand.

[1231] A "terminal" is a device that is directly operated by a user and has the role of obtaining and notifying information.

[1232] "Notification" refers to the means by which the device displays the analysis results to the user, and includes methods such as pop-up notifications and message displays.

[1233] "Real-time" refers to processes that are executed immediately, without delay.

[1234] "Reliability" is an indicator of the veracity and accuracy of information obtained through analysis.

[1235] "Multi-layered" means that the analysis is divided into multiple elements or layers, and refers to the simultaneous evaluation of different data formats such as images, audio, and text.

[1236] "Fake information" is data that contains intentionally misleading or false information.

[1237] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users. This system is realized by a user's device, a server, and an application that links them.

[1238] System Overview

[1239] Users use devices such as smartphones to obtain information (photos, videos, news articles, etc.). This information is then sent to a server via an application. The server analyzes the received information using a neural network model to determine whether it is fake or not. The server then sends back the organized analysis results to the device, which notifies the user in real time. This process allows users to quickly and accurately verify the reliability of the information.

[1240] Information acquisition means

[1241] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[1242] application

[1243] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[1244] server

[1245] The server receives the data sent from the application. After receiving the data, it is decrypted and input into a neural network model. Using deep learning technology, the server analyzes the authenticity of the information. The analysis results in a judgment of whether the information is fake and its reliability.

[1246] means of organization

[1247] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[1248] Notification means

[1249] The results are then encrypted again and sent back to the device, which then displays a pop-up notification or message to the user, instantly informing them of the authenticity of the information.

[1250] Specific examples

[1251] Example 1: Fake news detection

[1252] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, it sends a notification to the user saying, "This news article may be fake news (90% confidence)."

[1253] Example 2: Deepfake video detection

[1254] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server performs multiple analyses of the video frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)."

[1255] By combining multi-layered detection technology with real-time notification functions, this system provides an environment where users can enjoy digital information with peace of mind, thereby preventing social unrest and crime caused by fake information.

[1256] Prompt Sentence Examples

[1257] Please explain the process flow of the system program in detail using the following steps. The subject should be either the server, terminal, or user.

[1258] procedure:

[1259] 1. The user obtains the information.

[1260] 2. The device passes the information to the application.

[1261] 3. The application encrypts the information and sends it to the server.

[1262] 4. The server receives and decrypts the information.

[1263] 5. The server inputs the information into a neural network model for analysis.

[1264] 6. The server organizes the analysis results.

[1265] 7. The server encrypts the results and sends them to the device.

[1266] 8. The device receives the results and notifies the user.

[1267] Examples:

[1268] Users take screenshots of news articles and save them to their devices. The application encrypts them and sends them to a server. The server analyzes the screenshots and notifies the user if they are deemed to be fake news.

[1269] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1270] Step 1:

[1271] The user acquires information. The user acquires image or text information using the smartphone's camera or screenshot function. For example, the user views an article on a news site and takes a screenshot of it. The input data is the image or text. The output is the acquired information stored on the device.

[1272] Step 2:

[1273] The device passes information to the application. The device passes the captured image and text information to the application, which then retrieves it. During this process, screenshots stored on the device are imported into the application. Input data is information stored on the device, and output data is information imported into the application.

[1274] Step 3:

[1275] The application encrypts information and sends it to the server. The application encrypts information using the SSL / TLS protocol and sends it to the server. The input data here is the unencrypted information imported into the application, and the output data is the encrypted data sent to the server.

[1276] Step 4:

[1277] The server receives and decrypts the information. The server receives and decrypts the encrypted data. Inside the server, the data is decrypted using the AES algorithm. The input data is the encrypted information, and the output data is the decrypted information.

[1278] Step 5:

[1279] The server inputs the information into a neural network model for analysis. Using deep learning technology, the server inputs the decoded information into the neural network model for analysis. For example, the text of a news article is analyzed using the BERT model, and the image is analyzed using the ResNet model. The input data is the decoded information, and the output data is the analysis result.

[1280] Step 6:

[1281] The server organizes the analysis results. The server converts the analysis results into a user-friendly format and assigns a reliability rating. For example, the server might say, "This article may be fake news (90% reliability)." The input data is the analysis results, and the output data is the organized results.

[1282] Step 7:

[1283] The server encrypts the organized results and sends them to the terminal. The server re-encrypts the organized results and sends them to the terminal. The input data is the organized results, and the output data is the encrypted results.

[1284] Step 8:

[1285] The device receives the results and notifies the user. The device receives the encrypted data sent from the server and decrypts it. It then displays a pop-up notification or message to the user. For example, it displays a notification saying, "This article may be fake news (90% confidence level)." The input data is the encrypted result, and the output data is the result notified to the user.

[1286] (Application example 1)

[1287] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1288] The widespread use of the Internet has led to the rapid spread of harmful information, such as false information, fake news, and phishing links. This has led to many users becoming victims of fraud and experiencing social unrest due to misinformation. However, current systems lack a way to detect this fake information in real time and promptly notify users. In particular, there is a need for a means to instantly evaluate the reliability of links and attachments in electronic messages.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1290] In this invention, the server includes means for acquiring information about links and attachments in electronic messages in real time, means for determining the authenticity of links and attachments using a neural network, and means for notifying the user of the authenticity information, which allows the user to evaluate the reliability of links and attachments in emails and messages in real time and prevent damage from fake information and phishing sites.

[1291] "Information acquisition means" refers to the function of a device or software that collects information (links, attachments, images, audio, text, etc.) that a user acquires.

[1292] An "application" is a program that sends acquired information to a server and analyzes and organizes the results.

[1293] A "server" is a computer system that receives information sent from an application and analyzes it using a neural network.

[1294] A "neural network" is a machine learning model that uses multiple layers of artificial neurons, and is a technology used to determine the authenticity of information.

[1295] The "organizing means" is a process in which the server converts the analysis results into a format that is easy for the user to understand and returns it to the terminal.

[1296] "Notification means" is a function that notifies the user of the organized results as a pop-up notification or message display.

[1297] "Links and attachments in electronic messages" are URL links and files contained in email or messaging applications.

[1298] "Real-time retrieval" refers to the ability to retrieve links and attachments in electronic messages immediately upon their arrival.

[1299] The "means for determining authenticity" is a function that uses a neural network to analyze acquired information and determine whether the information is fake.

[1300] "Means of notifying including reliability information" is a function that informs the user of the analysis results and their reliability (e.g., reliability 85%).

[1301] The present invention is a system that allows users to evaluate the reliability of links and attachments in electronic messages in real time, quickly determining whether the information is fake and notifying users. This system uses a smartphone, a server, and an application that links them.

[1302] System Overview

[1303] Users use an application installed on their smartphone to retrieve links and attachments from electronic messages and emails. This information is encrypted and sent to a server via the application. The server analyzes the information using a neural network model and notifies the user of the results in real time.

[1304] System Details

[1305] 1. Information acquisition means:

[1306] After users retrieve links and attachments in electronic messages and emails, the information is automatically collected by the application, which uses a module that retrieves information from the API of the smartphone's default email and messaging apps.

[1307] 2. Application:

[1308] The application receives the acquired information as input and encrypts the data, using the SSL / TLS protocol to ensure secure communication. The encrypted data is then sent to the server.

[1309] 3. Server:

[1310] The server is installed on AWS EC2 or Google Cloud Platform. The server decrypts the received data and analyzes it using a neural network model using PyTorch or TensorFlow. This analysis determines the authenticity of links and attachments and evaluates their reliability as a numerical value (e.g., 85% reliability).

[1311] 4. Organizational means:

[1312] The server then organizes the analysis results and converts them into a user-friendly format. For example, the result may be displayed as "This link may be a phishing site (confidence 85%)." The confidence level is also displayed, allowing users to trust the results.

[1313] 5. Means of notification:

[1314] The results are then encrypted and sent back to the device, where the smartphone application receives them and displays a pop-up notification or message to the user.

[1315] Specific examples

[1316] Example 1: Evaluating the trustworthiness of links in emails

[1317] When a user receives an email and tries to click on a link, the application automatically captures the link and sends it to the server, which analyzes the link's reliability and sends a notification to the user saying, "This link may be a phishing site (85% confidence)."

[1318] Example 2: Authentication of attached files

[1319] To verify the authenticity of an attachment received by a user via a messaging app, the user uses the application to send the file to a server, which analyzes the contents of the file and notifies the user that "This file may contain a virus (90% confidence level)."

[1320] Example prompts to input to the generative AI model

[1321] Generative AI model, please analyze whether the following link is fake: example_link. Print the result and confidence level.

[1322] This embodiment allows users to quickly evaluate the reliability of links and attachments in electronic messages and emails, and can prevent damage from fake information and phishing sites.

[1323] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1324] Step 1:

[1325] A user receives an email or message and retrieves a link or attachment from it. An application automatically detects the link or attachment information. The input here is the link or attachment from the electronic message, and the output is the raw data passed to the application.

[1326] Step 2:

[1327] The information obtained by the application is encrypted using the SSL / TLS protocol. The input is raw data and the output is encrypted data. Encryption ensures the protection of transmitted data.

[1328] Step 3:

[1329] The terminal sends encrypted data to the server. The input is the encrypted data, and the output is the received data that is recorded in the server's reception log. This transmission uses Internet communication.

[1330] Step 4:

[1331] The server receives the encrypted data and decrypts it using the SSL / TLS protocol. The input is the encrypted data and the output is the original, decrypted raw data.

[1332] Step 5:

[1333] The server uses PyTorch or TensorFlow to analyze the authenticity of links and attachments using a neural network model. The input is the decoded raw data, and the output is the analysis result (evaluation of whether the information is fake or not) and its reliability score. Specific operations include pattern recognition, feature extraction, and evaluation of links and files.

[1334] Step 6:

[1335] The server organizes the analysis results and converts them into a user-friendly format. The input is the analysis results and confidence score, and the output is a user notification such as "This link may be a phishing site (confidence 85%)."

[1336] Step 7:

[1337] The server re-encrypts the result and sends it to the terminal. The input is the result, and the output is the encrypted result data.

[1338] Step 8:

[1339] The terminal receives the encrypted result data and decrypts it again. The input is the encrypted result data, and the output is the decrypted notification data.

[1340] Step 9:

[1341] The application notifies the user of the notification data as a popup notification or message display. The input is the decoded notification data, and the output is the message or notification that is displayed on the user's device screen.

[1342] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1343] This invention is a system that uses multi-layered detection technology to detect fake information in real time and notify users by combining it with an emotion engine to provide appropriate notifications that take user emotions into consideration. This system is realized by the user's device, a server, an application, and an emotion engine.

[1344] System Overview

[1345] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[1346] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[1347] System Details

[1348] 1. Information acquisition means:

[1349] Users capture information using their smartphone's camera, screenshot function, or existing media files, which are then stored on the device and accessed by the application.

[1350] 2. Application:

[1351] The application receives the acquired information as input. When the user starts the analysis using the application, the information is encrypted and sent to the server. The application uses the SSL / TLS protocol to ensure secure communication.

[1352] 3. Server:

[1353] The server receives data sent from the application. First, the server decrypts the received data. Next, it inputs the decrypted data into a neural network model and uses deep learning technology to evaluate the authenticity of the information. For example, in the case of images, it applies an image analysis algorithm, and in the case of text, it uses natural language processing technology.

[1354] 4. Organizational means:

[1355] The server then organizes the analysis results and converts them into a user-friendly format, such as "High probability of fake information (95% confidence)."

[1356] 5. Emotion Engine Processing:

[1357] The emotion engine analyzes the user's facial expressions and voice data in real time, thereby determining the user's current emotional state and sending it to the server.

[1358] 6. Sentiment data analysis on the server:

[1359] The server then combines the received emotional data with the results of its fake information analysis to generate notification content that reflects the user's emotions. For example, if the server determines that the user is feeling extremely anxious, it will adjust the notification content to be gentler.

[1360] 7. Means of notification:

[1361] The sorted results are then encrypted again and sent back to the device. After receiving the results, the device displays a pop-up notification or message to the user, instantly informing them of the reliability of the information. Appropriate notification content is also displayed based on the data from the emotion engine.

[1362] Specific examples

[1363] Example 1: Fake news detection and emotional response

[1364] When a user reads an article on a news site, they take a screenshot of the article and send it to the server via the application. The server analyzes the text and images in the screenshot, and if it determines that the content is likely to be fake news, the emotion engine analyzes the user's facial expression. If the user looks anxious, the system displays a message such as, "This news article may be fake news (90% confidence). Don't worry, you can also find other reliable information."

[1365] Example 2: Deepfake video detection and emotion response

[1366] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server. The server then performs multi-layered analysis of the video's frames and audio to detect deepfake characteristics. As a result, a notification appears on the device stating, "This video may be a deepfake (85% confidence)." Furthermore, if the emotion engine analyzes the user's tone of voice and determines that the user is surprised, a message similar to, "This video is suspicious, but it's okay. We recommend further verification." is displayed.

[1367] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[1368] The processing flow will be explained below.

[1369] Step 1:

[1370] A user acquires information. Specifically, the user uses the camera function of their smartphone to take a photo or video, or to take a screenshot of a news article.

[1371] Step 2:

[1372] The information acquired by the device is input to the application, which then accesses the data stored on the device and prepares it for analysis.

[1373] Step 3:

[1374] The device sends information to the server through the application, which is then encrypted for security purposes and sent to the server using the SSL / TLS protocol.

[1375] Step 4:

[1376] The server receives the transmitted information and first decrypts the received data.

[1377] Step 5:

[1378] The server analyzes the information. The received and decoded data is fed into a neural network model, which uses deep learning techniques to evaluate the authenticity of the information. For example, if it is an image, an image analysis algorithm is applied, and if it is text, natural language processing technology is used for analysis.

[1379] Step 6:

[1380] The server organizes the analysis results. It summarizes the judgment result regarding whether the information is fake and its reliability. For example, it may be formatted as "Possibly fake information (90% reliability)."

[1381] Step 7:

[1382] The device recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice in real time to understand their current emotional state.

[1383] Step 8:

[1384] The device sends emotional data to the server, which is also encrypted and sent to the server using the SSL / TLS protocol.

[1385] Step 9:

[1386] The server receives and analyzes the emotion data, decodes the received data, and evaluates the user's emotional state.

[1387] Step 10:

[1388] The server integrates the emotion data and the results of the information analysis. It adjusts the notification content based on the user's emotion and returns it to the device in the most appropriate format. For example, if the user is feeling anxious, the notification will be delivered in a softer tone.

[1389] Step 11:

[1390] The server encrypts the final notification and sends it to the device, again using the SSL / TLS protocol to send the data securely.

[1391] Step 12:

[1392] The device receives and decrypts the final notification. The device receives and decrypts the data sent from the server.

[1393] Step 13:

[1394] The device will notify the user of the results. Specifically, it will inform the user of the notification content through a pop-up notification or message display. For example, a tailored notification such as "This news article may be fake news (90% confidence level). Rest assured, there are other reliable sources available." will be displayed.

[1395] This completes the process flow of the system. This series of steps allows users to quickly and accurately check the reliability of the information they have acquired, while also receiving appropriate notifications based on their emotional state.

[1396] Example 2

[1397] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1398] In modern society, there is a need to quickly and accurately evaluate the reliability of information on the Internet. However, existing technologies lack the accuracy to detect fake information, which creates the risk that users will believe false information. Furthermore, there is a lack of technology that can adjust notification content taking into account the user's emotional state, which hinders efforts to improve the user experience.

[1399] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1400] In this invention, the server includes a means for using a neural network to evaluate the reliability of information, a means for using an emotion engine that analyzes the user's facial expressions and voice data, and a means for integrating the emotion data and the analysis results to adjust the notification content, thereby enabling highly accurate detection of fake information and flexible notification that takes the user's emotional state into consideration.

[1401] "Means of information acquisition" refers to the means by which users acquire information such as photos, videos, and news articles using smartphones or other devices.

[1402] "Application" means software that processes the acquired information on the user's device and transmits it to the server.

[1403] The "server" is a central component of the system that analyzes the received information using a neural network, organizes the results, and returns them to the terminal.

[1404] A "neural network" is a mathematical model that uses artificial intelligence techniques to evaluate the reliability of information.

[1405] The "emotion engine" is a component of the system that analyzes the user's facial expressions and voice data to determine their emotional state.

[1406] "Notification means" is a function for informing the user of the sorting results sent from the server, and includes pop-up notifications and messages displayed on the terminal.

[1407] MODE FOR CARRYING OUT THE INVENTION

[1408] The present invention provides a system for detecting and notifying fake information that takes into account user emotions. This system is realized by a user terminal, a server, an application, and an emotion engine. Detailed embodiments of the system are described below.

[1409] System configuration

[1410] First, the user acquires information (photos, videos, news articles, etc.) using a device such as a smartphone. This information is then sent to a server via an application. The server then analyzes the received information using a neural network model to identify fake information. The results of the analysis are then sent back to the device, which then notifies the user in real time.

[1411] Additionally, an emotion engine will be added that analyzes the user's facial expressions and voice data. This emotion engine will be able to recognize the user's emotions and adjust the content of notifications accordingly. This process allows users to quickly and accurately confirm the reliability of information while receiving appropriate notifications according to their emotional state.

[1412] Hardware and Software Used

[1413] The system uses the following hardware and software:

[1414] 1. Information acquisition means:

[1415] Users capture information using their smartphone's camera, screenshot function, or existing media files.

[1416] Information is stored on the device and accessed by applications.

[1417] 2. Application:

[1418] The acquired information is taken as input, and once the user initiates the analysis using the application, the information is encrypted and sent to the server.

[1419] The SSL / TLS protocol is used to ensure secure communications.

[1420] 3. Server:

[1421] The server decodes the received data and inputs it into a neural network model (e.g., PyTorch or TensorFlow) to evaluate the authenticity of the information.

[1422] For images, image analysis algorithms are applied, and for text, natural language processing techniques are used.

[1423] 4. Emotion Engine:

[1424] The device collects facial and voice data from the camera and microphone and analyzes it in real time.

[1425] An emotion engine (e.g., OpenFace or IBM Watson) determines the user's emotional state.

[1426] 5. Organizational means:

[1427] The server organizes the analysis results in a format such as "High probability of fake news (95% confidence)" and integrates it with data from the emotion engine to adjust the content of the notification.

[1428] 6. Means of notification:

[1429] The server re-encrypts the results and sends them back to the terminal.

[1430] After the device receives the results, it will display a pop-up notification or message to the user to inform them of the reliability of the information.

[1431] Specific examples

[1432] Example 1: Fake news detection and emotional response

[1433] 1. Information acquisition

[1434] When a user reads an article on a news site, they take a screenshot of the article and send it to a server through the application.

[1435] 2. Data Analysis

[1436] The server analyzes the text and images in the screenshot and determines that the content is likely to be fake news.

[1437] 3. Sentiment analysis

[1438] The emotion engine analyzes the user's facial expressions, and if the user looks anxious, the system will notify them with a message such as, "This news article may be fake (90% confidence). Don't worry, there are other reliable sources available."

[1439] Example 2: Deepfake video detection and emotion response

[1440] 1. Information acquisition

[1441] To verify the authenticity of a video sent by a friend, a user uses the application to send the video to a server.

[1442] 2. Data Analysis

[1443] The server performs multi-layered analysis of video frames and audio to detect deepfake characteristics.

[1444] 3. Sentiment analysis

[1445] If the emotion engine analyzes the user's tone of voice and determines that the user is surprised, it will display a message such as, "This video is suspicious, but it's okay. We recommend you check it out further."

[1446] Prompt Sentence Examples

[1447] To the generative AI model: Please provide an example prompt that uses a user-generated screenshot of a news article to detect fake news.

[1448] In this way, by combining the detection of fake information with flexible notifications that take user emotions into consideration, the present invention provides an environment in which users can safely confirm the reliability of information.

[1449] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1450] System program processing flow

[1451] Step 1: Information Acquisition

[1452] A user uses the camera function or screenshot function of their smartphone to obtain information (photos, videos, news articles, etc.).

[1453] Input: Information from social media, news sites, etc.

[1454] What happens: A user taps the camera icon on their smartphone to launch the camera app and take a screenshot of a suspicious news article.

[1455] Output: Information stored on the device (image files, etc.)

[1456] Step 2: Send data

[1457] The information acquired by the terminal is sent to the server via the application.

[1458] Input: Information stored on the device (image files, etc.)

[1459] What it does: When you tap the "Start Analysis" button in the app, the screenshot is encrypted and uploaded to the server, using the SSL / TLS protocol to ensure secure communication.

[1460] Output: Encrypted information (such as an image file) is sent to the server

[1461] Step 3: Data analysis

[1462] The server decodes the received information and analyzes it using a neural network model.

[1463] Input: Encrypted and transmitted information (such as image files)

[1464] What it does: The server decodes the received screenshot and evaluates its content using an image analysis algorithm (for example, a model using PyTorch or TensorFlow). If text is included, it extracts it using OCR (Optical Character Recognition) technology and analyzes the text using natural language processing technology.

[1465] Output: Evaluation result of whether the information is fake or not and the confidence level

[1466] Step 4: Sentiment analysis

[1467] The device uses a camera and microphone to collect the user's facial expressions and voice data, which is then analyzed in real time by an emotion engine.

[1468] Input: User's facial expression data and speech sound data

[1469] How it works: When a user looks into the smartphone camera, the camera captures their facial expressions and the microphone collects audio data. An emotion engine (e.g., OpenFace or IBM Watson) analyzes this data and estimates the user's emotional state.

[1470] Output: Evaluation of the user's emotional state

[1471] Step 5: Organize the results

[1472] The server organizes the analysis results and converts them into a format that is easy for the user to understand.

[1473] Input: Evaluation results of fake information and evaluation results of user's emotional state

[1474] How it works: The server generates results in the form of "Highly likely to be fake news (95% confidence)" and integrates information and emotional data. The tone and content of the notification message are also adjusted depending on the user's emotional state.

[1475] Output: Organized results (information message)

[1476] Step 6: Send results

[1477] The server re-encrypts the organized results and sends them to the terminal.

[1478] Input: Organized results (notification message)

[1479] How it works: The server encrypts the results again using SSL / TLS and sends them to the device. The notification contains the fake news analysis results and sentiment-based content.

[1480] Output: Encrypted notification message

[1481] Step 7: Notifications

[1482] The terminal receives the results and displays a pop-up notification or message to the user.

[1483] Input: Encrypted notification message

[1484] What it does: The device will display the results it receives from the server, with a pop-up notification on the screen. For example, it might say, "This news article may be fake (90% confidence level). Don't worry, there are other reliable sources available."

[1485] Output: A notification message that is displayed to the user

[1486] Through these steps, the system can analyze the reliability of the information obtained by the user with high accuracy and provide flexible notifications according to the user's emotional state.

[1487] (Application example 2)

[1488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1489] Current fake information detection systems can instantly assess the authenticity of information, but lack the ability to adjust notification content based on the user's emotional state. This means that if a user feels uneasy or surprised about the reliability of the information, they are unable to respond appropriately, increasing the user's psychological burden. Furthermore, their multi-layered analysis of each type of information is insufficient, limiting the accuracy of their fake information detection.

[1490] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing information using a neural network, means for organizing the analysis results and returning them to the terminal, and means having an emotion engine for analyzing the user's facial expressions and voice. This makes it possible to evaluate the reliability of information in real time and adjust the notification content according to the user's emotional state. In addition, multi-layered analysis according to the type of information, such as images, voice, and text, improves the accuracy of detecting fake information.

[1491] "Means of information acquisition" refers to the means by which a user acquires information such as images, audio, and text using a smartphone or other device.

[1492] "Means for inputting information into an application and transmitting it to a server" refers to means for inputting acquired information into an application, encrypting it, and transmitting it securely to a server.

[1493] "Means for analyzing information using neural networks" refers to means for analyzing information obtained on a server using deep learning technology and evaluating its authenticity.

[1494] The "means for organizing the analysis results and returning them to the terminal" refers to a means for converting the results analyzed by the server into a format that is easy for the user to understand and returning them to the terminal for notification.

[1495] "Means for notifying the user of the organized results" refers to means for informing the user of the analysis results using a pop-up notification or message display on the terminal.

[1496] "Means having an emotion engine that analyzes the user's facial expressions and voice" refers to means that uses the user's camera and microphone to analyze the user's facial expressions and voice in real time and recognize the user's emotional state.

[1497] The "means for adjusting the notification content based on the analysis results of the emotion engine" refers to a means for appropriately changing the notification content based on the analysis results in accordance with the emotional state of the user.

[1498] The present invention is a system that analyzes information about videos watched by users in real time, detects fake information, and notifies users appropriately based on their emotional state. This system includes the following elements:

[1499] System configuration and program processing

[1500] 1. Information acquisition means

[1501] Devices such as smartphones and tablets capture frames of the video a user is watching using their camera or screen recording capabilities.

[1502] 2. A means of inputting data into the application and sending it to the server

[1503] The captured video frames are input into a dedicated application, which then encrypts the frames and securely transmits them to the server. The communication between the client and server is secured using the SSL / TLS protocol.

[1504] 3. How to analyze information using neural networks

[1505] The server analyzes the received video frames using deep learning techniques, using OpenCV for image analysis, an NLP (natural language processing) library for text analysis, and a speech recognition library for audio analysis.

[1506] 4. A means of organizing the analysis results and returning them to the device

[1507] The server compiles the analysis results and returns them to the device in an easy-to-understand format, such as "Possibly fake information (confidence level XX%)." This result is also sent encrypted.

[1508] 5. A method for notifying users of the organized results

[1509] The analysis results received by the device are immediately communicated to the user via pop-up notifications or message displays, with the content of the notifications tailored based on the user's emotional state.

[1510] 6. Means for providing an emotion engine that analyzes the user's facial expressions and voice

[1511] The system uses the smartphone's camera and microphone to collect the user's facial expressions and voice in real time, using the emotion analysis library emotion_detection, and the engine analyzes the user's emotional state (anxiety, surprise, etc.).

[1512] 7. A way to tailor notifications based on the emotion engine analysis results

[1513] After the emotion engine determines the user's emotional state, the server will modify the notification accordingly. For example, if the user is anxious, the notification will say something like, "This information is suspicious, but it's okay. Please check for reliable information."

[1514] Specific examples

[1515] Example 1: Fake news detection and emotional response

[1516] When a user reads an article on a news site, they take a screenshot of the article and send it to the server. The server analyzes the text and images in the screenshot to detect suspicious content. Based on this result, if the sentiment analysis engine detects an anxious expression on the user's face, a notification will be displayed saying, "This news article may be fake (90% confidence). Don't worry, there is other reliable information available."

[1517] Example 2: Deepfake video detection and emotion response

[1518] To verify the authenticity of a video sent by a friend, the user uses the application to send the video to a server. The server analyzes the video frames and audio to detect deepfake characteristics. After analyzing the user's surprised tone of voice, a notification appears saying, "This video may be a deepfake (85% confidence). It's okay, but we recommend further verification."

[1519] Prompt Sentence Examples

[1520] Analyze the video frames being watched to detect fake information. Extract text and objects in the video frames, and comprehensively evaluate all information to determine whether it is fake. Analyze the user's facial expressions and voice, and generate appropriate notifications based on the results.

[1521] By detecting fake information and providing flexible notifications that adapt to the user's emotions, the present invention creates an environment where users can safely confirm the reliability of information.

[1522] The above configuration enables the rapid and accurate detection of fake information and notifications that reduce the psychological burden on users.

[1523] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1524] Step 1:

[1525] The video being watched by the user is captured using the camera or screen recording function of the smartphone or tablet device. When the user presses a specific button, the application captures and temporarily saves video frames. The input is the video being watched, and the output is the captured video frames.

[1526] Step 2:

[1527] The terminal application inputs the captured video frames, encrypts them, and sends them to the server. The application uses the SSL / TLS protocol to ensure secure data transmission. The input is the captured video frames, and the output is the encrypted video frame data.

[1528] Step 3:

[1529] The server decrypts the received encrypted data and analyzes the video frames. A generative AI model is used for the analysis, and image analysis is performed using OpenCV. The input is the encrypted video frame data, and the output is the analysis result (e.g., "Possible fake information").

[1530] Step 4:

[1531] The server organizes the analysis results and converts them into a user-friendly format. At this time, detailed information is added according to the reliability and type of the analysis results (image, audio, text). The input is the analysis results, and the output is the organized analysis results (e.g., "Possibly fake information (reliability 90%)").

[1532] Step 5:

[1533] The server again encrypts the compiled analysis results and sends them back to the terminal, again using the SSL / TLS protocol to ensure secure data transmission. The input is the compiled analysis results, and the output is the encrypted analysis result data.

[1534] Step 6:

[1535] The device decrypts the encrypted data received and notifies the user. Notifications are made using pop-ups or messages. The input is the encrypted analysis result data, and the output is a notification message for the user.

[1536] Step 7:

[1537] The emotion engine collects the user's facial expression and voice data and performs real-time emotion analysis. The emotion_detection library is used for emotion analysis. The input is the user's facial expression and voice data, and the output is the emotion analysis result (e.g., "anxious state").

[1538] Step 8:

[1539] The server adjusts the notification content based on the analysis results of the emotion engine. For example, if the user is in an anxious state, it generates a gentle notification message. The input is the emotion analysis result and the analyzed fake information, and the output is the adjusted notification message.

[1540] Step 9:

[1541] The device displays the final notification message to the user, allowing the user to confirm the reliability of the information in a way that takes into account their emotional state. The input is the adjusted notification message, and the output is the final notification to the user.

[1542] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1543] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1544] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1545] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1546] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1547] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1548] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1549] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1550] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1551] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1552] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1553] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1554] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1555] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1556] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1557] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1558] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1559] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1560] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1561] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1562] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1563] The following is further disclosed regarding the above embodiment.

[1564] (Claim 1)

[1565] An information acquisition means;

[1566] A means for inputting the acquired information into an application and transmitting it to a server;

[1567] a means for the server to analyze the information using a neural network;

[1568] A means for the server to organize the analysis results and return them to the terminal;

[1569] A means for notifying the user of the results of the device organization;

[1570] A system including:

[1571] (Claim 2)

[1572] The system according to claim 1, characterized in that it evaluates the reliability of information in real time and notifies the user immediately.

[1573] (Claim 3)

[1574] The system of claim 1 is provided with a means for analyzing each element of image, audio, and text depending on the type of information, and for identifying fake information in a multi-layered manner.

[1575] "Example 1"

[1576] (Claim 1)

[1577] An information acquisition means;

[1578] A means for inputting the acquired information into an application, encrypting it, and transmitting it to a server;

[1579] a means for the server to receive the encrypted information, decrypt it, and analyze it using a neural network;

[1580] The server organizes the analysis results, re-encrypts them, and sends them back to the device.

[1581] A means for the terminal to receive, decrypt, and notify the user of the organized result;

[1582] A system including:

[1583] (Claim 2)

[1584] The system according to claim 1, characterized in that it evaluates the reliability of information in real time and notifies the user immediately.

[1585] (Claim 3)

[1586] The system of claim 1 is provided with a means for analyzing each element of image, audio, and text depending on the type of information, and for identifying fake information in a multi-layered manner.

[1587] "Application Example 1"

[1588] (Claim 1)

[1589] An information acquisition means;

[1590] A means for inputting the acquired information into an application and transmitting it to a server;

[1591] a means for the server to analyze the information using a neural network;

[1592] A means for the server to organize the analysis results and return them to the terminal;

[1593] A means for notifying the user of the results of the device organization;

[1594] A means for obtaining information about links and attachments in electronic messages in real time;

[1595] A method for determining the authenticity of links and attachments using neural networks,

[1596] a means for notifying the user, including reliability information;

[1597] A system including:

[1598] (Claim 2)

[1599] The system according to claim 1, characterized in that it evaluates the reliability of information in real time, immediately analyzes links and attachments in emails and messages, and notifies the user.

[1600] (Claim 3)

[1601] The system of claim 1 is equipped with a means for analyzing image, audio, and text elements depending on the type of information, identifying fake information at multiple levels, and assessing the reliability of links and attachments in electronic messages.

[1602] "Example 2: Combining Emotion Engines"

[1603] (Claim 1)

[1604] An information acquisition means;

[1605] A means for inputting the acquired information into an application and transmitting it to a server;

[1606] a means for the server to analyze the information using a neural network;

[1607] A means for the device to use an emotion engine that analyzes the user's facial expressions and voice data;

[1608] A means for the server to integrate the emotion data and analysis results and adjust the notification content;

[1609] A means for the server to return the sorting results to the terminal;

[1610] A means for notifying the user of the results of the device organization;

[1611] A system including:

[1612] (Claim 2)

[1613] The system according to claim 1, characterized in that it evaluates the reliability of information in real time and provides immediate notification taking into account the user's emotional state.

[1614] (Claim 3)

[1615] The system of claim 1 includes an emotion engine that analyzes image, audio, and text elements according to the type of information, identifies fake information at multiple levels, and analyzes the user's emotional state.

[1616] "Application example 2 when combining emotion engines"

[1617] (Claim 1)

[1618] An information acquisition means;

[1619] A means for inputting the acquired information into an application and transmitting it to a server;

[1620] a means for the server to analyze the information using a neural network;

[1621] A means for the server to organize the analysis results and return them to the terminal;

[1622] A means for notifying the user of the results of the device organization;

[1623] means for providing an emotion engine for analyzing a user's facial expressions and voice;

[1624] a means for adjusting the notification content based on the analysis result of the emotion engine;

[1625] A system including:

[1626] (Claim 2)

[1627] The system according to claim 1, characterized in that it evaluates the reliability of information in real time and notifies the user immediately according to the user's emotional state.

[1628] (Claim 3)

[1629] The system of claim 1 includes a means for analyzing each element of image, audio, and text depending on the type of information, identifying fake information in a multi-layered manner, and further integrating emotional data to generate notification content. [Explanation of symbols]

[1630] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. An information acquisition means; A means for inputting the acquired information into an application and transmitting it to a server; a means for the server to analyze the information using a neural network; A means for the server to organize the analysis results and return them to the terminal; A means for notifying the user of the results of the device organization; A system including:

2. 2. The system according to claim 1, wherein the reliability of information is evaluated in real time and the user is notified immediately.

3. The system according to claim 1, further comprising means for analyzing each element of image, sound, and text according to the type of information, and identifying fake information in a multi-layered manner.

Images